develooper Front page | perl.cpan.workers | Postings from May 2015

Test-Simple status update

From:
Chad Granum
Date:
May 4, 2015 05:38
Subject:
Test-Simple status update
Message ID:
CAJFr3kuKdm4XKfR4pfxybzMK_LUAPaH7v51=MmrR4RdpkaudaA@mail.gmail.com
I don't think anyone ever regretted taking some extra time to test and
verify something critical before deploying it.

Thanks to the QA Hackathon I have gotten a lot of extra eyes and review to
look at what I have been working on. Many of you have probably noticed a
recent upswing in code churn and discussion related to this. Because of
this I have decided that more time is needed before the new stuff is
deployed as stable. I do not expect many people to complain about taking
some extra time to make sure things are right.

I am not abandoning the plans, efforts, or decisions reached at the QAH. I
am however extending the timeline, and will insert an extra step between
now and stable-release.

The main issue here is that the QAH focused mainly on backwards
compatibility and testing that nothing is broken. This happened because it
is BY FAR the most critical thing. The QAH discussion ended with a
punch-list of things that needed to be done before we can replace the
internals of Test-Simple. I will stick with this punch list, and before
anything goes stable it will be done, with rjbs, xdg, and the others
involved to verify it. However this will happen a bit later than we
expected.

What new eyes, and attempts to use the new stuff (Test-Stream) have
revealed is that there are still some design elements in the new API that
need work. These are mainly in things that are completely new, and do not
have backwards compatibility to drive them. Once there is a stable release
these become hard to change. The fact that I have found the need to make 2
significant API changes in the weeks since the hackathon tells me that it
would be irresponsible to release it right now. Others have also expressed
concern after seeing the recent churn.

The recent discoveries and improvements have largely been driven by the new
interest and usage of Test-Stream. If usage tapers off then the pipeline
likely will as well. Because of this I do feel there is a need to make
Test-Stream, the new stuff, available as an experimental and independent
release fairly soon. This dist will be available for early adopters and
interested parties to play with and provide feedback. Existing tools,
specially toolchain level ones, should NOT switch whole-sale to Test-Stream
while it is in this experimental phase.

I am also trying to get a handful of people willing to commit to helping me
with review, I don't like working in a box, and I suspect others don't
either.

So to be clear:
 * Not forking the ecosystem, the Test-More/Builder/Simple internals will
be changed, but not yet
 * Test-Stream will have an experimental phase where it will be released to
cpan for people to play with
 * The punch-list, and the QAH discussion are still key to any future
stable release
 * I am just taking a bit more time to make sure I don't lock people into
another insufficient system.

-Chad 'Exodist' Granum



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About