develooper Front page | perl.qa | Postings from March 2009

Re: Counting tests

Thread Previous | Thread Next
From:
Michael G Schwern
Date:
March 13, 2009 22:58
Subject:
Re: Counting tests
Message ID:
49BB4753.2060700@pobox.com
Let's sum up.

The "why can't a program count its own tests" page refers to the problem of
counting the tests *without* running the code.

`use Test::More "no_plan";` is the most used way to run a test without having
to hard code the number of tests beforehand.

The test numbering exists to ensure that all your tests run, and in the right
order.  XUnit frameworks don't need to know the number of tests because they
simply don't have this type of protection. [1]

`use Test::Most "deferred_plan";` is a safer way to do that.  It ensures your
test runs to completion and it has the option of taking a number of tests.
Sometimes you can calculate the number as you go.

Test::More is adding this feature, but there's nothing wrong with Test::Most.
 Unlike other testing systems, Perl does not use a single "test framework"
like JUnit or whatever.  Most Test:: modules on CPAN will work together. [2]
You can mix and match.  In this sense it kicks the crap out of everyone else. :)

Test::More and Test::Harness will see if your test dies or segfaults.
Test::More will see a normal die and Test::Harness will fail any test script
with a non-zero exit code.

A human should never be necessary to determine if a test passed or failed.
Humans are bad at rote tasks and reading huge wads of output.  They will
eventually tire of the task or simply miss a failure.  Also the next human
will not know what to look for.  And it kills test automation.  You may wish
to look at http://testers.cpan.org/ to understand the scale of test automation
we're talking about.

I have no idea what Cucumber is, but I have no doubt it can be implemented as
a Perl testing module to work with everything else.  It looks like a clever
FIT framework.  For the moment using Test::More will work, but eventually
you'll want to switch to Test::Builder.


[1] Perl has this because Larry needed it 21 years ago when he came up with
all this, not necessarily because it's important.

[2] Through the magic of the Test Anything Protocol and Test::Builder.  I
happen to have just given a talk about how this all works.
http://schwern.org/talks/TB2.pdf
(Ignore the title, it's not actually about Test::Builder2)
The audio from the meeting should show up here shortly.
http://pdxpm.podasp.com/archive.html


-- 
40. I do not have super-powers.
    -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
           http://skippyslist.com/list/

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About