develooper Front page | perl.perl5.porters | Postings from January 2001

Unifying 'skip' and 'todo' test output

Thread Next
Michael G Schwern
January 17, 2001 20:20
Unifying 'skip' and 'todo' test output
Message ID:
So I stumbled on this 'todo' thing in Test::Harness and in the new
Test.  The way its formatted bothers me, mostly because its purpose is
very similar to 'skip' but used very differently.

Some background... todo is used for tests against future features of
your code and they are expected to fail.  skipped tests are those
which for one reason or another are inappopriate to run (platform
specific tests, for instance).  Either way, they're placeholders.

Here's how you declare tests 3, 4 and 5 are todos (Test::Harness
expects 3, 4 and 5 to fail).

        1..6 todo 3 4 5
        ok 1
        ok 2
        not ok 3
        not ok 4
        not ok 5
        ok 6

And here's how you declare that tests 3, 4 and 5 are to be skipped,
declaring why test #4 was skipped.

        ok 1
        ok 2
        ok 3    # skip
        ok 4    # skip Small furry animals ate my socks
        ok 5    # skip
        ok 6

You can also skip an entire test script, declaring why...

        1..0 skip Life has no meaning, why bother?

I propose a unification of the twos syntaxes and features.  I find
todo's syntax to be flawed because it relies on test numbers which
will change every time a test is inserted.  Looking at the first
example above, were we to add a test between 3 and 4, the todo now
becomes 3, 5 and 6.  Action at a distance.  By declaring the skips at
the point of the ok, the test numbers become unimportant.  It also
means that the code which declares the skip or todo is at the same
place that the test is, so you're less likely to forget to remove a
todo or a skip.

Fortunately, the todo syntax is undocumented.  Only the latest version
of uses it (abstracted, fortunately) and Test::Harness
understands it.  The flexibility is still there to modify its syntax
without effecting existing code (unless you broke abstraction,
then... tsk tsk naughty you ;).

I propose that todo's syntax become the same as skip.  You'd declare 3, 4
and 5 like this.

        ok 1
        ok 2
        not ok 3        # todo
        not ok 4        # todo I'm a lazy ass
        not ok 5        # todo
        ok 6

Declare an entire test todo like this...

        1..0 todo When pigs fly and Test::Harness would need some minor modifications, which
I'll do shortly.  It'll actually simplify the Test::Harness code a


Michael G. Schwern   <>
<Skrewtape> I've heard that semen tastes different depending on diet.  Is
            that true?
<Skrewtape> Hello?
<Schwern> Skrewtape:  Hang on, I'm conducting research.

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About