develooper Front page | perl.qa | Postings from November 2016

Re: test/subtest flow control with exceptions

Thread Previous | Thread Next
From:
Chad Granum
Date:
November 30, 2016 15:30
Subject:
Re: test/subtest flow control with exceptions
Message ID:
CAJFr3ks7rKnwV6gAFyBy=sEH+OFH3An9tEhYoFbmwMr5-+gLgw@mail.gmail.com
I don't have much comment on the functionality you want, seems reasonable
enough...

I do have implementation commentary however:

 * You should not be obtaining a context inside your subtest (specifically
line 18
https://github.com/rjbs/Test-Abortable/blob/master/lib/Test/Abortable.pm#L18).
Obtaining that context means that tests run inside your eval are likely to
report errors to the wrong files+lines. If you need a context that is
within the subtest you should obtain it after your eval.

 *
https://github.com/rjbs/Test-Abortable/blob/master/lib/Test/Abortable.pm#L30
 this should probably be $ctx->throw() which is essentially a die, but it
"aborts" the context so that it is released properly (not the die you use
prevents context->release from being called). Though on second thought,
throw() will append a file+line number to your exception, so you should
probably just add a $ctx->release right before that die. (and on further
reading your second sub asks if release should be done, the answer is yes.

-Chad







On Tue, Nov 29, 2016 at 5:52 PM, Ricardo Signes <perl.qa@rjbs.manxome.org>
wrote:

> Often, I have a test like this:
>
>   subtest "do things with an api" => sub {
>     my $result = $api_client->do_first_thing;
>
>     is(
>       $result->documents->first->title,
>       "The Best Thing",
>     );
>
>     ...
>   };
>
> Sometimes, the result comes back with zero documents.  ->first throws an
> exception and then my whole test program comes crashing down and it's
> miserable.  For a while, I've been meaning to make it possible for some
> exceptions to be recognized by my test programs as instructions to emit a
> failure, stop this subtest, and move on.
>
> It's important to note that I'm talking, in the code above, about an
> exception
> that would be thrown by ($result->documents) when ->first is called on it.
> This kind of flow control eliminates needing to write:
>
>   my $result = $api_client->do_first_thing;
>   my $docs   = $result->documents;
>   fail("no docs"), return unless $docs->has_entries;
>   my $first  = $docs->first;
>   fail("no title"), return unless $first->has_title;
>
>   is(...);
>
> In my case, I'm working with an API client that's specifically designed to
> be
> used for testing, so this kind of loose coupling between thrown exceptions
> and
> the test code is a good fit.  Not every exception should be caught this
> way.
> Truly unexpected ones should still die.
>
> So, I've written something to do this, and I'm sharing it here before going
> further with it.  In my code, if an exception is meant to be caught and
> used as
> a local abort instruction, it has a method called as_test_abort_events.
> This
> method returns a reference to an array of Test2 event descriptions.  For
> example, maybe:
>
>   sub as_test_abort_events {
>     return [
>       [ Ok   => (pass => 0, name => "no documents, but ->first called") ],
>       [ Diag => (message => "collection state: ....") ],
>     ];
>   }
>
> This method is easy to add to any exception you want, and you don't need to
> change your exception class hierarchy in any way.
>
> Next, you need something to run the tests and look for these exceptions
> being
> thrown.  I have written a library for this, called Test::Abortable.
>
>   https://github.com/rjbs/Test-Abortable
>
> Test::Abortable provides two subroutines: subtest and testeval.  subtest
> acts
> just like Test::More's subtest, but catches abort exceptions, emits their
> events, and returns normally.  (I've also updated my library
> Test::Routine, in
> a branch, to behave this way, as I use it in place of subtest for many
> things.)
>
> testeval acts like eval, but only catches abort exceptions.
>
>   ok(1);
>   testeval {
>     ok(2);
>     this_throws_an_abort;
>     ok(3);
>   };
>   ok(4);
>
> This ends up emitting ok 1, ok 2, whatever the abort wants, and ok 4.
> testeval
> returns the return value of the code block if it succeeds.  If it fails
> due to
> abort, it returns false, emits the abort events, and puts the abort in
> $@.  If
> it fails because of any other exception, the exception is re-thrown.
>
> Let me know if you have any thoughts before I begin using this in anger.
> :-)
>
> --
> rjbs
>

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About