Front page | perl.perl5.porters |
Postings from June 2001
Re: on testing
Thread Previous
|
Thread Next
From:
John Peacock
Date:
June 4, 2001 10:44
Subject:
Re: on testing
Message ID:
3B1BC6FF.A2801D35@rowman.com
schwern@pobox.com wrote:
>
> Yes, this is fine for t/lib, but more basic things will cause
> problems. Consider what would happen if use() or import() broke.
Yes, deep core still requires manual intervention. However, once we
test use() and import(), everything after that point can use() them
(pun intended :~) with impunity.
> This assumes the tests have no side-effects. A dubious assumption at
> best.
I was supposing that any single test would be inside an eval() to limit
the bleed (unless that was what we were testing for, hmmm). Of course,
we have to add eval() to the list of things which must be tested
outside this framework (but we knew that already).
> It would be better to simply make your test scripts smaller and
> shorter so its not such a burden to rerun the whole thing.
>
> t/TEST supports subdirectories now, so there's no reason you couldn't
> have, for example... t/op/numconvert/foo.t t/op/numconvert/bar.t
This is a good point and the subdir feature I was not aware of. This
may change my answer. ;~) I may rewrite bigintpm.t to use this mode
just to see how that looks.
However, we need to make sure that we don't mess with the VMS people
too much by having limitless directory recursion in the testsuite.
>
> > > Instead, I'm planning on having Test::Harness simply dump the verbose
> > > output to a file on each test run. test.out or something. That way
> > > you always have the verbose output to look at if you're interested.
> > > Sort of brute force, but covers all the bases without extra work for
> > > the test author.
> >
> > Good, but not far enough.
>
> Not far enough how?
Because a default TEST_VERBOSE output contains a varying amount of
useful
data. If I have a single test failure out of 400, the dump file
contains
399 lines of "ok" noise and 1 error. My screen doesn't scroll that far
back!
If ok() were extracted out of the individual scripts, then the default
behavior could be that all "not ok" messages would always go to STDERR
as verbose messages (as well as to STDOUT to be counted and tabulated).
Then the TEST or Test::Harness framework could dump STDERR to file
(or even feed it to perlbug, should we so decide).
I know there are existing tests which will not lend themselves to this;
however, there is a lot of wheel reinvention going on in the existing
testsuite. Going forward, I think it would be a good thing to try and
provide a consistent test framework; refitting the existing tests to
use the new mode is dependent on the finite number of tuits in the
world. ;~)
John
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4720 Boston Way
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5747
Thread Previous
|
Thread Next