Re: [perl #119619] Contradictory advice and test on using Test::More

James E Keenan via RT writes:

> On Thu Sep 05 04:44:55 2013, smylers@stripey.com wrote:
> 
> > perlhack, in its TESTING section, says that tests other than in
> > specific subdirectories should use Test::More (rather than
> > t/test.pl).
> > 
> > But t/porting/test_bootstrap.t specifically tests that no t/**.t
> > files use Test::More — adding the line:
> > 
> >   use Test::More;
> > 
> > into a test in t/porting/ (not one of the directories perlhack
> > mentions as requiring ad-hoc testing or test.pl) makes
> > test_bootstrap.t fail.

Note that t/porting/ is simply an example which demonstrates the
contradiction; the problem isn't restricted to t/porting. My apologies
if that wasn't clear in my bug report.

Here's the relevant text from perlhack:

  ·   t/base, t/comp and t/opbasic

      ... use ad hoc tests for these three ...
      
  ·   t/cmd, t/run, t/io and t/op

      Now that basic require() and subroutines are tested, you can use
      the t/test.pl library. ...

  ·   Everything else

      Now that the core of Perl is tested, Test::More can and should be
      used. ...

And here are t/ subdirectories which include .t files but aren't
mentioned in either of the first two bullet points above:

  lib x2p benchmark mro bigmem porting uni japh re test_pl win32


> > These obviously contradict each other, and I don't know what should
> > be used for a new test in t/porting/.
> > 
> > Either test_bootstrap.t needs restricting to the directories
> > mentioned in perlhack, or the advice in perlhack needs changing to
> > say never to use Test::More.
> 
> I can confirm that adding or editing a file in t/porting which
> contains 'use Test::More' will cause t/porting/test_bootstrap.t to
> fail.
> 
> Would the patch attached suffice?

That patch simply moves t/porting into the second bullet point's list,
which obviously doesn't resolve the contradiction in general.

As far as I can tell, test_bootstrap.t prevents any test in t/ from
using Test::More, and as such the ‘everything else’ bullet point in
perlhack can't ever apply.

So making the documentation consistent with test_bootstrap.t would
require entirely removing that third bullet point which mentions
Test::More, and renaming the second one to say ‘everything else’ (that
is, everything but the three files mentioned in the first bullet point).

But it isn't clear to me that that's the right thing to do. The author
of that advice intended that Test::More could be used in some
circumstances, and the reasoning sound plausible to me. As such, it may
be that adjusting test_bootstrap.t to match the advice (only prohibiting
Test::More selectively) would be better.

So before any solution can be effected, somebody needs to make a
judgement call about what's desired here.

Cheers

Smylers
-- 
Stop drug companies hiding negative research results.
Sign the AllTrials petition to get all clinical research results published.
Read more: http://www.alltrials.net/blog/the-alltrials-campaign/

• Re: [perl #119619] Contradictory advice and test on using Test::More by Smylers
• [perl #119619] Contradictory advice and test on using Test::More by James E Keenan via RT
• [perl #119619] Contradictory advice and test on using Test::More by Smylers