IPC-System-Simple: unexplained test failures prevent good analysisof CPAN River 3000

Yesterday I ran my "test the CPAN River top 3000 against the
monthly dev release" program for release perl-5.29.8.  I
have not yet posted the results because of one anomaly for
which I request comments from this list.

Background:  the program builds and installs the monthly dev
release on FreeBSD-11, installs cpanm against that perl,
then uses cpanm to try to install a list of 3000 CPAN
distributions sorted in CPAN river order.  A build-time or
test-time failure in one distribution means that its reverse
dependencies (and their revdeps, etc.) are not reached
during the testing process, so their current PASS/FAIL
status is not assessable.

Current problem:  Test failure in IPC-System-Simple.

IPC::System::Simple sits high on the CPAN river -- number
91, in my version of the river.  Its immediate revdeps
include DateTime::Locale and DateTime::TimeZone, which mean
that the entire sub-universe of DateTime, Dist::Zilla, etc.
are not reached during testing unless IPC::System::Simple
receives a PASS.

When cpanm reached IPC-System-Simple in yesterday's run, the
distro's test suite FAILed in one file:

#####
t/06_fail.t ............... ok

#   Failed test 'Sanity - ENV vars are tainted'
#   at t/07_taint.t line 21.

#   Failed test 'Sanity - Evil zero is tainted'
#   at t/07_taint.t line 25.

#   Failed test 'Single-arg, tainted data'
#   at t/07_taint.t line 41.
#                   ''
#     doesn't match '(?^:called with tainted argument)'

#   Failed test 'multi-arg, tainted data'
#   at t/07_taint.t line 44.
#                   ''
#     doesn't match '(?^:called with tainted argument)'
# Looks like you failed 4 tests of 13.
t/07_taint.t ..............
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/13 subtests
#####

I am attaching t/07_taint.t to this message.

I should note that neither this particular test nor the
distro's test suite as a whole has ever FAILed for me in
several dozen runs of this program or its predecessors over
the last 16 months.

Since IPC-System-Simple FAILed, it was not installed, which
means that all of its revdeps were not reached during
testing.  The total of such revdeps is close to 238.

After I discovered this result, I went to the
.cpanm/latest-build directory and ran the distro's test
suite manually, in several ways, each time getting a PASS.

#####
$ cd 
/home/jkeenan/var/tad/testing/perl-5.29.8/.cpanm/latest-build/IPC-System-Simple-1.25

$ PERL_DL_NONLAZY=1 
"/usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/perl" 
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef 
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
...
All tests successful.
Files=19, Tests=130,  3 wallclock secs ( 0.05 usr  0.07 sys +  1.52 cusr 
  1.02 csys =  2.65 CPU)
Result: PASS
#####

# Also:

#####
$ /usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/prove 
-I/usr/home/jkeenan/var/tad/testing/perl-5.29.8/lib -b t/*.t
t/01_load.t ............... ok
...
t/07_taint.t .............. ok
...
t/win32.t ................. skipped: Win32 only tests
All tests successful.
Files=19, Tests=130,  2 wallclock secs ( 0.07 usr  0.05 sys +  1.51 cusr 
  0.97 csys =  2.59 CPU)
Result: PASS
#####

Can anyone identify a reason why, in one particular run of this distro's 
test
suite, one test file FAIL while PASSing in all other runs?

Thank you very much.
Jim Keenan

• IPC-System-Simple: unexplained test failures prevent good analysisof CPAN River 3000 by James E Keenan
• Re: IPC-System-Simple: unexplained test failures prevent goodanalysis of CPAN River 3000 by Slaven Rezic
• Re: IPC-System-Simple: unexplained test failures preventgoodanalysis of CPAN River 3000 by James E Keenan