Front page | perl.perl5.porters |
Postings from February 2019
IPC-System-Simple: unexplained test failures prevent good analysisof CPAN River 3000
Thread Next
From:
James E Keenan
Date:
February 22, 2019 22:06
Subject:
IPC-System-Simple: unexplained test failures prevent good analysisof CPAN River 3000
Message ID:
20190222220645.6670.qmail@lists-nntp.develooper.com
Yesterday I ran my "test the CPAN River top 3000 against the
monthly dev release" program for release perl-5.29.8. I
have not yet posted the results because of one anomaly for
which I request comments from this list.
Background: the program builds and installs the monthly dev
release on FreeBSD-11, installs cpanm against that perl,
then uses cpanm to try to install a list of 3000 CPAN
distributions sorted in CPAN river order. A build-time or
test-time failure in one distribution means that its reverse
dependencies (and their revdeps, etc.) are not reached
during the testing process, so their current PASS/FAIL
status is not assessable.
Current problem: Test failure in IPC-System-Simple.
IPC::System::Simple sits high on the CPAN river -- number
91, in my version of the river. Its immediate revdeps
include DateTime::Locale and DateTime::TimeZone, which mean
that the entire sub-universe of DateTime, Dist::Zilla, etc.
are not reached during testing unless IPC::System::Simple
receives a PASS.
When cpanm reached IPC-System-Simple in yesterday's run, the
distro's test suite FAILed in one file:
#####
t/06_fail.t ............... ok
# Failed test 'Sanity - ENV vars are tainted'
# at t/07_taint.t line 21.
# Failed test 'Sanity - Evil zero is tainted'
# at t/07_taint.t line 25.
# Failed test 'Single-arg, tainted data'
# at t/07_taint.t line 41.
# ''
# doesn't match '(?^:called with tainted argument)'
# Failed test 'multi-arg, tainted data'
# at t/07_taint.t line 44.
# ''
# doesn't match '(?^:called with tainted argument)'
# Looks like you failed 4 tests of 13.
t/07_taint.t ..............
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/13 subtests
#####
I am attaching t/07_taint.t to this message.
I should note that neither this particular test nor the
distro's test suite as a whole has ever FAILed for me in
several dozen runs of this program or its predecessors over
the last 16 months.
Since IPC-System-Simple FAILed, it was not installed, which
means that all of its revdeps were not reached during
testing. The total of such revdeps is close to 238.
After I discovered this result, I went to the
.cpanm/latest-build directory and ran the distro's test
suite manually, in several ways, each time getting a PASS.
#####
$ cd
/home/jkeenan/var/tad/testing/perl-5.29.8/.cpanm/latest-build/IPC-System-Simple-1.25
$ PERL_DL_NONLAZY=1
"/usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/perl"
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
...
All tests successful.
Files=19, Tests=130, 3 wallclock secs ( 0.05 usr 0.07 sys + 1.52 cusr
1.02 csys = 2.65 CPU)
Result: PASS
#####
# Also:
#####
$ /usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/prove
-I/usr/home/jkeenan/var/tad/testing/perl-5.29.8/lib -b t/*.t
t/01_load.t ............... ok
...
t/07_taint.t .............. ok
...
t/win32.t ................. skipped: Win32 only tests
All tests successful.
Files=19, Tests=130, 2 wallclock secs ( 0.07 usr 0.05 sys + 1.51 cusr
0.97 csys = 2.59 CPU)
Result: PASS
#####
Can anyone identify a reason why, in one particular run of this distro's
test
suite, one test file FAIL while PASSing in all other runs?
Thank you very much.
Jim Keenan
Thread Next
-
IPC-System-Simple: unexplained test failures prevent good analysisof CPAN River 3000
by James E Keenan