develooper Front page | perl.perl5.porters | Postings from February 2019

Re: IPC-System-Simple: unexplained test failures preventgoodanalysis of CPAN River 3000

Thread Previous
From:
James E Keenan
Date:
February 23, 2019 03:40
Subject:
Re: IPC-System-Simple: unexplained test failures preventgoodanalysis of CPAN River 3000
Message ID:
20190223034007.13155.qmail@lists-nntp.develooper.com
On 2/22/19 6:24 PM, Slaven Rezic wrote:
> James E Keenan <jkeenan@pobox.com> writes:
> 
>> Yesterday I ran my "test the CPAN River top 3000 against the
>> monthly dev release" program for release perl-5.29.8.  I
>> have not yet posted the results because of one anomaly for
>> which I request comments from this list.
>>
>> Background:  the program builds and installs the monthly dev
>> release on FreeBSD-11, installs cpanm against that perl,
>> then uses cpanm to try to install a list of 3000 CPAN
>> distributions sorted in CPAN river order.  A build-time or
>> test-time failure in one distribution means that its reverse
>> dependencies (and their revdeps, etc.) are not reached
>> during the testing process, so their current PASS/FAIL
>> status is not assessable.
>>
>> Current problem:  Test failure in IPC-System-Simple.
>>
>> IPC::System::Simple sits high on the CPAN river -- number
>> 91, in my version of the river.  Its immediate revdeps
>> include DateTime::Locale and DateTime::TimeZone, which mean
>> that the entire sub-universe of DateTime, Dist::Zilla, etc.
>> are not reached during testing unless IPC::System::Simple
>> receives a PASS.
>>
>> When cpanm reached IPC-System-Simple in yesterday's run, the
>> distro's test suite FAILed in one file:
>>
>> #####
>> t/06_fail.t ............... ok
>>
>> #   Failed test 'Sanity - ENV vars are tainted'
>> #   at t/07_taint.t line 21.
>>
>> #   Failed test 'Sanity - Evil zero is tainted'
>> #   at t/07_taint.t line 25.
>>
>> #   Failed test 'Single-arg, tainted data'
>> #   at t/07_taint.t line 41.
>> #                   ''
>> #     doesn't match '(?^:called with tainted argument)'
>>
>> #   Failed test 'multi-arg, tainted data'
>> #   at t/07_taint.t line 44.
>> #                   ''
>> #     doesn't match '(?^:called with tainted argument)'
>> # Looks like you failed 4 tests of 13.
>> t/07_taint.t ..............
>> Dubious, test returned 4 (wstat 1024, 0x400)
>> Failed 4/13 subtests
>> #####
>>
>> I am attaching t/07_taint.t to this message.
>>
>> I should note that neither this particular test nor the
>> distro's test suite as a whole has ever FAILed for me in
>> several dozen runs of this program or its predecessors over
>> the last 16 months.
>>
>> Since IPC-System-Simple FAILed, it was not installed, which
>> means that all of its revdeps were not reached during
>> testing.  The total of such revdeps is close to 238.
>>
>> After I discovered this result, I went to the
>> .cpanm/latest-build directory and ran the distro's test
>> suite manually, in several ways, each time getting a PASS.
>>
>> #####
>> $ cd
>> /home/jkeenan/var/tad/testing/perl-5.29.8/.cpanm/latest-build/IPC-System-Simple-1.25
>>
>> $ PERL_DL_NONLAZY=1
>> "/usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/perl"
>> "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
>> *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')"
>> t/*.t
>> ...
>> All tests successful.
>> Files=19, Tests=130,  3 wallclock secs ( 0.05 usr  0.07 sys +  1.52
>> cusr 1.02 csys =  2.65 CPU)
>> Result: PASS
>> #####
>>
>> # Also:
>>
>> #####
>> $ /usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/prove
>> -I/usr/home/jkeenan/var/tad/testing/perl-5.29.8/lib -b t/*.t
>> t/01_load.t ............... ok
>> ...
>> t/07_taint.t .............. ok
>> ...
>> t/win32.t ................. skipped: Win32 only tests
>> All tests successful.
>> Files=19, Tests=130,  2 wallclock secs ( 0.07 usr  0.05 sys +  1.51
>> cusr 0.97 csys =  2.59 CPU)
>> Result: PASS
>> #####
>>
>> Can anyone identify a reason why, in one particular run of this
>> distro's test
>> suite, one test file FAIL while PASSing in all other runs?
>>
> 
> This is a known problem, see
> https://github.com/pjf/ipc-system-simple/issues/21
> 

And I see from that issue that one of the people who already got bit by 
this was ... me! 
https://github.com/pjf/ipc-system-simple/issues/21#issuecomment-397109751

Thanks for the reminder!

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About