develooper Front page | perl.perl5.porters | Postings from February 2019

Re: IPC-System-Simple: unexplained test failures prevent goodanalysis of CPAN River 3000

Thread Previous | Thread Next
From:
Slaven Rezic
Date:
February 22, 2019 23:53
Subject:
Re: IPC-System-Simple: unexplained test failures prevent goodanalysis of CPAN River 3000
Message ID:
87d0nj9zl5.fsf@cvrsnica.herceg.de
James E Keenan <jkeenan@pobox.com> writes:

> Yesterday I ran my "test the CPAN River top 3000 against the
> monthly dev release" program for release perl-5.29.8.  I
> have not yet posted the results because of one anomaly for
> which I request comments from this list.
>
> Background:  the program builds and installs the monthly dev
> release on FreeBSD-11, installs cpanm against that perl,
> then uses cpanm to try to install a list of 3000 CPAN
> distributions sorted in CPAN river order.  A build-time or
> test-time failure in one distribution means that its reverse
> dependencies (and their revdeps, etc.) are not reached
> during the testing process, so their current PASS/FAIL
> status is not assessable.
>
> Current problem:  Test failure in IPC-System-Simple.
>
> IPC::System::Simple sits high on the CPAN river -- number
> 91, in my version of the river.  Its immediate revdeps
> include DateTime::Locale and DateTime::TimeZone, which mean
> that the entire sub-universe of DateTime, Dist::Zilla, etc.
> are not reached during testing unless IPC::System::Simple
> receives a PASS.
>
> When cpanm reached IPC-System-Simple in yesterday's run, the
> distro's test suite FAILed in one file:
>
> #####
> t/06_fail.t ............... ok
>
> #   Failed test 'Sanity - ENV vars are tainted'
> #   at t/07_taint.t line 21.
>
> #   Failed test 'Sanity - Evil zero is tainted'
> #   at t/07_taint.t line 25.
>
> #   Failed test 'Single-arg, tainted data'
> #   at t/07_taint.t line 41.
> #                   ''
> #     doesn't match '(?^:called with tainted argument)'
>
> #   Failed test 'multi-arg, tainted data'
> #   at t/07_taint.t line 44.
> #                   ''
> #     doesn't match '(?^:called with tainted argument)'
> # Looks like you failed 4 tests of 13.
> t/07_taint.t ..............
> Dubious, test returned 4 (wstat 1024, 0x400)
> Failed 4/13 subtests
> #####
>
> I am attaching t/07_taint.t to this message.
>
> I should note that neither this particular test nor the
> distro's test suite as a whole has ever FAILed for me in
> several dozen runs of this program or its predecessors over
> the last 16 months.
>
> Since IPC-System-Simple FAILed, it was not installed, which
> means that all of its revdeps were not reached during
> testing.  The total of such revdeps is close to 238.
>
> After I discovered this result, I went to the
> .cpanm/latest-build directory and ran the distro's test
> suite manually, in several ways, each time getting a PASS.
>
> #####
> $ cd
> /home/jkeenan/var/tad/testing/perl-5.29.8/.cpanm/latest-build/IPC-System-Simple-1.25
>
> $ PERL_DL_NONLAZY=1
> "/usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/perl"
> "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef
> *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')"
> t/*.t
> ...
> All tests successful.
> Files=19, Tests=130,  3 wallclock secs ( 0.05 usr  0.07 sys +  1.52
> cusr 1.02 csys =  2.65 CPU)
> Result: PASS
> #####
>
> # Also:
>
> #####
> $ /usr/home/jkeenan/var/tad/testing/perl-5.29.8/bin/prove
> -I/usr/home/jkeenan/var/tad/testing/perl-5.29.8/lib -b t/*.t
> t/01_load.t ............... ok
> ...
> t/07_taint.t .............. ok
> ...
> t/win32.t ................. skipped: Win32 only tests
> All tests successful.
> Files=19, Tests=130,  2 wallclock secs ( 0.07 usr  0.05 sys +  1.51
> cusr 0.97 csys =  2.59 CPU)
> Result: PASS
> #####
>
> Can anyone identify a reason why, in one particular run of this
> distro's test
> suite, one test file FAIL while PASSing in all other runs?
>

This is a known problem, see
https://github.com/pjf/ipc-system-simple/issues/21

Regards,
    Slaven

-- 
Slaven Rezic - slaven <at> rezic <dot> de
  BBBike - route planner for cyclists in Berlin
  WWW version:                           http://www.bbbike.de
  Perl/Tk version for Unix and Windows:  http://bbbike.sourceforge.net

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About