Front page | perl.perl5.porters |
Postings from March 2017
Re: Shortage of Red / Yellow on CPAN Testers Matrix
Thread Previous
|
Thread Next
From:
Dan Collins
Date:
March 29, 2017 18:38
Subject:
Re: Shortage of Red / Yellow on CPAN Testers Matrix
Message ID:
CA+tt54K56ipbNf2p6F+55VZd9jAVrh-K7bAQUpUjBg-6DgYojg@mail.gmail.com
Huh, I just found another interesting thing. CPAN::Reporter makes no effort
to report on whether the `install` phase succeeded, but some particularly
creative dists manage to fail in the `install` phase under
PERL_USE_UNSAFE_INC=0. Look at this example
from GUIDO/libintl-perl-1.26.tar.gz - does anyone know how common this sort
of thing is? There aren't many using this exact name (
http://grep.cpan.me/?q=package+MyInstall).
All tests successful.
Files=164, Tests=3750, 5 wallclock secs ( 0.87 usr 0.18 sys + 8.23 cusr
0.30 csys = 9.58 CPU)
Result: PASS
(/usr/bin/make test exited with 0)
CPAN::Reporter: Test result is 'pass', 'make test' no errors.
CPAN::Reporter: preparing a CPAN Testers report for libintl-perl-1.26
CPAN::Reporter: this appears to be a duplicate report for the test phase:
PASS libintl-perl-1.26 x86_64-linux-thread-multi 3.16.0-4-amd64
Test report will not be sent.
GUIDO/libintl-perl-1.26.tar.gz
/usr/bin/make test -- OK
Running make install
make[1]: Entering directory
'/home/cpan2/.cpan/build/libintl-perl-1.26-55/gettext_xs'
"/home/cpan2/install/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' --
gettext_xs.bs ../blib/arch/auto/Locale/gettext_xs/gettext_xs.bs 644
make[1]: Leaving directory
'/home/cpan2/.cpan/build/libintl-perl-1.26-55/gettext_xs'
Manifying 30 pod documents
Manifying 29 pod documents
Manifying 31 pod documents
Manifying 29 pod documents
Manifying 30 pod documents
Can't locate MyInstall.pm in @INC (you may need to install the MyInstall
module) (@INC contains:
/home/cpan2/.cpan/build/libintl-perl-1.26-55/blib/arch
/home/cpan2/.cpan/build/libintl-perl-1.26-55/blib/lib
/home/cpan2/install/lib/perl5/site_perl/5.26.0/x86_64-linux-thread-multi
/home/cpan2/install/lib/perl5/site_perl/5.26.0
/home/cpan2/install/lib/perl5/5.26.0/x86_64-linux-thread-multi
/home/cpan2/install/lib/perl5/5.26.0).
BEGIN failed--compilation aborted.
Makefile:1389: recipe for target 'pure_site_install' failed
make: *** [pure_site_install] Error 2
GUIDO/libintl-perl-1.26.tar.gz
/usr/bin/make install -- NOT OK
Stopping: 'install' failed for 'Locale::TextDomain'.
On Wed, Mar 29, 2017 at 2:25 PM, Dan Collins <dcollinsn@gmail.com> wrote:
> Porters, I heard you wanted some red and yellow...
>
> I have tooling set up that allows me to test/install most of CPAN, saving
> the CPAN::Reporter results to file, and compare those stored files between
> two different versions of perl. I installed two instances of perl - cpan1
> and cpan2 - which are completely identical. For one of the environments, I
> set PERL_USE_UNSAFE_INC=0, in the other, I left it undefined.
>
> I'm not done yet, but I have over 5,000 dists that have been tested on
> both configurations. Of those dists, I've collected the reports that PASSED
> with PERL_USE_UNSAFE_INC=1 and FAILED or were UNKNOWN with
> PERL_USE_UNSAFE_INC=0. Those reports can be downloaded in tarball form at:
> http://dcollins.cc/p5p.html
>
> Of the discrepancies, 242 FAILED, and 237 were UNKNOWN. The former means
> they failed in tests, the latter means they failed in Build.PL or
> Makefile.PL. Of course, some particularly special dists had multiple
> problems. There are a few cases where a dist FAILed in the control: 5 where
> it PASSed in the experiment, which are probably signs of an intermittent
> test failure and which should be ignored, and 6 where it got UNKNOWN in the
> experiment, which are probably signs of a no-dot-in-inc error *in addition
> to* a test failure.
>
> I haven't uploaded these test reports yet, because I want to include a
> message in the "Tester Comments" explaining the failures that module
> authors might be able to use to fix their stuff. I'll be watching the other
> thread that Kent started to see when that resource is available
>
> Happy to answer any questions or take any suggestions...
>
> --
> Dan
>
>
> On Mon, Mar 27, 2017 at 8:42 PM, Ryan Voots <simcop2387@simcop2387.info>
> wrote:
>
>> I've gone and built a naive version of this for my own uses. It doesn't
>> yet hook into cpanm-reporter but i don't think it'd be hard to do. It's
>> intending to take a --module or --cpanfile to start from and works its way
>> through the dependency tree (fetches it all with cpanm --showdeps) and then
>> tests them with perlbrew exec --with ... so that the script can run in a
>> stable perl install while testing current blead/git easily. I'll toss it
>> up on github if there's any big call for it (otherwise it'll just live
>> publicly on my server).
>>
>> https://gitea.simcop2387.info/simcop2387/treedeps
>>
>> On Sun, Mar 26, 2017 at 8:39 PM, Kent Fredric <kentfredric@gmail.com>
>> wrote:
>>
>>> On 27 March 2017 at 16:10, Dan Collins <dcollinsn@gmail.com> wrote:
>>> > I've had a chance to make sure that this test environment isn't
>>> horribly
>>> > broken...
>>>
>>>
>>> Yeah, there's enough broken you sort of need a double-testing system.
>>>
>>> Start with =0 , and then upon hitting a failure, retry with =1 ...
>>> while paying care
>>> not to let the =1 bleed into children in the recursive step.
>>>
>>> eg:
>>>
>>> 1. Install X with UNSAFE_INC=0
>>> 2. Configure fails?
>>> 1. Report N/A Fail
>>> 2. Generate META.JSON with UNSAFE_INC=1
>>> 3. Installdeps with UNSAFE_INC=0
>>> 4. Run tests with UNSAFE_INC=0
>>> 5. Tests Fail?
>>> 1. Report FAIL
>>> 2. Re-run tests with UNSAFE_INC=1
>>> 3. Report Test result
>>>
>>> I don't know how to make my tools do this automatically so I've been
>>> doing it manually and filing bugs, to fill the gaps in automation.
>>>
>>>
>>>
>>> --
>>> Kent
>>>
>>> KENTNL - https://metacpan.org/author/KENTNL
>>>
>>
>>
>
Thread Previous
|
Thread Next