develooper Front page | perl.perl5.porters | Postings from August 2018

CPAN river 3000: status as of perl-5.29.2 release

From:
James E Keenan
Date:
August 21, 2018 20:27
Subject:
CPAN river 3000: status as of perl-5.29.2 release
Message ID:
20180821202741.28477.qmail@lists-nntp.develooper.com
You can find data from a test run of the "CPAN-river-3000" against
all releases in the current (perl-5.29) development cycle here:

#####
http://thenceforward.net/perl/misc/cpan-river-3000.perl-5.29.master.psv.gz
#####

In addition, please find attached a file in CSV format holding data
about those CPAN distributions which received a grade other than
'PASS' during the most recent monthly run or whose grade changed
between the two most recent runs.

#####
changes-5.29.1-to-5.29.2.csv
#####

I. General Comments

The data in these files are intended to be used in the assessment of the
impact which Perl 5 development is having on the installability of an
important set of CPAN distributions.  These are not the only data which
can be used or which ought to be used in that assessment.

The following items should be noted:

1. The number of CPAN modules selected for attempted build against
development releases has been increased from 1000 (in the 5.27 cycle)
to 3000 (in the 5.29 cycle).  We hope that this will provide a more
thorough basis for an assessment of the impact of blead on CPAN.

2. The analysis is being conducted on FreeBSD rather than Linux.

#####
$ uname -mrs
FreeBSD 11.1-RELEASE amd64
$ clang --version
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on 
LLVM 4.0.0)
Target: x86_64-unknown-freebsd11.1
Thread model: posix
InstalledDir: /usr/bin
######

By doing so, we hope to avoid a monoculture of operating systems and
call attention to the fact that the impact of blead on CPAN can differ
across platforms.

3. The list of modules selected is curated so that the automated
test-against-dev program can run smoothly over its more-than-six-hours
length.  That means excluding modules that require interactive
configuration, modules that make excessive demands with respect to
external prerequisites, modules that require too much network access,
modules that are subject to timeouts, and so forth.

4. On the other hand, the list of modules selected has not been
filtered to exclude modules which, nothwithstanding the fact that
other CPAN distributions depend on them, have been failing tests for
years.

5. The test-against-dev process is built on top of Miyagawa's 'cpanm'
and Breno G. de Oliveira's CPAN library App::cpanminus::reporter.  It
therefore benefits from the strengths, and suffers from the flaws, of
those two excellent libraries.  In particular, if distribution XYZ has
a dependency on distribution ABC and if, earlier in the process, ABC
has failed, the test-against-dev process will steer clear of
attempting to fully test XYZ and will fail to write a report for it.
So some modules will "not be reached" and will not show up in the
output data until such time as their upstream dependencies are fixed.

6. There are certain CPAN distributions whose prerequisites I have not
yet figured out how to install on the platform being used.  Those
distributions are likely to have a grade of UNKNOWN and their reverse
dependencies will not be reached.  The most important case of this:
Tk.

If you would like further information about this project or to help it
along, please contact me off-list.

II. Specific Comments

1. If a distribution PASSed when run against perl-5.29.1 last month and
then received a grade other than PASS when run against perl-5.29.2
today, then none of its reverse dependencies would have been reached in
the perl-5.29.2 process run today.  Those reverse dependencies have
therefore been graded 'x' for 'perl-5.29.2-grade' even if they got a
'PASS' for 'perl-5.29.1-grade'.

The most significant instance of this in this month's run is test
failures in Archive-Zip.  These failures reported by Slaven in
https://rt.cpan.org/Ticket/Display.html?id=126905 and confirmed by one
other contributor and myself.  Because Archive-Zip sits high up on the
CPAN river (156 reverse dependencies per
https://metacpan.org/requires/distribution/Archive-Zip?p=1&sort=%5B%5B2%2C1%5D%5D),
the impact of this failure will be broad and deep.

2. Conversely, there are distributions which were not reached in last
month's run for perl-5.29.1 and which were therefore graded 'x' last
month which were reached in today's run for perl-5.29.2 and which have
therefore been given a true grade (whether that be 'PASS', 'FAIL',
etc.).  In many cases this is due to the fact that CPAN distribution
Text-SimpleTable -- a dependency to Catalyst-Runtime and all of its reverse
dependencies -- PASSed in perl-5.29.2 after having FAILed in
perl-5.29.1.

III.  Acknowledgements

This project benefited from discussion at the Perl 5 Core Hackathon
held in Amsterdam in October 2017, so thanks to participants in that
event and to the organizations which sponsored it.

This project also benefits from the generous donation of a server by
the New York City BSD Users Group and the system administration
expertise of Mark Saad and George Rosamond.

This project also benefitted from assistance in reviewing test failure
reports by three members of Philadelphia Perlmongers:  John Karr, Walt
Mankowski and Thomas McKernan.

This project also benefits from the system administration expertise of
Andrew Villano of New York Perlmongers.

IV. References

See my presentation at the North American Perl Conference held in Salt
Lake City in June 2018:
http://thenceforward.net/perl/tpc/TPC-NA-2018/test-against-dev.pdf

Thank you very much.
Jim Keenan



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About