develooper Front page | perl.perl5.porters | Postings from July 2018

CPAN river 3000: status as of perl-5.29.1 release

From:
James E Keenan
Date:
July 21, 2018 16:16
Subject:
CPAN river 3000: status as of perl-5.29.1 release
Message ID:
8a77b54b-30c6-09a2-00ba-d92c695ef196@pobox.com
You can find data from a test run of the "CPAN-river-3000" against
all releases in the current (perl-5.29) development cycle here:

#####
http://thenceforward.net/perl/misc/cpan-river-3000.perl-5.29.master.psv.gz
#####

In addition, please find attached a file in CSV format holding data
about those CPAN distributions which received a grade other than
'PASS' during the most recent monthly run or whose grade changed
between the two most recent runs.

#####
changes-5.29.0-to-5.29.1.csv
#####

I. General Comments

The data in these files are intended to be used in the assessment of the
impact which Perl 5 development is having on the installability of an
important set of CPAN distributions.  These are not the only data which
can be used or which ought to be used in that assessment.

The following items should be noted:

1. The number of CPAN modules selected for attempted build against
development releases has been increased from 1000 (in the 5.27 cycle)
to 3000 (in the 5.29 cycle).  We hope that this will provide a more
thorough basis for an assessment of the impact of blead on CPAN.

2. The analysis is being conducted on FreeBSD rather than Linux.

#####
$ uname -mrs
FreeBSD 11.1-RELEASE amd64
$ clang --version
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on 
LLVM 4.0.0)
Target: x86_64-unknown-freebsd11.1
Thread model: posix
InstalledDir: /usr/bin
######

By doing so, we hope to avoid a monoculture of operating systems and
call attention to the fact that the impact of blead on CPAN can differ
across platforms.

3. The list of modules selected is curated so that the automated
test-against-dev program can run smoothly over its more-than-six-hours
length.  That means excluding modules that require interactive
configuration, modules that make excessive demands with respect to
external prerequisites, modules that require too much network access,
modules that are subject to timeouts, and so forth.

4. On the other hand, the list of modules selected has not been
filtered to exclude modules which, nothwithstanding the fact that
other CPAN distributions depend on them, have been failing tests for
years.

5. The test-against-dev process is built on top of Miyagawa's 'cpanm'
and Breno G. de Oliveira's CPAN library App::cpanminus::reporter.  It
therefore benefits from the strengths, and suffers from the flaws, of
those two excellent libraries.  In particular, if distribution XYZ has
a dependency on distribution ABC and if, earlier in the process, ABC
has failed, the test-against-dev process will steer clear of
attempting to fully test XYZ and will fail to write a report for it.
So some modules will "not be reached" and will not show up in the
output data until such time as their upstream dependencies are fixed.

6. There are certain CPAN distributions whose prerequisites I have not
yet figured out how to install on the platform being used.  Those
distributions are likely to have a grade of UNKNOWN and their reverse
dependencies will not be reached.  The most important case of this:
Tk.

If you would like further information about this project or to help it
along, please contact me off-list.

II. Specific Comments

1. The data reported for the perl-5.29.0 release in the master PSV file
differs slightly from that reported on June 28
(https://www.nntp.perl.org/group/perl.perl5.porters/2018/06/msg251309.html).
My colleague Andrew Villano figured out how to grow the disk size
inside a Vagrant box, so we created a new virtual machine and re-did
the perl-5.29.0 run therein.  The state of CPAN of course changed in
that three-week timespan, so some distros which FAILed on perl-5.29.0
then PASSed on perl-5.29.0 earlier this week -- and vice versa.

2. If a distribution PASSed when run against perl-5.29.0 earlier this
week and then received a grade other than PASS when run against
perl-5.29.1 today, then none of its reverse dependencies would have
been reached in the perl-5.29.1 process run today.  Those reverse
dependencies have therefore been graded 'x' for 'perl-5.29.1-grade'
even if they got a 'PASS' for 'perl-5.29.0-grade'.

Of particular concern in this regard is CPAN distribution
Text-SimpleTable.  Catalyst-Runtime and all of its reverse
dependencies are dependent upon Text-SimpleTable, but this
distribution has failures which have not yet been addressed by its
author.  (https://rt.cpan.org/Ticket/Display.html?id=125657)
Consequently the current impact of blead on the Catalyst ecosystem
cannot be assessed by this process (though other assessment processes
which, for example, might force an installation of Text-SimpleTable
would be capable of generating data).

III.  Acknowledgements
this month
This project benefited from discussion at the Perl 5 Core Hackathon
held in Amsterdam in October 2017, so thanks to participants in that
event and to the organizations which sponsored it.

This project also benefits from the generous donation of a server by
the New York City BSD Users Group and the system administration
expertise of Mark Saad and George Rosamond.

This project also benefitted from assistance in reviewing test failure
reports by three members of Philadelphia Perlmongers:  John Karr, Walt
Mankowski and Thomas McKernan.

This project also benefits from the system administration expertise of
Andrew Villano of New York Perlmongers.

IV. References

See my presentation at the north American Perl Conference held in Salt
Lake City in June 2018:
http://thenceforward.net/perl/tpc/TPC-NA-2018/test-against-dev.pdf

Thank you very much.
Jim Keenan



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About