Challenges in CPAN Releases of 'dist/' Libraries (1 of 3)

 From time to time there has been discussion on this list about the fact 
that we have been laggard in arranging for fresh CPAN releases of 
libraries which are maintained in the Perl 5 core distribution on a 
"blead-first" basis.  (See, for example, the thread starting here: 
https://www.nntp.perl.org/group/perl.perl5.porters/2021/12/msg262208.html.) 
  In this post I introduce a tool which may be of some benefit to Perl 5 
Porters in remedying this situation.  In two subsequent posts I will 
discuss the data this tool generates and make some recommendations for 
action.

(If you're not already familiar with the concept of 'dual-life' 
distributions and the distinction between 'blead-first' and 
'cpan-first', here is a brief explanation: 
http://thenceforward.net/perl/misc/blead-first-vs-cpan-first.html.)

The Presenting Problem

There are 41 libraries which are 'blead-first', i.e., maintained 
primarily by Perl 5 Porters in the 'dist/' directory of the core 
distribution which are eligible for separate CPAN release.  As of today, 
40 of those libraries have actually had CPAN releases.  Some have not 
had a separate CPAN release in more than a decade -- but if their code 
has not changed much in a decade, that's not a big problem.

Problems arise, however, when we make changes in the core distribution 
to libraries such as PathTools, Time-HiRes, Storable and others which 
have XS components.  When we make changes in these libraries, we test 
them thoroughly against the 'blead' development branch during the 
current development cycle.  So we have high confidence that they will 
work with the production release of perl that is scheduled for May of 
each year.  However, we have no systematic way of ensuring that, if we 
were to make changes in core and subsequently release those changes 
separately on CPAN, those changes would then work properly against 
*older* versions of perl.

Suppose, for instance, that we were to make changes to the PathTools 
distribution (which contains the Cwd and File::Spec modules) right now 
during the 5.35 development cycle.  Those changes would appear in the 
5.36 production release in May.  If we were to take the PathTools code 
base at that point and try to release it to CPAN, would it work against 
perl-5.34?  Against perl-5.32?  Against perl-5.8?  Whoever volunteered 
to perform the CPAN release of PathTools might find that the PathTools 
shipped with 5.36 would fail to compile or pass its tests when run 
against an older version.

I'll discuss this problem more in a bit, but first let me mention an 
additional complication.  For dual-life distributions, like any other 
CPAN distributions, you need to have the proper permissions on PAUSE. 
But in this case you need permissions equivalent to that of the "P5P" 
user -- which traditionally was the Perl 5 pumpking and now (I believe) 
is the Perl Steering Council (PSC).  So in the past all releases of 
'dist/' distros to CPAN had to be conducted by the pumpking or someone 
authorized to act on his behalf.  That logistical hurdle has contributed 
to our failure to keep CPAN releases of 'dist/' distros up-to-date.

A New Diagnostic

I have released to CPAN a distribution called 'Perl5-Dist-Backcompat' 
(https://metacpan.org/pod/Perl5::Dist::Backcompat).  This distribution 
contains a command-line executable, 'p5-dist-backcompat' 
(https://metacpan.org/release/JKEENAN/Perl5-Dist-Backcompat-0.10/source/scripts/p5-dist-backcompat), 
which can be used to answer the following question, which you should 
think of as being posed in the subjunctive mood:

"If we were to take the main code of a given 'dist/' library as it 
exists in the core distribution today and make a fresh release of that 
code to CPAN, would it configure, build and test properly against all 
the older versions of perl against which the existing CPAN release of 
that codebase works properly today?"

To answer this question we need to have a machine on which we have 
installed (i) a set of executables of older versions of perl; (ii) a set 
of tarballs of 'dist/' distros downloaded from CPAN (or functionality to 
download such tarballs); and (iii) Perl5-Dist-Backcompat.  Fortunately, 
we have such a machine:  the Perl 5 development server known as 
Dromedary.  Thanks to decades of diligent work on the part of H. Merijn 
Brand ("Tux"), we have non-threaded and threaded versions of each 
development, production and maintenance release of perl going back at 
least to 1998 on this machine.  In my account on that machine I have 
built a perl-5.34.0 against which I have installed 
Perl5-Dist-Backcompat, and I have also downloaded 40 'dist/' tarballs 
from CPAN.  By running p5-dist-backcompat, I can generate data which 
gives us an estimate of the degree of difficulty we will have when we 
next go to perform CPAN releases of any of these 40 distros.

This library is meant for diagnostic purposes only and it is very much a 
hack.  Some of the 40 'dist/' distros have their tests set up in a very, 
shall we say, idiosyncratic manner which p5-dist-backcompat does not yet 
know how to handle smoothly.  In my next post I will present what 
inferences and recommendations I have been able to make so far about the 
"CPAN-releasability" of these 40 'dist/' distros.  If you would like to 
help me improve this diagnostic tool, please contact me *off-list*.

Thank you very much.
Jim Keenan

• Challenges in CPAN Releases of 'dist/' Libraries (1 of 3) by James E Keenan
• Re: Challenges in CPAN Releases of 'dist/' Libraries (1 of 3) by Tom Molesworth via perl5-porters
• Re: Challenges in CPAN Releases of 'dist/' Libraries (1 of 3) by Dan Book
• Re: Challenges in CPAN Releases of 'dist/' Libraries (1 of 3) by Dan Book