Scalar-List-Utils: blead and upstream out-of-synch

We have some problems with the integration of the Scalar-List-Utils 
distribution with the Perl 5 core distribution.  Scalar-List-Utils is 
maintained upstream on CPAN, which means that reports of bugs are 
supposed to be reported first on its bug tracker, 
https://rt.cpan.org/Dist/Display.html?Name=Scalar-List-Utils.  Once 
corrections are made upstream, we patch the core distribution.  Changes 
should not be made first in core unless there is an urgent, compelling 
reason to do so.

In late October and early November a series of corrections were made to 
Scalar-List-Utils in the core distribution with respect to its 
performance on the netbsd-vax platform.  In reverse chronological order, 
those commits were:

aa3ecafd01033b5ee5566f754a882d417a39e0ad
8e21c40378fa83db73acbf74b1cb99ac60432ee8
d1deb742f98169ce12c103478382902d83f6717a
059d8568a141af07d90f38d161fc6ba4993d4467
5bc8afd5b5a672585a115f4556f6b734dae83184
1476bbfa98fcd0efc3a4d1ccdd0b021ad7403986
cdeaf44d582bf74f4c686dc58b288d8d33f25b95
3ad3fb7edd9a5db1e0cf38ad50a84824ef641c39
5b15f36c4b1c5025dccabeef937495997e37e512

Because this work was committed to blead first, customizations were 
necessary in Porting/Maintainers.pl and other porting-tests-related 
fixes were needed as well.  So there is more code in the commits above 
than was, strictly speaking, needed for Scalar-List-Utils.  The code 
that was needed for that was subsequently reported upstream at 
https://rt.cpan.org/Ticket/Display.html?id=118470 but upstream has not 
yet been patched -- which means we have to continue to maintain the 
customizations in Porting/Maintainers.pl and elsewhere.

At the same time, our smoke testers were reporting intermittent failures 
in cpan/Scalar-List-Utils/t/tainted.t.  See, e.g., 
http://perl5.test-smoke.org/report/52519.  These test failures were 
first reported to the Scalar-List-Utils bug tracker 
(https://rt.cpan.org/Ticket/Display.html?id=119169) on Dec 07 2016.  A 
correction was discussed there and the upstream maintainer released 
version 1.47 on Dec 22 2016.

That version was not and has not yet been integrated into the core 
distribution.  Such integration would be complex given that blead 
contains the previously discussed customizations.  As a stopgap measure, 
on Dec 23 2016, the file with the failing tests was patched in blead 
(only) -- but with an expiration date on the corrective action of the patch:

#####
diff --git a/cpan/Scalar-List-Utils/t/tainted.t 
b/cpan/Scalar-List-Utils/t/tainted.t
index e483dfd..9e0df5e 100644
--- a/cpan/Scalar-List-Utils/t/tainted.t
+++ b/cpan/Scalar-List-Utils/t/tainted.t
@@ -15,10 +15,13 @@ ok( !tainted($var), 'known variable');

  my $key = (grep { !/^PERL/ } keys %ENV)[0];

+SKIP: { # Skip these to get blead to pass, but the skip expires soon
+skip 'is randomly failing', 2 unless $] gt 5.025009;
  ok( tainted($ENV{$key}),       'environment variable');

  $var = $ENV{$key};
  ok( tainted($var),     'copy of environment variable');
+}
#####

In blead we are now past 5.25.9.

#####
$ ./perl -v | head -2 | tail -1
This is perl 5, version 25, subversion 10 (v5.25.10 
(v5.25.9-15-g03fedef)) built for x86_64-linux

$ ./perl -Ilib -E 'say $]'
5.025010
#####

Not surprisingly, we are once again experiencing intermittent test 
failures in cpan/Scalar-List-Utils/t/tainted.t.  See, e.g., 
http://perl.develop-help.com/raw/?id=198039.

So we are now in a situation where blead and upstream are out of synch 
with respect to Scalar-List-Utils in multiple ways.  We are not getting 
the benefits of the upstream maintainer's work on 
https://rt.cpan.org/Ticket/Display.html?id=119169 and, consequently, 
that ticket remains open -- as does the upstream ticket with the 
netbsd-vax corrections.

This is not good.

We have several other, much larger problems in the interaction between 
blead and CPAN that we have to address before the upcoming pre-5.26.0 
code freeze.  Having this Scalar-List-Util problem is a distraction.

My recommendation is that we:

1. Revert the netbsd-vax-related changes out of Perl 5 blead.

2. Integrate Scalar-List-Utils v1.47 into blead.

3. Urge the Scalar-List-Utils maintainer to accept the 
netbsd-vax-related changes and release a v1.48 with those changes.  (We 
would then integrate 1.48 into blead.)

I recommend this approach because:

1. Getting S-L-U 1.47 into blead would get us smoke test PASSes across 
all platforms.  That's a greater benefit than having the 
netbsd-vax-related changes in blead without also having them upstream.

2. Getting S-L-U 1.47 into blead would enable us to get rid of the 
customizations in Porting/Maintainers.pl and the no-longer-working 
stopgap in tainted.t.

3. This would be consistent with our standard procedure for reporting 
problems with libraries maintained upstream but distributed with core.

To this end, I have created the following branch:  revert-cpan-118470. 
I am attaching a diff of that branch against blead.  You can review its 
smoke test results at 
http://perl.develop-help.com/?b=smoke-me%2Fjkeenan%2Frevert-cpan-118470.

Thank you very much.
Jim Keenan

• Scalar-List-Utils: blead and upstream out-of-synch by James E Keenan
• Re: Scalar-List-Utils: blead and upstream out-of-synch by Sawyer X
• Re: Scalar-List-Utils: blead and upstream out-of-synch by demerphq
• Re: Scalar-List-Utils: blead and upstream out-of-synch by Dave Mitchell