[perl #129229] [PATCH] Fix Parallel Building

On Mon Sep 19 06:50:57 2016, tomhukins wrote:
> On Sun, Sep 18, 2016 at 04:52:00PM -0700, James E Keenan via RT wrote:
> > I'm a bit puzzled by this report.  I have been smoke-testing blead on
> > FreeBSD approximately twice a week for the past month, typically with
> > '-j8', and I have yet to see evidence of this problem.
> 
> In case I was unclear, I wouldn't expect a dependency error to always
> occur under parallel builds in every environment.  However, in one of
> my
> environments, it reliably does.
> 
> > Could you provide more evidence of what you're seeing?
> 
> Here's the end of the output I see when running
> sh -c 'sh Configure -des -Dusedevel && make -j4'
> 
> Processing PropertyAliases.txt
> cc -c    -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
> -pipe -fstack-protector -I/usr/local/include -D_FORTIFY_SOURCE=2 -Wall
> -Werror=declaration-after-statement -Wextra -Wc++-compat -Wwrite-
> strings -O    -DVERSION=\"1.39\"  -DXS_VERSION=\"1.39\"  "-I../.."
> -DLIBC="" DynaLoader.c
> Finishing property setup
> Processing PropValueAliases.txt
> rm -rf ../../DynaLoader.o
> cp DynaLoader.o ../../DynaLoader.o
> rm -f libperl.a
> /usr/bin/ar rc libperl.a op.o     perl.o  gv.o toke.o perly.o pad.o
> regcomp.o dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o
> run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o
> regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o
> perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o caretx.o
> dquote.o time64.o   DynaLoader.o
> cc -o perl -Wl,-E  -fstack-protector -L/usr/local/lib  perlmain.o
> libperl.a `cat ext.libs` -lpthread -lm -lcrypt -lutil -lc
> Processing extracted/DGeneralCategory.txt
> Can't locate Cwd.pm in @INC (you may need to install the Cwd module)
> (@INC contains: lib /usr/local/lib/perl5/site_perl/5.25.5/amd64-
> freebsd /usr/local/lib/perl5/site_perl/5.25.5
> /usr/local/lib/perl5/5.25.5/amd64-freebsd /usr/local/lib/perl5/5.25.5
> /usr/local/lib/perl5/site_perl .) at lib/File/Find.pm line 8.
> Compilation failed in require at Porting/pod_lib.pl line 4.
> BEGIN failed--compilation aborted at Porting/pod_lib.pl line 4.
> Compilation failed in require at Porting/manisort line 17.
> WARNING: re-sorting MANIFEST
> Can't locate Cwd.pm in @INC (you may need to install the Cwd module)
> (@INC contains: lib /usr/local/lib/perl5/site_perl/5.25.5/amd64-
> freebsd /usr/local/lib/perl5/site_perl/5.25.5
> /usr/local/lib/perl5/5.25.5/amd64-freebsd /usr/local/lib/perl5/5.25.5
> /usr/local/lib/perl5/site_perl .) at lib/File/Find.pm line 8.
> Compilation failed in require at Porting/pod_lib.pl line 4.
> BEGIN failed--compilation aborted at Porting/pod_lib.pl line 4.
> Compilation failed in require at Porting/manisort line 17.
> *** [MANIFEST.srt] Error code 2
> magic: PERL_MAGIC_qr already provided by misc
> including ppphdoc
> including ppphbin
> including version
> including threads
> including limits
> including uv
> including memory
> including misc
> including variables
> including mPUSH
> including call
> including newRV
> including newCONSTSUB
> including MY_CXT
> including format
> including SvREFCNT
> including newSV_type
> including newSVpv
> including SvPV
> including Sv_set
> including sv_xpvf
> including shared_pv
> including HvNAME
> including gv
> including warn
> including pvs
> including magic
> including cop
> including grok
> including snprintf
> including sprintf
> including exception
> including strlfuncs
> including pv_tools
> running "/usr/home/tom/checkouts/perl5/miniperl" -I../../lib
> ppport_h.PL
> Processing extracted/DCombiningClass.txt
> installing ppport.h for cpan/DB_File
> installing ppport.h for cpan/IPC-SysV
> installing ppport.h for cpan/Scalar-List-Utils
> installing ppport.h for cpan/Win32API-File
> installing ppport.h for dist/PathTools
> installing ppport.h for dist/Time-HiRes
> removing temporary file PPPort.pm
> removing temporary file ppport.h
> Processing extracted/DNumType.txt
> Processing extracted/DEastAsianWidth.txt
> Processing extracted/DLineBreak.txt
> Processing extracted/DBidiClass.txt
> Processing extracted/DDecompositionType.txt
> Processing extracted/DBinaryProperties.txt
> Processing extracted/DNumValues.txt
> Processing extracted/DJoinGroup.txt
> Processing extracted/DJoinType.txt
> Processing Jamo.txt
> Processing UnicodeData.txt
> Processing ArabicShaping.txt
> Processing Blocks.txt
> Processing PropList.txt
> Processing SpecialCasing.txt
> Processing LineBreak.txt
> Processing EastAsianWidth.txt
> Processing CompositionExclusions.txt
> Processing BidiMirroring.txt
> Processing CaseFolding.txt
> Processing DCoreProperties.txt
> Processing Scripts.txt
> Processing DNormalizationProps.txt
> Processing DAge.txt
> Processing HangulSyllableType.txt
> Processing auxiliary/WordBreakProperty.txt
> Processing auxiliary/GraphemeBreakProperty.txt
> Processing auxiliary/GCBTest.txt
> Processing auxiliary/SBTest.txt
> Processing auxiliary/WBTest.txt
> Processing auxiliary/SentenceBreakProperty.txt
> Processing NamedSequences.txt
> Processing NameAliases.txt
> Processing auxiliary/LBTest.txt
> Processing ScriptExtensions.txt
> Processing IndicSyllabicCategory.txt
> Processing BidiBrackets.txt
> Processing IndicPositionalCategory.txt
> Finishing processing Unicode properties
> Compiling Perl properties
> Creating Perl synonyms
> Writing tables
> Making pod file
> Making test script
> Updating 'mktables.lst'
> 1 error
> 

Aha!  You are not alone!  I reported this (albeit, without much detail or insight) on list a little over two weeks ago:

http://www.nntp.perl.org/group/perl.perl5.porters/2016/09/msg239501.html

And because the errors were only intermittent, I forgot about them.

And the OS in question was FreeBSD 10.3; I don't see this on Ubuntu Linux 16.04 LTS.

Replying to Leon:

The version of make which I use there is whatever the default version of make is on FreeBSD.  There doesn't appear to be a --version switch for make, but 'man make' contains the following:

#####
Other make dialects (GNU make, SVR4 make, POSIX make, etc.) do not support most of the features of make as described in this manual.
...
This make implementationis based on Adam De Boor's pmake program which was written for Sprite at Berkeley.
#####

> > Also, AFAICT there was nothing FreeBSD-specific in commit
> > 3dfcef7e97bf1b516f.  Nor is there anything FreeBSD-specific in your
> > patch.
> 
> I agree with both your statements.
> 
> > 3dfcef7e97bf1b516f was committed to blead on June 21.  Hence, if
> > there
> > were a problem with parallel builds I would have expected it to show
> > up on multiple OSes, not just FreeBSD, and to have started to appear
> > three months ago.  Can you clarify?
> 
> Yes, I don't mean to suggest that the build will fail every time time
> in
> every environment.  I have one particular environment where I can make
> it occur reliably, but it may not do so elsewhere.
> 
> I noticed that setting "-Doptimize='-O2'" makes the problem go away in
> my environment, for example.
> 
> As I wrote in my commit message, I believe the cause of these symptoms
> is an additional dependency on Cwd added in 3dfcef7e97bf1b516f that
> has
> not been reflected in the build system.  I believe my patch addresses
> that, although I recognise it might do so suboptimally or inelegantly.
> 
> Tom

I will try to build some perls on FreeBSD and see if I can capture error output like yours.

Thank you very much.


-- 
James E Keenan (jkeenan@cpan.org)

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=129229

• [perl #129229] [PATCH] Fix Parallel Building by James E Keenan via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by Kent Fredric
• Re: [perl #129229] [PATCH] Fix Parallel Building by Tom Hukins
• [perl #129229] [PATCH] Fix Parallel Building by Desmond Daignault via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by Aristotle Pagaltzis
• Re: [perl #129229] [PATCH] Fix Parallel Building by Kent Fredric
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by demerphq
• Re: [perl #129229] [PATCH] Fix Parallel Building by Dan Collins
• [perl #129229] [PATCH] Fix Parallel Building by bulk88 via RT
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by demerphq
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by demerphq
• Re: [perl #129229] [PATCH] Fix Parallel Building by demerphq
• Re: [perl #129229] [PATCH] Fix Parallel Building by Kent Fredric
• Re: [perl #129229] [PATCH] Fix Parallel Building by Dave Mitchell
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• [perl #129229] [PATCH] Fix Parallel Building by Father Chrysostomos via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by Tom Hukins
• [perl #129229] [PATCH] Fix Parallel Building by James E Keenan via RT
• [perl #129229] [PATCH] Fix Parallel Building by James E Keenan via RT
• Re: [perl #129229] [PATCH] Fix Parallel Building by Leon Timmermans
• Re: [perl #129229] [PATCH] Fix Parallel Building by Tom Hukins