develooper Front page | perl.perl5.porters | Postings from July 2013

[perl #23212] Can't build blead with -Duseshrplib on linux2.4/ppc since 19262

Nicholas Clark via RT
July 19, 2013 13:00
[perl #23212] Can't build blead with -Duseshrplib on linux2.4/ppc since 19262
Message ID:
On Tue Apr 24 09:59:39 2012, doughera wrote:

> Building with -Duseshrplib should work just fine on all platforms
> where it
> is supported.  No additional documentation is needed, in my opinion.
> In this particular ticket, the compiler was actually a perl script
> that
> called the real compiler. That configuration also currently works,
> under
> the reasonable restriction that the compiler script is to be run with
> a
> pre-existing perl installation, not with the one currently being
> built.
> I have just tested that precise set-up, and it worked fine.  I think
> you
> can close this ticket.

I have verified that building with -Duseshrplib works on the linux/PPC
system that I have access to.

I can also verify *most* of the problems referred to in this ticket,
#22941 and the thread "Re: Smoke failure" here:

Please note that the perl -V in this bug report *doesn't* reflect the
problematic configuration, as the -V has useshrplib as false.
However, the build output is consistent with a build of perl@20172 or
later (commit 830717a75f047ef8, July 15 2003), where Makefile.SH
was run with LD_LIBRARY_PATH set to /home/alb/.soft/perl-current
(Makefile.SH adds the value again to anything existing in the
environment, hence why it's doubled up in the Makefile)
That commit added the conditional code to Makefile.SH to
`test -f $archlib/CORE/$libperl` and only generate a preload script
if it's present. Immediately prior to that a preload script was
always generated for the Linux -Duseshrplib case, using LD_PRELOAD
to force the loading of the freshly built in preference to
any installed version. (Added by commit 9b9c6f34276ee8dd, April 3
2003, as change 19150)

For the setup described:
*) Building with -Duseshrplib
*) cc is actually perl script
*) the script's perl uses a shared perl library

At commit 20171 I get a SEGV from the cc script, as the LD_PRELOAD
forces its perl to load an incompatible

At commit 19310 I get a symbol lookup failure from the script (which
I infer happens before anything prompts a SEGV - the is
still incompatible, but by chance not in a way that would SEGV):

/home/nick/Perl/perl2/preload /home/nick/Perl/perl2/
./miniperl -Ilib lib/lib_pm.PL
Extracting (with variable substitutions)
        AutoSplitting perl library
/home/nick/Perl/perl2/preload /home/nick/Perl/perl2/
./miniperl -Ilib -e 'use AutoSplit; \
                autosplit_lib_modules(@ARGV)' lib/*.pm
/home/nick/Perl/perl2/preload /home/nick/Perl/perl2/
./miniperl -Ilib -e 'use AutoSplit; \
                autosplit_lib_modules(@ARGV)' lib/*/*.pm
make lib/
make[1]: Entering directory `/home/nick/Perl/perl2'
make[1]: `lib/' is up to date.
make[1]: Leaving directory `/home/nick/Perl/perl2'
/home/nick/Perl/perl2/preload /home/nick/Perl/perl2/
./miniperl > minimod.tmp
sh mv-if-diff minimod.tmp lib/ExtUtils/
File lib/ExtUtils/ not changed.
touch lib/ExtUtils/

        Making DynaLoader (static_pic)
make[1]: Entering directory `/home/nick/Perl/perl2/ext/DynaLoader'
make[1]: Leaving directory `/home/nick/Perl/perl2/ext/DynaLoader'
make[1]: Entering directory `/home/nick/Perl/perl2/ext/DynaLoader'
/home/nick/test/toolchain/cc -c   -D_REENTRANT -D_GNU_SOURCE
-DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g  
-DVERSION=\"1.04\" -DXS_VERSION=\"1.04\" -fpic "-I../.."  -DPERL_CORE
-DLIBC="" DynaLoader.c
/home/nick/test/toolchain/perl: symbol lookup error:
/home/nick/Perl/perl2/ undefined symbol: pthread_key_create
make[1]: *** [DynaLoader.o] Error 127
make[1]: Leaving directory `/home/nick/Perl/perl2/ext/DynaLoader'
make: *** [lib/auto/DynaLoader/DynaLoader.a] Error 2

Comments at the time suggest that there was a bug with MakeMaker
where it sometimes would pick up the wrong perl, which might explain
attempting to build DynaLoader with /usr/bin/perl instead of miniperl.

*Finally*, just as I am writing this I can find a way to recreate
the error shown in the report:

$ make LD_LIBRARY_PATH=/bogus
        AutoSplitting perl library
LD_LIBRARY_PATH=/home/nick/Perl/perl5:/home/nick/Perl/perl5 ./miniperl
-Ilib -e 'use AutoSplit; \
                autosplit_lib_modules(@ARGV)' lib/*.pm
LD_LIBRARY_PATH=/home/nick/Perl/perl5:/home/nick/Perl/perl5 ./miniperl
-Ilib -e 'use AutoSplit; \
                autosplit_lib_modules(@ARGV)' lib/*/*.pm
make lib/
make[1]: Entering directory `/home/nick/Perl/perl5'
make[1]: `lib/' is up to date.
make[1]: Leaving directory `/home/nick/Perl/perl5'

        Making DynaLoader (static_pic)
Processing hints file hints/
No such file or directoryWriting Makefile for DynaLoader
make[1]: Entering directory `/home/nick/Perl/perl5/ext/DynaLoader'
make[1]: Leaving directory `/home/nick/Perl/perl5/ext/DynaLoader'
make[1]: Entering directory `/home/nick/Perl/perl5/ext/DynaLoader'
../../miniperl "-I../../lib" "-I../../lib" "-I../../lib" "-I../../lib"
../../miniperl: error while loading shared libraries: cannot
open shared object file: No such file or directory
make[1]: *** [] Error 127
make[1]: Leaving directory `/home/nick/Perl/perl5/ext/DynaLoader'

Setting LD_LIBRARY_PATH on the make command line will override the
environment setting in the recursive make call into ext/DynaLoader.
The behaviour is identical to LD_LIBRARY_PATH not being set, which
is what the symptoms look like, but seems to be "impossible" without
editing the Makefile.

Note also (to add to the fun), when Makefile.SH generates Makefile
it "bakes in" the value of LD_LIBRARY_PATH found in *its*
environment. So if LD_LIBRARY_PATH at that time is /foo/bar/baz then
the Makefile's macro LDLIBPTH is set like this:

LDLIBPTH = LD_LIBRARY_PATH=/foo/bar/baz:/path/to/the/build/tree

Which means that if you then run make like this

LD_LIBRARY_PATH=/zowie/thwape/boff make LD_LIBRARY_PATH=/hex/diamond/hash

with a different LD_LIBRARY_PATH in its environment, and a potentially
different LD_LIBRARY_PATH on its command line, then

1) any unadorned commands in the Makefile will be run with a
   LD_LIBRARY_PATH of /zowie/thwape/boff
2) any commands that the Makefile explicitly has $(LDLIBPTH) in will
   be run with a LD_LIBRARY_PATH of
3) any unadorned commands in a recursively invoked Makefile will be
   run with a LD_LIBRARY_PATH of /hex/diamond/hash

Yes, this is the absolute worst case, but you can see the potential
for confusion with up to 4 different values in play.

The make command line is not shown in the original bug report. I
can't think of any other way to have LD_LIBRARY_PATH become unset, so
I will assume that something like this is what is going on - ie that
in an attempt to work around the problems the reporter was setting a
different LD_LIBRARY_PATH on the command line.

As best I can tell from trying to replicate the reporter's setup

1) (on a current toolchain) setting LD_LIBRARY_PATH doesn't override
   any rpath set in the binary (ie, effectively directories in
   LD_LIBRARY_PATH are appended to the search path, not prepend, and
   not a replacement). Hence the binary uses the installed
   (if it is in the right place), not the newly created
   which is in LD_LIBRARY_PATH
2) (on a current toolchain) you can only force the loading of that
   local by using LD_PRELOAD
3) (on a standard toolchain) the compiler does not confuse
   LD_LIBRARY_PATH with LD_RUN_PATH. The *latter* compiles search
   paths into the executable (only), the former specifies paths at
   runtime (only)

and hence

1) Commit 20172 resolves the reporter's bug whereby the Makefile was
   inserting a LD_PRELOAD that forced loading of the newly build (crashing his installed toolchain)
2) I infer that the reporter was attempting to work round problems by
   setting some sort of override
3) I infer that for this bug (that removing the LD_LIBRARY_PATH force
   from the linker line) either
   a) The reporter's setup had a LD_LIBRARY_PATH on the make command
      line or in the make environment which messed up the compiler
      (but only the link phase, as the C compilation rules ran just
   b) The reporter's C compiler, or the wrapper, "helpfully" converted
      LD_LIBRARY_PATH values to LD_RUN_PATH (or -Wl,-rpath command
      line link flags), which latter parts of the build happened to be
      relying on for miniperl to find
      (and the usual mechanism was being suppressed by a
       LD_LIBRARY_PATH override)


1) that this ticket doesn't tell the whole story about the make
2) that the toolchain was more non-standard than described in this

Note that reverting commit 19282 as 20478 was not the solution: which
suggests that our understanding at the time of the problem was never
complete, and the change of those two commits (removing, then
re-adding LD_LIBRARY_PATH to the cc invocation for linking) quite
possibly was only moving the symptoms around, not fixing the cause.

On a current PPC linux system I can build perl@20172 using a cc
wrapper running on 5.8.0 built with a shared perl library *if it is
not installed*. I can't build 20172's parent (it SEGVs). I still can't
build at 20172 with a perl-based toolchain if I run Makefile.SH
*after* the library has been installed.

The specific symptom reported in this ticket is impossible as of
this commit:

commit 908fcb8bef8cbab8cfe098520d89599eb7d1a16c
Author: Gisle Aas <>
Date:   Mon Mar 20 10:21:50 2006 +0000

    Move DynaLoader.o into
    This avoids the need to statically link DynaLoader into the stub perl
    executable and make provide all the code needed to get a
    functional embedded perl interpreter up running.  As a side effect
    this also moves DynaLoader into libperl.a for non-useshrplib builds.
    Fixes [perl #32539]
    p4raw-id: //depot/perl@27549

because a side effect not noted in the commit message is that from
here on miniperl is statically linked against the object files. As
it's no longer a stub executable dynamically linked against, it doesn't need to find at runtime, so can't
fall foul of the specific symptom reported.

However, I believe that even in blead it remains impossible to build
blead (on Linux*) if

1) you are building with a shared perl library
2) you are building with a toolchain written in perl using a shared
   perl library
3) you have already installed a to the install location of
   the perl you are building

Certainly, with blead on a current Linux system I see this:

make[1]: Entering directory `/home/nick/Perl/perl/ext/DynaLoader'
make[1]: Leaving directory `/home/nick/Perl/perl/ext/DynaLoader'
Making all in ext/DynaLoader
 make all PERL_CORE=1 LINKTYPE=static
make[1]: Entering directory `/home/nick/Perl/perl/ext/DynaLoader'
/home/nick/test/toolchain/bin//cc -c   -D_REENTRANT -D_GNU_SOURCE
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-DXS_VERSION=\"1.18\" -fPIC "-I../.."  -DLIBC="/lib/"
make[1]: *** [DynaLoader.o] Segmentation fault (core dumped)
make[1]: Leaving directory `/home/nick/Perl/perl/ext/DynaLoader'
Unsuccessful make(ext/DynaLoader): code=512 at line 490.
make: *** [DynaLoader.o] Error 2

The SEGV comes from the perl interpreter running the script
because that (old) perl interpreter is being forced to load the
build tree's thanks to this in the Makefile:

LDLIBPTH = LD_LIBRARY_PATH=/home/nick/Perl/perl
/home/nick/Perl/perl/preload /home/nick/Perl/perl/

*This* is the answer to the question "Why I see /usr/bin/perl ??" in

The /usr/bin/perl which is reporting errors loading the shared object
is the /usr/bin/perl running the Perl script which wraps the compiler.

This is also why the (then) 5.9.0 is trying to load Term::ANSIColor
in the message here:

It's actually the compiler wrapper, running /usr/bin/perl (a 5.8.0).
However, the C code to initialise @INC is in, so if that
5.8.0 /usr/bin/perl has been forced to load the local (5.9.0) (and doesn't SEGV or have link errors) then it will think
that it's 5.9.0 from then on.

I fear that chunks of this problem are intractable. Specifically,
I can't see any way to have all of

0) Testing the actual binary that will get installed
1) Overriding the shared library that it loads to use the newly built
   library, not the existing installed library
2) Without overriding the shared library loaded by any other installed
   perl binary

In particular, I don't think that renaming to (etc) will help, because it's LD_PRELOAD hackery,
not LD_LIBRARY_PATH, that is needed to force the under-test binary
to ignore its search path, so such hackery *forces* *any* executable
(and every executable) to load Not simply to tell
the /usr/bin/perl to look for its (starting) in a
different place.

And I suspect that any logic for "am I in a build tree" would also be
picked up by the installed perl (certainly by the time one is building
5.21.3 using a toolchain running on 5.20.0) so that won't win either.


(The problem is neither PPC nor Linux specific)

Nicholas Clark

* The code in Makefile.SH only does the LD_PRELOAD hackery on Linux.
  I think that this is a bug, in that it ought to be doing it on all based platforms. And arguably the analogous way on all
  Also the code should be using $archlibexp for the test, not $archlib

via perlbug:  queue: perl5 status: resolved Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About