develooper Front page | perl.perl5.porters | Postings from October 2018

Re: [perl #133587] toolchain possibilities for avoiding "binariesmismatched" error

Thread Previous | Thread Next
Graham Knop
October 16, 2018 11:15
Re: [perl #133587] toolchain possibilities for avoiding "binariesmismatched" error
Message ID:
On Mon, Oct 15, 2018 at 1:39 PM Dave Mitchell <> wrote:
> On Sat, Oct 13, 2018 at 12:15:09PM -0700,  (via RT) wrote:
> > 1. In using 'cpanm' and local::lib, I thought I was doing the
> > "standard thing", for example cpanm mentions local::lib under the
> > documentation for the "-l, --local-lib" flag. But other people seem to
> > be installing modules in "version-specific directories", which I don't
> > even know how to do. It doesn't seem to be the default, since I just
> > ran 'cpan' in a new user account and after hitting 'enter' at every
> > prompt and typing 'install List::BinarySearch', I found that the XS
> > module was installed here:
> >
> > ~/perl5/lib/perl5/x86_64-linux-thread-multi/auto/List/BinarySearch/XS/
> >
> > I assume this location is not version-specific, since it has no
> > version number, just the architecture. How does someone (like Dave?)
> > install a module like List::BinarySearch without encountering the
> > above scenario, which he describes as "relatively rare"? (I don't
> > think my distribution has List::BinarySearch pre-packaged, but let's
> > assume that it doesn't)
> A perl configured and built using default options *will* use
> version-specific paths. Here's a simple test I did just now:
> $ tar xfz ~/perl5/tarballs/perl-5.28.0.tar.gz
> $ cd perl-5.28.0/
> $ PATH=/bin:/usr/bin sh Configure -des -Dprefix=/home/davem/tmp/foo/perl  && TEST_JOBS=16  HARNESS_OPTIONS=j16  make  -j 16 test_harness && make  -j 16 install
> $ cd ..; $ rm -rf perl-5.28.0
> $ cd /home/davem/tmp/foo
> $ perl/bin/cpan
> cpan[1]> install List::BinarySearch
> Would you like to install List::BinarySearch::XS? [y]
> $ find . |grep BinarySearch
> ./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch
> ./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS
> ./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS/
> ./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS/.packlist
> ./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/.packlist
> ./perl/lib/site_perl/5.28.0/x86_64-linux/List/BinarySearch
> ./perl/lib/site_perl/5.28.0/x86_64-linux/List/BinarySearch/
> ./perl/lib/site_perl/5.28.0/List/
> ./perl/lib/site_perl/5.28.0/List/BinarySearch
> ./perl/lib/site_perl/5.28.0/List/BinarySearch/
> ./perl/man/man3/List::BinarySearch.3
> ./perl/man/man3/List::BinarySearch::XS.3
> ./perl/man/man3/List::BinarySearch::PP.3
> Please look carefully at the above, and bear it very keenly in mind for
> any further discussion. If that doesn't convince you that perl does in
> indeed use version-specific paths *by default*, then you'll have to be
> very clear and specific about what your doubts are.
> > 2. I think people were taken aback when I mentioned that I have common
> > binary modules like Compress::Raw::Zlib installed in $HOME. I later
> > learned this is something that 'cpan' and 'cpanm' do automatically
> No, by default cpan etc will try to install under the site_perl directory
> of wherever perl was installed, unless you tell it otherwise, e.g. with
> options at perl build time, or by using an external tool like local::lib
> to set some environment variables which tells cpan to install somewhere
> else.

This is true if you have permissions to install in site_perl.  But if
you try to install modules using system perl without running as root,
you probably won't have access to do that.

In that case, both and cpanm will default to using local::lib
and installing into ~/perl5, using ExtUtils::MakeMaker's INSTALL_BASE
option and Module::Build's --install_base option.  Those options don't
use versioned paths.

> > How are users notified of the fact that system-level modules are being
> > reinstalled locally, turned into "dual-life modules" as it were?
> Users will only get this behaviour if they request it.
> The term 'dual-life' is usually used (by us at least) to refer to
> something different - where a module is both bundled with perl and
> available (possibly in a newer version) on CPAN. It doesn't (very much)
> relate to where the module gets installed).
> > How are other people (like Andy Dougherty?) avoiding 'cpan' install
> > actions that pull in binary dependencies like Compress::Raw::Zlib? I'm
> > guessing he didn't just "hit enter" at one of the prompts, like I did?
> Again, this just doesn't normally occur with a default installation.
> > Perhaps I'm taking the wrong approach, maybe I should not be using the
> > 'cpan' tool in my use case - namely "unprivileged user with simple
> > utility scripts that have a few external dependencies". However, I'm
> > guessing it is a pretty common case, and I realized that the problem
> > I'm running into with the confusing "binaries mismatched" error
> > message is something that I could completely avoid if I could tell
> > 'cpan' to refrain from installing what Dan called "dual-life modules",
> > in other words to stick with installing only modules that are not
> > already installed system-wide (or elsewhere in PERL5LIB).
> No, what you really want to do is, if installing under your home
> directory, install version-sensitive files under version-specific paths.
> This appears to be a infelicity in the current local::lib tool. I don't
> know a lot about local::lib, and I'm not a "toolchain guy", so I don't
> know whether there's a good reason why local::lib does things the way it
> does or whether it can be altered / extended to include versions in paths.

local::lib is using the options provided by ExtUtils::MakeMaker and
Module::Build.  I've worked in the past on using additional options to
force installing into versioned directories, but didn't end up
finishing the work.  I had found other options for myself, and nobody
else pushed for fixing the issue until now, so I hadn't gotten back to

> > Further, when there is an option to install a binary version of a
> > module, it would be nice to be able to tell 'cpan'/'cpanm' that I am
> > only interested in the pure-Perl implementation, or at least for it to
> > install the binary component in such a way that Perl can automatically
> > fall back on the pure-Perl version of the module, when I find myself
> > using my scripts on a machine with a different architecture or Perl
> > version or whatever else would make it impossible to employ the binary
> > module. In rare cases where I want to depend on a binary module, I
> > would probably do it by installing the distribution package for that
> > module, since there is a big overlap between modules which have XS
> > code and modules which are available in distribution-specific
> > packages. Basically, a big feature of Perl is backwards compatibility,
> > and it seems like I should be able to assemble a collection of
> > modules, using automatic tools like 'cpan', in such a way that they
> > continue to run correctly on a wide variety of systems.
> I think such functionality would have to be done by the distribution
> itself on a case-by-case basis.
> > It seems to me that there are a few bugs or potential areas for
> > improvement here. Is anyone else concerned about these things, what is
> > the status, and am I missing something?
> My thoughts:
> * The situation is not ideal, but
> * fixes are not easy: this whole area is complex (and not one I
>   understand well)
> --
> Technology is dominated by two types of people: those who understand what
> they do not manage, and those who manage what they do not understand.

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About