develooper Front page | perl.perl5.porters | Postings from October 2018

Re: [perl #133587] toolchain possibilities for avoiding "binariesmismatched" error

Thread Previous | Thread Next
From:
Dave Mitchell
Date:
October 15, 2018 11:38
Subject:
Re: [perl #133587] toolchain possibilities for avoiding "binariesmismatched" error
Message ID:
20181015113844.GR3102@iabyn.com
On Sat, Oct 13, 2018 at 12:15:09PM -0700,  (via RT) wrote:
> 1. In using 'cpanm' and local::lib, I thought I was doing the
> "standard thing", for example cpanm mentions local::lib under the
> documentation for the "-l, --local-lib" flag. But other people seem to
> be installing modules in "version-specific directories", which I don't
> even know how to do. It doesn't seem to be the default, since I just
> ran 'cpan' in a new user account and after hitting 'enter' at every
> prompt and typing 'install List::BinarySearch', I found that the XS
> module was installed here:
> 
> ~/perl5/lib/perl5/x86_64-linux-thread-multi/auto/List/BinarySearch/XS/XS.so
> 
> I assume this location is not version-specific, since it has no
> version number, just the architecture. How does someone (like Dave?)
> install a module like List::BinarySearch without encountering the
> above scenario, which he describes as "relatively rare"? (I don't
> think my distribution has List::BinarySearch pre-packaged, but let's
> assume that it doesn't)

A perl configured and built using default options *will* use
version-specific paths. Here's a simple test I did just now:

$ tar xfz ~/perl5/tarballs/perl-5.28.0.tar.gz
$ cd perl-5.28.0/
$ PATH=/bin:/usr/bin sh Configure -des -Dprefix=/home/davem/tmp/foo/perl  && TEST_JOBS=16  HARNESS_OPTIONS=j16  make  -j 16 test_harness && make  -j 16 install
$ cd ..; $ rm -rf perl-5.28.0
$ cd /home/davem/tmp/foo
$ perl/bin/cpan
cpan[1]> install List::BinarySearch
Would you like to install List::BinarySearch::XS? [y] 
$ find . |grep BinarySearch
./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch
./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS
./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS/XS.so
./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/XS/.packlist
./perl/lib/site_perl/5.28.0/x86_64-linux/auto/List/BinarySearch/.packlist
./perl/lib/site_perl/5.28.0/x86_64-linux/List/BinarySearch
./perl/lib/site_perl/5.28.0/x86_64-linux/List/BinarySearch/XS.pm
./perl/lib/site_perl/5.28.0/List/BinarySearch.pm
./perl/lib/site_perl/5.28.0/List/BinarySearch
./perl/lib/site_perl/5.28.0/List/BinarySearch/PP.pm
./perl/man/man3/List::BinarySearch.3
./perl/man/man3/List::BinarySearch::XS.3
./perl/man/man3/List::BinarySearch::PP.3

Please look carefully at the above, and bear it very keenly in mind for
any further discussion. If that doesn't convince you that perl does in
indeed use version-specific paths *by default*, then you'll have to be
very clear and specific about what your doubts are.

> 2. I think people were taken aback when I mentioned that I have common
> binary modules like Compress::Raw::Zlib installed in $HOME. I later
> learned this is something that 'cpan' and 'cpanm' do automatically

No, by default cpan etc will try to install under the site_perl directory
of wherever perl was installed, unless you tell it otherwise, e.g. with
options at perl build time, or by using an external tool like local::lib
to set some environment variables which tells cpan to install somewhere
else.

> How are users notified of the fact that system-level modules are being
> reinstalled locally, turned into "dual-life modules" as it were?

Users will only get this behaviour if they request it.

The term 'dual-life' is usually used (by us at least) to refer to
something different - where a module is both bundled with perl and
available (possibly in a newer version) on CPAN. It doesn't (very much)
relate to where the module gets installed).

> How are other people (like Andy Dougherty?) avoiding 'cpan' install
> actions that pull in binary dependencies like Compress::Raw::Zlib? I'm
> guessing he didn't just "hit enter" at one of the prompts, like I did?

Again, this just doesn't normally occur with a default installation.

> Perhaps I'm taking the wrong approach, maybe I should not be using the
> 'cpan' tool in my use case - namely "unprivileged user with simple
> utility scripts that have a few external dependencies". However, I'm
> guessing it is a pretty common case, and I realized that the problem
> I'm running into with the confusing "binaries mismatched" error
> message is something that I could completely avoid if I could tell
> 'cpan' to refrain from installing what Dan called "dual-life modules",
> in other words to stick with installing only modules that are not
> already installed system-wide (or elsewhere in PERL5LIB).

No, what you really want to do is, if installing under your home
directory, install version-sensitive files under version-specific paths.
This appears to be a infelicity in the current local::lib tool. I don't
know a lot about local::lib, and I'm not a "toolchain guy", so I don't
know whether there's a good reason why local::lib does things the way it
does or whether it can be altered / extended to include versions in paths.

> Further, when there is an option to install a binary version of a
> module, it would be nice to be able to tell 'cpan'/'cpanm' that I am
> only interested in the pure-Perl implementation, or at least for it to
> install the binary component in such a way that Perl can automatically
> fall back on the pure-Perl version of the module, when I find myself
> using my scripts on a machine with a different architecture or Perl
> version or whatever else would make it impossible to employ the binary
> module. In rare cases where I want to depend on a binary module, I
> would probably do it by installing the distribution package for that
> module, since there is a big overlap between modules which have XS
> code and modules which are available in distribution-specific
> packages. Basically, a big feature of Perl is backwards compatibility,
> and it seems like I should be able to assemble a collection of
> modules, using automatic tools like 'cpan', in such a way that they
> continue to run correctly on a wide variety of systems.

I think such functionality would have to be done by the distribution
itself on a case-by-case basis.

> It seems to me that there are a few bugs or potential areas for
> improvement here. Is anyone else concerned about these things, what is
> the status, and am I missing something?

My thoughts:

* The situation is not ideal, but
* fixes are not easy: this whole area is complex (and not one I
  understand well)

-- 
Technology is dominated by two types of people: those who understand what
they do not manage, and those who manage what they do not understand. 

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About