develooper Front page | perl.perl5.porters | Postings from November 2015

Re: eliminate bootstrap files on platforms that do notuse@DynaLoader::dl_resolve_using?

Thread Previous | Thread Next
From:
bulk88
Date:
November 2, 2015 08:15
Subject:
Re: eliminate bootstrap files on platforms that do notuse@DynaLoader::dl_resolve_using?
Message ID:
20151102081524.19950.qmail@lists-nntp.develooper.com
Leon Timmermans wrote:
> On Fri, Oct 30, 2015 at 12:00 PM, bulk88 <bulk88@hotmail.com
> <mailto:bulk88@hotmail.com>> wrote:
>
>     $(BASEEXT) is "Bar", as in "Foo::Bar", so the .bs file was built in
>     the root src tree, next to the .o file and next to the .xs file, not
>     in blib, next to the dll/shared lib (.so/.dll). So
>     DynaLoader/XSLoader never found the file during a "dmake test" or a
>     "perl -Mblib t/test.t", unless I install the module, the .bs file
>     won't be found and wont run, so you can't pass a "make test" if you
>     actually MUST run a .bs file, and MUST load additional shlibs before
>     loading the XS shlib. Therefore, I say nobody is using .bs files.
>
>
> The $(INST_DYNAMIC) rule copies the .bs file to blib if non-empty,
> though that rule doesn't depend on $(BOOTSTRAP) directly so maybe you're
> observing a side effect of that.

Chunks of a Win32 EUMM makefile.
--------------------
DLEXT = dll
--------------------
INST_DYNAMIC     = $(INST_ARCHAUTODIR)\$(DLBASE).$(DLEXT)
--------------------
$(INST_DYNAMIC): $(OBJECT) $(MYEXTLIB) $(BOOTSTRAP) 
$(INST_ARCHAUTODIR)$(DFSEP).exists $(EXPORT_LIST) $(PERL_ARCHIVEDEP) 
$(INST_DYNAMIC_DEP)
	$(LD) -out:$@ $(LDDLFLAGS) $(LDFROM) $(OTHERLDFLAGS) $(MYEXTLIB) 
"$(PERL_ARCHIVE)" $(LDLOADLIBS) -def:$(EXPORT_LIST)
	if exist $@.manifest mt -nologo -manifest $@.manifest -outputresource:$@;2
	if exist $@.manifest del $@.manifest
	$(CHMOD) $(PERM_RWX) $@
--------------------
The copying to blib rule doesn't exist on Win32 in MM_Win32.pm's sub 
dynamic_lib, it hasn't existed since day 1 of Win32 on Perl in 
http://perl5.git.perl.org/perl.git/commitdiff/68dc074516a6859e3424b48d1647bcb08b1a1a7d 
in 5.003_94 from 1997 . Okay, the root .bs to blib copy exists on EUMM 
on unix's sub dynamic_lib, but not in Win32. Chunk of sub dynamic_lib 
from MM_Unix.pm.
--------------------
     push @m, <<'MAKE';
	$(CHMOD) $(PERM_RWX) $@
	$(NOECHO) $(RM_RF) $(BOOTSTRAP)
	- $(CP_NONEMPTY) $(BOOTSTRAP) $(INST_BOOT) $(PERM_RW)
MAKE
--------------------

>
>     The only thing I can find with CPAN grep (my tries
>     http://grep.cpan.me/?q=\w_BS\W&page=1
>     <http://grep.cpan.me/?q=%5Cw_BS%5CW&page=1>
>     http://grep.cpan.me/?q=file%3A.[bB][sS]+...
>     http://grep.cpan.me/?q=file%3A_[bB][sS]+... ) that uses *_BS files,
>     is the EUMM test
>     https://metacpan.org/source/RJBS/perl-5.22.0/cpan/ExtUtils-MakeMaker/t/Mkbootstrap.t#L54
>     for *_BS files
>
>
> I know of one dist in the wild that used it:
> https://metacpan.org/source/PMQS/DB_File-1.835/DB_File_BS. Given how
> long Next has been dead that's not exactly an argument for pretending
> it's alive though.

I would guess all shared lib file formats on planet earth by now have a 
way of specifying additional dependent shared libs to load into the 
process. I would guess one use of .bs files is, and I am making this up, 
libc has a socket() function, it is a stub that always returns EINVAL. 
Your [perl?] app only links again libc for some reason, and you didnt 
load "libip4" after "libc", now you call socket(), it always fails with 
EINVAL, it does NOT fail at 1st execution with lazy binding with linker 
error, since libc has an implementation of socket(). For whatever 
reason, you put libipv4.so in the .bs file, now you module magically 
works fine. Are there OSes that backwards, no file list in the shared 
lib? Does the C linker think that because it found socket() in -lc, it 
will remove and ignore your -lipv4 C linker arg and never put the the 
libipv4.so entry in the XS module shared lib.

>
>     Or eliminate generation and reading on every platform except HPUX
>     and FREEMINT?
>
>     are .bs files bs? (pun intended)
>
>
> I suspect that treating platforms special isn't very helpful.

EUMM and Dynaloader are already highly platform specific, dropping out 
code and features not applicable or not possible on the platform 
wouldn't change the design that both are OS specific.

>
>     I managed to actually use a .bs file for its supposed purpose on
>     Win32, but there are a huge amount of hurdles and bugs in the way,
>     to use it seriously, I am thinking it is dead code.
>
>
> I suspect the same TBH, but is that a problem?

The .bs files (Mkbootstrap.pm specifically) are the holding me back from 
breaking dynamic XS module's dependency's on DynaLoader.pm being built 
first (see Extensions node in attached pic). There is another way to 
remove that dependency without removing bootstrap files, but while I am 
on the topic of bootstrap files, I need to fully work them up and 
examine their implementation and there should be a discussion on whether 
they are still needed in 2015 and how much they are needed. Rewriting 
code and leaving all bugs in, because they were there before, isnt very 
smart. Either fix it and advertise it, or throw it out.

Checking for bootstrap files are an IO call (or 5 of them on Win32 due 
to Perl's win32 stat) to check for the file. Reducing disk I/O is never 
a bad thing. Also less memory for ops in XSLoader/DynaLoader since some 
stmts and blocks were removed.

The bootstrap target takes 157 ms (make tool's timestamps, entire Foo.bs 
target) for me to execute all together, so making the section empty 
would speedup XS module building.

"C:\p523\bin\perl.exe" -l -e "binmode STDOUT, qq{:raw}; print qq{@ARGV}" 
-- "Running Mkbootstrap for XML::Parser::Expat ()" 26 ms

"C:\p523\bin\perl.exe" "-MExtUtils::Mkbootstrap" 
-e"Mkbootstrap('Expat','');" 43 ms

"C:\p523\bin\perl.exe" -MExtUtils::Command -e touch -- "Expat.bs" 45 ms
"C:\p523\bin\perl.exe" -MExtUtils::Command -e chmod -- 644 "Expat.bs" 25 ms

If bootstrap files are fixed to work on Win32, my concern is, will 
anyone on CPAN ever use them, other than me using when I become the 
primary maintainer of something, and I want to "force" usage of 
bootstrap files for "uniformity" and throw out whatever previous scheme 
of finding non-XS DLLs the module used before. I could imagine using a 
FIXED .bs file implementation on Win32 if I was writing a new TK/GTK/WX 
XS module from scratch for CPAN, but if nobody will ever use it...

I'd also have to require the future new TK/GTK/WX XS module to be EUMM 
version >7.DOESNTEXISTYET and also in effect ban every perl < 5.24 
(doesnt exist yet).

There is a family of "Alien" modules which I've never understood what 
they exactly do but that they are often mentioned for compiling and 
linking with non-native to the OS libraries, but I believe they are 
nowadays a replacement for bootstrap files. Alien.pm files containing 
absolute paths specific to each machine (not each OS), which isn't 
possible with regular .pm files (lets not discuss PL_FILES), since 
regular .pm'es have to be and installed unmodified from the tarball.

Bootstrap files haven't worked (wont pass "make test"), EVER, on Win32, 
since day 1 of EUMM::MM_Win32.pm

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About