Re: Any empty members in struct xpvcv AKA CV that an XSUB can use?

Andy Dougherty wrote:
> On Wed, Oct 22, 2014 at 03:16:25PM -0400, bulk88 wrote:
> 
>> Must be 64 bit/void * sized. 
> 
> (Just a minor quibble, but I'm afraid I don't know exactly what you
> mean here.  What if perl is compiled with 64-bit IVs, but (void *)
> is only 32 bit?)

Egh, I should've said void * sized, not the "64 bit" part. What I ment 
was it can't be a reused 32 bit only member.

> 
>> Must be 64 bit/void * sized. What I am trying to do is stop
>> ABI/linking problems, where Dynaloader loads a XS module compiled
>> with 5.18 into a 5.20 process, or DEBUGGING vs no DEBUGGING or
>> DEBUG_LEAKING_SCALARS vs no DEBUG_LEAKING_SCALARS. On Win32 unlike
>> ELF, there is no global symbol table, so Perl/Dynaloader has no way
>> to know if Dynaloader from 5.18 loads a XS module that links with
>> 5.20, and both interps (5.18/5.20) happen to be in %PATH. 
> 
> On the other hand, on many systems it should be perfectly fine to load
> an XS module compiled with 5.20.0 into a 5.20.1 process, provided both
> were compiled with compatible Configure options.

That is fine. I am not trying to stop scenario at all. That is why I 
didn't include the 2nd "." in the version numbers above. But blead 
"maint" releases do currently break on the 3rd version part, rightfully so.

> 
> I'm not opposed to extra safeguards, but the installation hierarchy is
> supposed to automtically prevent this sort of mis-loading for the common
> cases (e.g. 5.18 vs. 5.20, 64-bit vs 32-bit, debugging vs. non-debugging,
> and threading).  I gather Windows installs things differently, but it
> may be worthwhile to revisit what it is doing differently and why,
> and whether that's where this problem should be addressed.

Windows builds usually dont use the arch specific path feature. I've 
never tried building with arch specific path feature. Specifically my 
setup has problems where I plan install a newer blead on top of an older 
one, or a blead ontop of stable. Also Perl has a tendency to find other 
perl installs on my system because of too many ".."s in @INC from 
test.pl. Also I have open tickets on the issue of test files using the 
installed directory during testing. Nobody else is interested in fixing 
them so I'll just add better diagnostics/failures. A couple of the 
EUMM-related or EUMM itself (I dont remember) modules use 
$Config{bin}/$Config{archlib}/etc in their tests and explicitly put 
$Config{archlib} into @INC, which is an older or ABI incompatible perl. 
So an uninstalled perl doing a make test, is using its "future" install 
dir in @INC, and loads XS modules from another perl.

Another case I run into is installing DEBUGGING blead over non-DEBUGGING 
blead. I delete /site but some things slip through the cracks. Another 
case is installing a rare build option like DEBUG_LEAKING_SCALARS over 
perl without it. Comparing "v5.21.4" to "v5.21.4" doesn't fix that.

There is another open bug with unthreaded (specifically no 
PERL_IMPLICIT_SYS) Win32 Dynaloader where Dynaloader passes "use 
absolute path to DLL" flag to Win32 API, then passes a relative path 
with ".."s in it. MS says this is "undefined behavior" on their docs. 
Perl then loads a XS DLL from my system perl and usually promptly SEGVs 
in Perl_xs_apiversion_bootcheck while its trying to do a croak/exception 
that the apiversion mismatch. This happens with installed no 
PERL_IMPLICIT_SYS perl and uninstalled no PERL_IMPLICIT_SYS perl during 
make test.

Another case would be installing a PPM/RPM/binary precompiled XS module 
since you can't compile it yourself for whatever reason, and it isn't 
ABI compatible with your perl interp and you dont know that beforehand. 
I think ActiveState has a couple DB modules they distribute as closed 
source ("Sybase" I think is the name) since the developer SDK is mega 
$$$ so compile your own aint happening. Also QT/GTK/WX/massive C++ 
libraries XS modules are very likely to be distributed as binaries since 
  compling them is as easy as putting together a wide body plane from a 
box of parts.

> Still, for more unusual build options (such as DEBUG_LEAKING_SCALRS), it
> definitely is the case that the installation hierarchy and XS apiversion
> protection are indeed insufficient.  The usual advice is to install such
> builds under a prefix where the regular build won't find them.

That doesn't work when a system perl is already in %PATH. The current 
syntax for arch dirs is too liberal to catch mistakes like DEBUGGING 
(but some argue that DEBUGGING and non-DEBUGGING are always ABI 
compatible and its a Perl API guarantee, but I have a ticket open that 
shows they aren't).

> Some
> extra protection for those cases wouldn't be inappropriate.
> 
> However, I confess I don't understand how what you are proposing would
> address these problems, at least in the Linux world.

My idea is to compute a constant/bitfield which will catch the worst 
mistakes, mainly using sizeof(PerlInterpreter). Dont worry about 
unthreaded perls that have no PerlInterpreter, I have a solution for 
that. An integer "==" is cheap at runtime.

AFAIK Linux perl uses libperl.so, it is unversioned unlike Win32 perl. 
Activestate extensively used embedding of perl into other processes in 
prior years, so libperl is versioned as "perl5**.dll" on Win32. Most 
Unix folks never use "shrplib" AFAIK so libperl builds and embedding is 
almost never done on Unix. If the symbol doesn't exist (un vs threaded, 
or a wide enough spread between versions), loading the XS module will 
fail since there the dynamic linker won't search for another libperl 
since the XS module only requests "libperl.so" (or on linux the absolute 
path to libperl.so is hard coded in the XS module since relocatable INC 
perl is also never used on Linux) to be in the process. Or will code 
inside the XS module only fail the first time the missing function is 
called because of "lazy binding" feature? which is even worse, since 
code inside the XS module will not fail on loading/Dynaload time but 
randomly later on segv due to struct difference.

• Any empty members in struct xpvcv AKA CV that an XSUB can use? by bulk88
• Re: Any empty members in struct xpvcv AKA CV that an XSUB can use? by Father Chrysostomos
• Re: Any empty members in struct xpvcv AKA CV that an XSUB can use? by bulk88
• Re: Any empty members in struct xpvcv AKA CV that an XSUB can use? by Andy Dougherty
• Re: Any empty members in struct xpvcv AKA CV that an XSUB can use? by bulk88
• Re: Any empty members in struct xpvcv AKA CV that an XSUB can use? by Father Chrysostomos