develooper Front page | perl.perl5.porters | Postings from October 2015

(version PP is not PP) RE: GCC builds failing many tests on Windows

Thread Next
From:
bulk 88
Date:
October 20, 2015 22:14
Subject:
(version PP is not PP) RE: GCC builds failing many tests on Windows
Message ID:
BAY182-W92DD7C73B730A6D6DB8DEDDF390@phx.gbl


> Date: Tue, 20 Oct 2015 13:50:19 +0100
> Subject: Re: GCC builds failing many tests on Windows
> From: steve.m.hay@googlemail.com
> To: bulk88@hotmail.com
> CC: perl5-porters@perl.org
> 
> I can't understand how this doesn't happen for you. It happens every
> time for me on two different machines (Windows 7 and Windows 8.1).
> 
> The build sequence with a straight "dmake" is:
> 
> miniperl.exe
> DynaLoader
> XS extensions
> perl.dll
> perl.exe
> non-XS extensions
> 
> The problem is that POSIX is an XS extension, but its Makefile.PL
> needs XSLoader, which is non-XS so hasn't been made yet.

So how does the Unix build resolve these dependencies in parallel mode? XSLoader isn't in write_buildcustomize.pl.


> The POSIX
> Makefile.PL fails every time with this error:
> 
> Can't locate XSLoader.pm in @INC (you may need to install the XSLoader
> module) (@INC contains: [...]) at C:\Dev\Git\perl\lib/B.pm line 26.
> BEGIN failed--compilation aborted at C:\Dev\Git\perl\lib/B.pm line 28.
> Compilation failed in require at
> C:\Dev\Git\perl\cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/vpp.pm
> line 954.
> 512 from ext/POSIX's Makefile.PL at ..\make_ext.pl line 523.

----------------------------------------------------
sub _find_magic_vstring {
    my $value = shift;
    my $tvalue = '';
    require B;<<<<<<<<<<<<<LINE 954
    my $sv = B::svref_2object(\$value);
    my $magic = ref($sv) eq 'B::PVMG' ? $sv->MAGIC : undef;
    while ( $magic ) {
    if ( $magic->TYPE eq 'V' ) {
        $tvalue = $magic->PTR;
        $tvalue =~ s/^v?(.+)$/v$1/;
        last;
    }
    else {
        $magic = $magic->MOREMAGIC;
    }
    }
    return $tvalue;
}
----------------------------------------------------

B::svref_2object is implemented in B.xs. There is no running/loading of XS code inside miniperl. Plus "version.pm pure perl" is not pure perl if it requires B (XS) to execute, that dep on B:: comes from https://bitbucket.org/jpeacock/version/commits/ba0f930f83c45f6c6972703fc533448b70cc9081 in vpp.pm from 2010 but  I can't find a ticket or context behind that commit

> 
> It's the WriteConstants() call that fails. WriteMakefile() has already
> run successfully by this stage, so a Makefile has been written and
> POSIX.dll builds, but const-c.inc and const-xs.inc end up zero size.
> (Crucially, they *exist* so the #include in POSIX.c doesn't fail,
> hence the build continues.)

I tried something different a
dmake -P3  ..\perl523.lib ..\DynaLoader.obj ..\perlglob.exe CCTYPE=MSVC120
which is the prereq list to build Extentions target followed by
..\miniperl.exe -I..\lib ..\make_ext.pl "MAKE=dmake" --dir=..\cpan --dir=..\dist --dir=..\ext ext/POSIX
to build just 1 XS module.
-------------------------------------------------------------------------------
C:\p523\src\win32>..\miniperl.exe -I..\lib ..\make_ext.pl "MAKE=dmake" --dir=..\
cpan --dir=..\dist --dir=..\ext ext/POSIX
Generating a dmake-style Makefile
Writing Makefile for POSIX
Can't locate B.pm in @INC (you may need to install the B module) (@INC contains:
 C:\p523\src\lib C:\p523\src\cpan\AutoLoader\lib C:\p523\src\dist\Carp\lib C:\p5
23\src\dist\PathTools C:\p523\src\dist\PathTools\lib C:\p523\src\cpan\ExtUtils-I
nstall\lib C:\p523\src\cpan\ExtUtils-MakeMaker\lib C:\p523\src\cpan\ExtUtils-Man
ifest\lib C:\p523\src\cpan\File-Path\lib C:\p523\src\ext\re C:\p523\src\dist\Ter
m-ReadLine\lib C:\p523\src\dist\Exporter\lib C:\p523\src\ext\File-Find\lib C:\p5
23\src\cpan\Text-Tabs\lib C:\p523\src\dist\constant\lib C:\p523\src\cpan\Text-Pa
rseWords\lib C:\p523\src\dist\ExtUtils-ParseXS\lib C:\p523\src\cpan\Getopt-Long\
lib C:\p523\src\cpan\parent\lib C:\p523\src\cpan\ExtUtils-Constant\lib .) at C:\
p523\src\cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/vpp.pm line 954.

512 from ext/POSIX's Makefile.PL at ..\make_ext.pl line 523.
Running Mkbootstrap for POSIX ()
C:\p523\src\miniperl.exe "-I..\..\lib" "-I..\..\lib" -MExtUtils::Command -e chmo
d -- 644 "POSIX.bs"
"..\..\miniperl.exe" "-I..\..\lib" "-I..\..\lib" "C:\p523\src\dist\ExtUtils-Pars
eXS\lib\ExtUtils\xsubpp" -noprototypes -typemap "..\..\lib\ExtUtils\typemap" -ty
pemap "typemap"  POSIX.xs > POSIX.xsc && C:\p523\src\miniperl.exe "-I..\..\lib"
"-I..\..\lib" -MExtUtils::Command -e mv -- POSIX.xsc POSIX.c
cl -c           -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO
_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_S
CRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -O1 -MD -Zi -DNDEBUG -GL
  -DVERSION=\"1.58\"    -DXS_VERSION=\"1.58\"  "-I..\..\lib\CORE"   POSIX.c
POSIX.c
"..\..\miniperl.exe" "-I..\..\lib" "-I..\..\lib" -MExtUtils::Mksymlists \
     -e "Mksymlists('NAME'=>\"POSIX\", 'DLBASE' => 'POSIX', 'DL_FUNCS' => {  },
'FUNCLIST' => [], 'IMPORTS' => {  }, 'DL_VARS' => []);"
link -out:..\..\lib\auto\POSIX\POSIX.dll -dll -nologo -nodefaultlib -debug -opt:
ref,icf -ltcg           -libpath:"c:\perl\lib\CORE"             -machine:x86 "/m
anifestdependency:type='Win32' name='Microsoft.Windows.Common-Controls' version=
'6.0.0.0' processorArchitecture='*' publicKeyToken='6595b64144ccf1df' language='
*'" -subsystem:console,"5.01" POSIX.obj   "..\..\lib\CORE\perl523.lib" oldnames.
lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib she
ll32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.l
ib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib -def:POSIX.def
   Creating library ..\..\lib\auto\POSIX\POSIX.lib and object ..\..\lib\auto\POS
IX\POSIX.exp
Generating code
Finished generating code
if exist ..\..\lib\auto\POSIX\POSIX.dll.manifest mt -nologo -manifest ..\..\lib\
auto\POSIX\POSIX.dll.manifest -outputresource:..\..\lib\auto\POSIX\POSIX.dll;2
if exist ..\..\lib\auto\POSIX\POSIX.dll.manifest del ..\..\lib\auto\POSIX\POSIX.
dll.manifest
C:\p523\src\miniperl.exe "-I..\..\lib" "-I..\..\lib" -MExtUtils::Command -e chmo
d -- 755 ..\..\lib\auto\POSIX\POSIX.dll

C:\p523\src\win32>
---------------------------------------------------------------------

The problem is this line is not inline with the Makefile.PL API http://perl5.git.perl.org/perl.git/blob/b9904204a39a06e70bac207805ef0ba22080bd35:/make_ext.pl#l523  non-zero exit code from Makefile.PL is fatal, do not proceed according to http://www.nntp.perl.org/group/perl.qa/2008/08/msg11236.html  that code is instead proceeding.  That warn line in make_ext.pl is from http://perl5.git.perl.org/perl.git/commitdiff/fc6784129696c3d1b807ce356dbd42133b41d86a which turned previously never capturing the exit code from Makefile.PL into a warning. I have posted a patch to fix this in https://rt.perl.org/Ticket/Display.html?id=126412

In parallel building there is so much mangled choppy text flowing through the console a human will probably not notice the warning. For automated testing, that warning is evil since it doesn't turn into a higher level fail status which stops the master/root Makefile.

With that patch and cluck(), I see the problem is related to EUMM/version.pm. miniperl.exe contains XS version.pm as xsubs. But somehow the POSIX Makefile.PL process is loading a version.pm PP version which calls _un_vstring/_find_magic_vstring which doesn't exist in the XS version of new() (XS version doesn't do a callback into B::). The alphabetic sorting in make_ext.pl accidentally means B is built before POSIX so everything accidentally magically works. IDK why I can build WIn32 dmake perl at all, since there is a die() from XSLoader/Dynaloader running on Miniperl, if I build Dynaloader, XSLoader, B, and then POSIX by hand on a clean (no Extensions_nonxs target run) tree.
------------------------------------------------------
C:\p523\src\win32>..\miniperl.exe -I..\lib ..\make_ext.pl "MAKE=dmake" --dir=..\
cpan --dir=..\dist --dir=..\ext ext/POSIX
Generating a dmake-style Makefile
Writing Makefile for POSIX
 at C:\p523\src\cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/vpp.pm li
ne 955.
        ExtUtils::MakeMaker::version::vpp::_find_magic_vstring("\x{5}\x{d}\x{5}"
) called at C:\p523\src\cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/v
pp.pm line 937
        ExtUtils::MakeMaker::version::vpp::_un_vstring("\x{5}\x{d}\x{5}") called
 at C:\p523\src\cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/vpp.pm li
ne 681
        ExtUtils::MakeMaker::version::vpp::new("version") called at C:\p523\src\
cpan\ExtUtils-MakeMaker\lib/ExtUtils/MakeMaker/version/vpp.pm line 806
        ExtUtils::MakeMaker::version::vpp::vcmp(version=HASH(0x444378), "\x{5}\x
{d}\x{5}", "") called at C:\p523\src\cpan\ExtUtils-Constant\lib/ExtUtils/Constan
t/ProxySubs.pm line 201
        ExtUtils::Constant::ProxySubs::WriteConstants("ExtUtils::Constant::Proxy
Subs", "C_SUBNAME", "constant", "C_FILE", "const-c.inc", "XS_FH", GLOB(0x20762b8
), "NAME", ...) called at C:\p523\src\cpan\ExtUtils-Constant\lib/ExtUtils/Consta
nt.pm line 530
        ExtUtils::Constant::WriteConstants("PROXYSUBS", HASH(0x2076030), "NAME",
 "POSIX", "NAMES", ARRAY(0xecda30)) called at Makefile.PL line 142
Can't load module B, dynamic loading not available in this perl.
  (You may need to build a new perl executable which either supports
  dynamic loading or has the B module statically linked into it.)
 at C:\p523\src\lib/B.pm line 28.
BEGIN failed--compilation aborted at C:\p523\src\lib/B.pm line 28.
Compilation failed in require at C:\p523\src\cpan\ExtUtils-MakeMaker\lib/ExtUtil
s/MakeMaker/version/vpp.pm line 956.
Unsuccessful Makefile.PL(ext/POSIX): code=65280 at ..\make_ext.pl line 526.

C:\p523\src\win32>
------------------------------------------------------
It is something to do with version.pm. I will need to research it further, perhaps EUMM needs fixing to better detect whether to use its bundled Version PP, or use miniperl's Version XS. And version on CPAN needs fixing since vpp is a lie, it is not pure perl.
Thread Next

nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About