develooper Front page | perl.perl5.porters | Postings from September 2015

[perl #126145] Problem with stack moving fix for Perl_load_module

Thread Next
From:
Michael Schout
Date:
September 24, 2015 12:02
Subject:
[perl #126145] Problem with stack moving fix for Perl_load_module
Message ID:
rt-4.0.18-26574-1443032460-747.126145-75-0@perl.org
# New Ticket Created by  Michael Schout 
# Please include the string:  [perl #126145]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=126145 >


Since upgrading to 5.20, we started seeing our modperl process crash
while pre-compiling a long list of modules in our code.  We have a chunk
of code that uses File::Find and converts the filenames it finds into
module names, then uses Class::Load to load each module in turn:

  for my $mod (@modules) {
      unless (is_class_loaded($mod)) {
          eval "require $mod;";
      }
   }

This crashes on my codebase under CentOS 7 since 5.20.

Here is one such crash with 5.20.3 with DEBUGGING enabled:

*** Error in `perl': free(): invalid pointer: 0x000000000adb5978 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7d1fd)[0x7f16c3dd01fd]
perl(Perl_parser_free+0xca)[0x45a78a]
perl(Perl_leave_scope+0xa48)[0x5500f8]
perl(Perl_pp_leaveeval+0x1eb)[0x56e0bb]
perl(Perl_runops_debug+0x43)[0x4c8b23]
perl(Perl_call_sv+0x5f1)[0x4406d1]
/home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Sub/Attribute/Attribute.so(+0x271b)[0x7f16ba2cb71b]
perl[0x4d2b38]
perl(Perl_mg_free+0x3e)[0x4d383e]
perl(Perl_sv_clear+0x807)[0x50f8e7]
perl(Perl_sv_free2+0x5d)[0x51045d]
perl(Perl_leave_scope+0x12ab)[0x55095b]
perl(Perl_block_end+0x3d6)[0x42c676]
perl(Perl_yyparse+0x127e)[0x483dae]
perl[0x5546d8]
perl(Perl_pp_require+0x12fa)[0x56b10a]
perl(Perl_runops_debug+0x43)[0x4c8b23]
perl(Perl_call_sv+0x5f1)[0x4406d1]
perl(Perl_call_list+0x465)[0x443be5]
perl[0x41f3e4]
perl(Perl_newATTRSUB_x+0xa6f)[0x43819f]
perl(Perl_utilize+0x3b0)[0x43ba30]
perl(Perl_yyparse+0x218c)[0x484cbc]
perl[0x5546d8]
perl(Perl_pp_require+0x12fa)[0x56b10a]
perl(Perl_runops_debug+0x43)[0x4c8b23]
perl(Perl_call_sv+0x5f1)[0x4406d1]
perl(Perl_call_list+0x465)[0x443be5]
perl[0x41f3e4]
perl(Perl_newATTRSUB_x+0xa6f)[0x43819f]
perl(Perl_utilize+0x3b0)[0x43ba30]
perl(Perl_yyparse+0x218c)[0x484cbc]
perl[0x5546d8]
perl(Perl_pp_require+0x12fa)[0x56b10a]
perl(Perl_runops_debug+0x43)[0x4c8b23]
perl(Perl_call_sv+0x5f1)[0x4406d1]
perl(Perl_call_list+0x465)[0x443be5]
perl[0x41f3e4]
perl(Perl_newATTRSUB_x+0xa6f)[0x43819f]
perl(Perl_utilize+0x3b0)[0x43ba30]
perl(Perl_yyparse+0x218c)[0x484cbc]
perl[0x5546d8]
perl(Perl_pp_require+0x12fa)[0x56b10a]
perl(Perl_runops_debug+0x43)[0x4c8b23]
perl(perl_run+0x663)[0x44a603]
perl(main+0x11b)[0x41e8fb]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f16c3d74af5]
perl[0x41e935]
======= Memory map: ========
00400000-006bd000 r-xp 00000000 fd:01 2200335                            /home/mschout/test/build/bin/perl
008bc000-008c1000 r--p 002bc000 fd:01 2200335                            /home/mschout/test/build/bin/perl
008c1000-008c4000 rw-p 002c1000 fd:01 2200335                            /home/mschout/test/build/bin/perl
008c4000-008c5000 rw-p 00000000 00:00 0 
02408000-0ade1000 rw-p 00000000 00:00 0                                  [heap]
7f16b301d000-7f16b3032000 r-xp 00000000 fd:01 67150711                   /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f16b3032000-7f16b3231000 ---p 00015000 fd:01 67150711                   /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f16b3231000-7f16b3232000 r--p 00014000 fd:01 67150711                   /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f16b3232000-7f16b3233000 rw-p 00015000 fd:01 67150711                   /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f16b3233000-7f16b3260000 r-xp 00000000 fd:01 67229680                   /home/mschout/test/build/lib/libGeoIP.so.1.5.0
7f16b3260000-7f16b345f000 ---p 0002d000 fd:01 67229680                   /home/mschout/test/build/lib/libGeoIP.so.1.5.0
7f16b345f000-7f16b3460000 r--p 0002c000 fd:01 67229680                   /home/mschout/test/build/lib/libGeoIP.so.1.5.0
7f16b3460000-7f16b3462000 rw-p 0002d000 fd:01 67229680                   /home/mschout/test/build/lib/libGeoIP.so.1.5.0
7f16b3462000-7f16b3477000 r-xp 00000000 fd:01 105018346                  /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Geo/IP/IP.so
7f16b3477000-7f16b3676000 ---p 00015000 fd:01 105018346                  /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Geo/IP/IP.so
7f16b3676000-7f16b3677000 r--p 00014000 fd:01 105018346                  /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Geo/IP/IP.so
7f16b3677000-7f16b3678000 rw-p 00015000 fd:01 105018346                  /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Geo/IP/IP.so
7f16b3678000-7f16b3691000 r-xp 00000000 fd:01 74911406                   /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Text/CSV_XS/CSV_XS.so
7f16b3691000-7f16b3890000 ---p 00019000 fd:01 74911406                   /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Text/CSV_XS/CSV_XS.so
7f16b3890000-7f16b3891000 r--p 00018000 fd:01 74911406                   /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Text/CSV_XS/CSV_XS.so
7f16b3891000-7f16b3892000 rw-p 00019000 fd:01 74911406                   /home/mschout/test/build/lib/perl5/site_perl/5.20.3/x86_64-linux/auto/Text/CSV_XS/CSV_XS.so
7f16b3892000-7f16b38b6000 r-xp 00000000 fd:01 67237851                   /usr/lib64/liblzma.so.5.0.99
7f16b38b6000-7f16b3ab5000 ---p 00024000 fd:01 67237851                   /usr/lib64/liblzma.so.5.0.99
7f16b3ab5000-7f16b3ab6000 r--p 00023000 fd:01 67237851                   /usr/lib64/liblzma.so.5.0.99
7f16b3ab6000-7f16b3ab7000 rw-p 00024000 fd:01 67237851                   /usr/lib64/liblzma.so.5.0.99
7f16b3ab7000-7f16b3b16000 r-xp 00000000 fd:01 67251479                   /usr/lib64/libpcre.so.1.2.0
7f16b3b16000-7f16b3d16000 ---p 0005f000 fd:01 67251479                   /usr/lib64/libpcre.so.1.2.0
7f16b3d16000-7f16b3d17000 r--p 0005f000 fd:01 67251479                   /usr/lib64/libpcre.so.1.2.0
7f16b3d17000-7f16b3d18000 rw-p 00060000 fd:01 67251479                   /usr/lib64/libpcre.so.1.2.0
7f16b3d18000-7f16b3d39000 r-xp 00000000 fd:01 67251496                   /usr/lib64/libselinux.so.1
7f16b3d39000-7f16b3f39000 ---p 00021000 fd:01 67251496                   /usr/lib64/libselinux.so.1
7f16b3f39000-7f16b3f3a000 r--p 00021000 fd:01 67251496                   /usr/lib64/libselinux.so.1
7f16b3f3a000-7f16b3f3b000 rw-p 00022000 fd:01 67251496                   /usr/lib64/libselinux.so.1
7f16b3f3b000-7f16b3f3d000 rw-p 00000000 00:00 0 
7f16b3f3d000-7f16b3f53000 r-xp 00000000 fd:01 67150751                   /usr/lib64/libresolv-2.17.so
7f16b3f53000-7f16b4153000 ---p 00016000 fd:01 67150751                   /usr/lib64/libresolv-2.17.so
7f16b4153000-7f16b4154000 r--p 00016000 fd:01 67150751                   /usr/lib64/libresolv-2.17.so
7f16b4154000-7f16b4155000 rw-p 00017000 fd:01 67150751                   /usr/lib64/libresolv-2.17.so
7f16b4155000-7f16b4157000 rw-p 00000000 00:00 0 
7f16b4157000-7f16b415a000 r-xp 00000000 fd:01 67328820                   /usr/lib64/libkeyutils.so.1.5
7f16b415a000-7f16b4359000 ---p 00003000 fd:01 67328820                   /usr/lib64/libkeyutils.so.1.5
7f16b4359000-7f16b435a000 r--p 00002000 fd:01 67328820                   /usr/lib64/libkeyutils.so.1.5

I bisected for where this broke, and the first bad commit is:

commit ebdc88085efa6fca8a1b0afaa388f0491bdccd5a
Author: Nicholas Clark <nick@ccl4.org>
Date:   Fri Oct 4 14:54:00 2013 +0200

    S_process_special_blocks() should use a new stack for BEGIN blocks.
    
    This avoids the stack moving underneath anything that directly or indirectly
    calls Perl_load_module().
    
    [Committer’s note:
    
     This fixes bug #119993.
    
     Furthermore, under STRESS_REALLOC, t/io/layers.t was crashing like this:
    
     $ ./perl -Ilib -e ' open(UTF, "<:raw:encoding(utf8)", 'tmp75851B') or die $!; ref #blahblahblahblahblahblahblahblahblah'
     Segmentation fault: 11
    
     (The comment seems to be necessary to make it crash.)
    
     It was happening because open() was causing a module to be loaded
     while the arguments to open() were still on the stack.
    ]

diff --git a/op.c b/op.c
index 030039d..8a60800 100644
--- a/op.c
+++ b/op.c
@@ -7957,8 +7957,10 @@ S_process_special_blocks(pTHX_ I32 floor, const char *const fullname,
     if (*name == 'B') {
        if (strEQ(name, "BEGIN")) {
            const I32 oldscope = PL_scopestack_ix;
+            dSP;
            if (floor) LEAVE_SCOPE(floor);
            ENTER;
+            PUSHSTACKi(PERLSI_REQUIRE);
            SAVECOPFILE(&PL_compiling);
            SAVECOPLINE(&PL_compiling);
            SAVEVPTR(PL_curcop);
@@ -7968,6 +7970,7 @@ S_process_special_blocks(pTHX_ I32 floor, const char *const fullname,
            GvCV_set(gv,0);             /* cv has been hijacked */
            call_list(oldscope, PL_beginav);
 
+            POPSTACK;
            LEAVE;
        }
        else

I also tested this on OS X Darwin and it does NOT crash there.  So perhaps this
is linux specific?

I reversed the above commit against 5.20.3 and the problem goes away (although 
probably bringing back whatever problem that commit was attempting to solve). 
A quick skim of #119993 leads me to think that this commit was not an ideal fix.
If anyone has any insight as to how to improve the fix or why it might cause the
crash I am seeing I'd love to hear it.

It does seem to matter what modules are that are loaded to trigger the bug, so
I have not been able to create a simplified test case for this at this time.

Regards,
Michael Schout


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About