develooper Front page | perl.perl5.porters | Postings from May 2013

[perl #117823] <glob*> under 'use threads;' segfaults

From:
Thomas Rast
Date:
May 1, 2013 18:07
Subject:
[perl #117823] <glob*> under 'use threads;' segfaults
Message ID:
rt-3.6.HEAD-28177-1367409331-141.117823-75-0@perl.org
# New Ticket Created by  Thomas Rast 
# Please include the string:  [perl #117823]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=117823 >


It seems <glob*> contains a race when used in parallel in threads.  This
test case reliably spews a few errors, and then segfaults, when I run it
on perl v5.16.2 (stock opensuse 12.3):

----- 8< -----
#!/usr/bin/perl

use warnings;
use strict;

use threads;
use threads::shared;

my $nthread = 20;

sub work {
    foreach (1..10000) {
	my @files = <*.supp>;
    }
}

my @threads;
foreach my $i (1..$nthread) {
    push @threads, threads->create(\&work);
}

foreach my $t (@threads) {
    $t->join();
}
----- >8 -----

There are two kinds of errors shown:

a) Attempt to free unreferenced scalar: SV 0xd1e738, Perl interpreter: 0xdf8900 at /home/thomas/tmp/perl-thread-glob-test.pl line 13.

b) Thread 1 terminated abnormally: sv_upgrade from type 255 down to type 11 at perl-thread-glob-test.pl line 13.

Followed by a segmentation fault or (rarely) a bus error.

It does not matter whether the glob actually matches any files, but
running in a directory where it does match something greatly increases
the frequency of (a) errors, whereas running in a directory where it
doesn't greatly increases the frequency of (b) errors.

The segfault looks like this:

  #0  0x0000000000499720 in Perl_hv_common (my_perl=0x8f7e80, hv=0x889720, keysv=keysv@entry=0x0, 
      key=key@entry=0x7fffe8001630 "0", klen=<optimized out>, flags=0, action=action@entry=48, 
      val=val@entry=0x0, hash=1849449579, hash@entry=0) at hv.c:638
  #1  0x000000000049b57e in Perl_hv_common_key_len (my_perl=my_perl@entry=0x8f7e80, hv=<optimized out>, 
      key=key@entry=0x7fffe8001630 "0", klen_i32=<optimized out>, action=action@entry=48, 
      val=val@entry=0x0, hash=hash@entry=0) at hv.c:333
  #2  0x00007ffff68b55d5 in iterate (my_perl=0x8f7e80, globber=0x7ffff68b5c60 <csh_glob>) at Glob.xs:84
  #3  0x00000000004e7225 in Perl_pp_glob (my_perl=0x8f7e80) at pp_sys.c:379
  #4  0x000000000049ee16 in Perl_runops_standard (my_perl=0x8f7e80) at run.c:41
  #5  0x0000000000436deb in Perl_call_sv (my_perl=0x8f7e80, sv=0x9baba0, flags=11) at perl.c:2705
  #6  0x00007ffff6ece0c9 in ?? ()
     from /usr/lib/perl5/5.16.2/x86_64-linux-thread-multi/auto/threads/threads.so
  #7  0x00007ffff7488e0f in start_thread () from /lib64/libpthread.so.0
  #8  0x00007ffff71bc7dd in clone () from /lib64/libc.so.6

or sometimes just starts at the iterate() frame:

  #0  0x00007ffff68b569b in iterate (my_perl=0xcbad60, globber=0x7ffff68b5c60 <csh_glob>) at Glob.xs:96
  #1  0x00000000004e7225 in Perl_pp_glob (my_perl=0xcbad60) at pp_sys.c:379
  #2  0x000000000049ee16 in Perl_runops_standard (my_perl=0xcbad60) at run.c:41
  #3  0x0000000000436deb in Perl_call_sv (my_perl=0xcbad60, sv=0xd7c690, flags=11) at perl.c:2705
  #4  0x00007ffff6ece0c9 in ?? ()
     from /usr/lib/perl5/5.16.2/x86_64-linux-thread-multi/auto/threads/threads.so
  #5  0x00007ffff7488e0f in start_thread () from /lib64/libpthread.so.0
  #6  0x00007ffff71bc7dd in clone () from /lib64/libc.so.6


vincent on #perl ran the test script on many perl versions (thanks!) and
reported this result:

  $ for p in ~/perl/builds/bin/perl5*thr* ; do echo $p ; $p x.pl ; done
  /home/vince/perl/builds/bin/perl5.10.0-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.10.1-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.12.0-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.12.1-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.12.2-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.12.3-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.12.4-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.12.5-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.14.0-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.14.1-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.14.2-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.14.3-dbg-psn-shr-thr-64
  /home/vince/perl/builds/bin/perl5.16.0-dbg-psn-shr-thr-64
  perl5.16.0-dbg-psn-shr-thr-64: Glob.xs:232: csh_glob: Assertion `((svtype)((entries)->sv_flags & 0xff)) != SVt_PVAV' failed.
  Aborted
  /home/vince/perl/builds/bin/perl5.16.1-dbg-psn-shr-thr-64
  perl5.16.1-dbg-psn-shr-thr-64: Glob.xs:232: csh_glob: Assertion `((svtype)((entries)->sv_flags & 0xff)) != SVt_PVAV' failed.
  Aborted
  /home/vince/perl/builds/bin/perl5.16.2-dbg-psn-shr-thr-64
  perl5.16.2-dbg-psn-shr-thr-64: Glob.xs:232: csh_glob: Assertion `((svtype)((entries)->sv_flags & 0xff)) != SVt_PVAV' failed.
  perl5.16.2-dbg-psn-shr-thr-64: Glob.xs:232: csh_glob: Assertion `((svtype)((entries)->sv_flags & 0xff)) != SVt_PVAV' failed.
  Aborted
  /home/vince/perl/builds/bin/perl5.17.3-dbg-psn-shr-thr-64
  perl5.17.3-dbg-psn-shr-thr-64: Glob.xs:232: csh_glob: Assertion `((svtype)((entries)->sv_flags & 0xff)) != SVt_PVAV' failed.
  Aborted
  /home/vince/perl/builds/bin/perl5.17.6-dbg-psn-shr-thr-64
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.17.7-dbg-psn-shr-thr-64
  Attempt to free nonexistent shared string ' ��', Perl interpreter: 0x1a4c5c0 at x.pl line 13.
  Segmentation fault
  /home/vince/perl/builds/bin/perl5.8.0-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.1-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.2-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.3-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.4-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.5-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.6-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.7-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.8-dbg-shr-thr-64
  /home/vince/perl/builds/bin/perl5.8.9-dbg-psn-shr-thr-64


--- perlbug -d ---
---
Flags:
    category=core
    severity=low
---
This perlbug was built using Perl 5.16.2 - Mon Mar 11 11:01:56 UTC 2013
It is being executed now by  Perl 5.16.2 - Mon Mar 11 10:54:42 UTC 2013.

Site configuration information for perl 5.16.2:

Configured by abuild at Mon Mar 11 10:54:42 UTC 2013.

Summary of my perl5 (revision 5 version 16 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=3.4.6-2.10-xen, archname=x86_64-linux-thread-multi
    uname='linux build05 3.4.6-2.10-xen #1 smp thu jul 26 09:36:26 utc 2012 (641c197) x86_64 x86_64 x86_64 gnulinux '
    config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector'
    ccversion='', gccversion='4.7.2 20130108 [gcc-4_7-branch revision 195012]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
    libpth=/lib64 /usr/lib64 /usr/local/lib64
    libs=-lm -ldl -lcrypt -lpthread
    perllibs=-lm -ldl -lcrypt -lpthread
    libc=/lib64/libc-2.17.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.16.2/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector'

Locally applied patches:
    

---
@INC for perl 5.16.2:
    /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.2
    /usr/lib/perl5/vendor_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.16.2
    /usr/lib/perl5/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/5.16.2
    /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.16.2
    /usr/lib/perl5/site_perl
    .

---
Environment for perl 5.16.2:
    HOME=/home/thomas
    LANG=en_US.UTF-8
    LANGUAGE=
    LC_NUMERIC=POSIX
    LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib64:/home/thomas/.local/lib:/home/thomas/.local/lib64:/home/thomas/.local/lib64/nest
    LOGDIR (unset)
    PATH=/home/thomas/.local/bin:/usr/lib64/mpi/gcc/openmpi/bin:/home/thomas/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/thomas/.local/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash


-- 
Thomas Rast
trast@{inf,student}.ethz.ch




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About