develooper Front page | perl.perl5.porters | Postings from March 2003

ithreads and usemymalloc

Thread Next
Slaven Rezic
March 30, 2003 08:47
ithreads and usemymalloc
Message ID:
Here's a mail from the Convert-Binary-C author regarding a problem
using ithreads and usemymalloc=y. I think a threads expert should
comment on this...


Slaven Rezic -

    Berlin Perl Mongers -
------- Start of forwarded message -------
Message-ID: <006001c2f6c1$62322650$8145eed9@R2D2>
From: "Marcus Holland-Moritz" <>
To: <>
Cc: <>
References: <>
Subject: Re: FAIL Convert-Binary-C-0.12 i386-freebsd-thread-multi-64int 4.6-stable
Date: Sun, 30 Mar 2003 15:36:30 +0200
MIME-Version: 1.0
Content-Type: multipart/mixed;

This is a multi-part message in MIME format.

Content-Type: text/plain;
Content-Transfer-Encoding: 7bit

Hi Slaven,

thanks for testing my module and for the information you supplied.

After looking at the stack trace, however, I didn't have any clue
how this problem could be related to my module (except of course,
that it occurs when my module's test executes ;-).

Fortunately, I was able to reproduce the segfault on my linux
laptop after building a 5.8.0 binary with your configuration.
The segfault happened about one in five times when I was running
the 802 test. Somehow the stack trace pointed me to perl's memory
allocator, so I built another binary with -Dusemymalloc=n, and I
didn't get a single segfault in over 1000 runs of the 802 test.

Running the 802 test under gdb for a couple of times, I always got
different backtraces, but they all came from either perl's or the
system's memory allocator. My guess was that the combination of

  * threads,
  * perl's memory allocator _and_
  * the system's memory allocator

wasn't that good. To prove it, I wrote a tiny XS extension called
Threads::Crasher (which I have attached). I guess this is as few
code as possible to reproduce the problem reliably. The extension
crashes in exactly the same manner as Convert::Binary::C does on
a binary with -Dusemymalloc=y, and runs fine when -Dusemymalloc=n.

So I guess the problem is that you cannot safely use XS code that
relies on sysmalloc in multithreaded applications running under a
perl binary built with -Dusemymalloc=y. I tend to call that a bug
(in perl) rather than a feature, because it may not always be
possible to tweak an XS extension not to use sysmalloc.

For example, my module uses the unmodified source code of a pre-
processor library. Although I could (and I've also tested that,
to prove my module is not the culprit) modify the code to use
perl's memory allocator (at least in cases where perl is built
with its own malloc), I would not want to do that for obvious

If you think that this would be interesting for the porters
(I guess so, since I didn't find similar problems in the archive)
please tell me or feel free to forward my mail to the list.

Any other comments or suggestions are always really appreciated!


> This distribution has been tested as part of the cpan-testers
> effort to test as many new uploads to CPAN as possible.  See
> Please cc any replies to to keep other
> test volunteers informed and to prevent any duplicate effort.
> --
> make test fails with a threaded perl:
> $ perl5.8.0t -Mblib t/802_threads.t
> 1..4
> # parse_file (1) called
> # parse (2) called
> Segmentation fault(core dumped)
> The failures seem to happen at random subtests. Here's a stack trace
> from gdb:
> GNU gdb 4.18 (FreeBSD)
> Copyright 1998 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-unknown-freebsd"...
> warning: core file may not match specified executable file.
> Core was generated by `perl5.8.0'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /usr/lib/
> Reading symbols from /usr/lib/
> Reading symbols from /usr/lib/
> Reading symbols from /usr/lib/
> Reading symbols from /usr/perl5.8.0t/lib/5.8.0/i386-freebsd-thread-multi-64int/auto/Cwd/
> Reading symbols from /usr/local/src/CPAN/build/Convert-Binary-C-0.12/blib/arch/auto/Convert/Binary/C/
> Reading symbols from /usr/perl5.8.0t/lib/5.8.0/i386-freebsd-thread-multi-64int/auto/threads/
> Reading symbols from /usr/libexec/
> #0  0x806a9b0 in morecore (bucket=10) at malloc.c:1548
> 1548 ovp->ov_next = (union overhead *)((caddr_t)ovp + siz);
> (gdb) bt
> #0  0x806a9b0 in morecore (bucket=10) at malloc.c:1548
> #1  0x806978b in Perl_malloc (nbytes=1020) at malloc.c:1073
> #2  0x80e696b in S_more_xpv (my_perl=0x8533000) at sv.c:740
> #3  0x80e690e in S_new_xpv (my_perl=0x8533000) at sv.c:715
> #4  0x80fa614 in Perl_sv_dup (my_perl=0x8533000, sstr=0x825866c,
>     param=0xbfbfefa4) at sv.c:9059
> #5  0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8243588,
>     param=0xbfbfefa4) at sv.c:9199
> #6  0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x82435a0,
>     param=0xbfbfefa4) at sv.c:9199
> #7  0x80fb7c1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x824357c,
>     param=0xbfbfefa4) at sv.c:9283
> #8  0x80fb892 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8259ee4,
>     param=0xbfbfefa4) at sv.c:9290
> #9  0x80f96fd in Perl_gp_dup (my_perl=0x8533000, gp=0x825b948,
>     param=0xbfbfefa4) at sv.c:8698
> #10 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8259f38,
>     param=0xbfbfefa4) at sv.c:9142
> #11 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x8234ee4, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #12 0x80d19ba in Perl_he_dup (my_perl=0x8533000, e=0x8234f50, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:109
> #13 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x82435e8,
>     param=0xbfbfefa4) at sv.c:9231
> #14 0x80fac73 in Perl_sv_dup (my_perl=0x8533000, sstr=0x82692b4,
>     param=0xbfbfefa4) at sv.c:9140
> #15 0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d94c,
>     param=0xbfbfefa4) at sv.c:9199
> #16 0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d964,
>     param=0xbfbfefa4) at sv.c:9199
> #17 0x80fb7c1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d940,
>     param=0xbfbfefa4) at sv.c:9283
> #18 0x80f96fd in Perl_gp_dup (my_perl=0x8533000, gp=0x8348608,
>     param=0xbfbfefa4) at sv.c:8698
> #19 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8343444,
>     param=0xbfbfefa4) at sv.c:9142
> #20 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x83120b0, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #21 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x8310708,
>     param=0xbfbfefa4) at sv.c:9231
> #22 0x80f96a7 in Perl_gp_dup (my_perl=0x8533000, gp=0x8319848,
>     param=0xbfbfefa4) at sv.c:8696
> #23 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x83106f0,
>     param=0xbfbfefa4) at sv.c:9142
> #24 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x831202c, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #25 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x81cb0a4,
>     param=0xbfbfefa4) at sv.c:9231
> #26 0x80fac73 in Perl_sv_dup (my_perl=0x8533000, sstr=0x81dd444,
>     param=0xbfbfefa4) at sv.c:9140
> #27 0x80fdb0f in perl_clone (proto_perl=0x81ca000, flags=2) at sv.c:9932
> #28 0x28323191 in Perl_ithread_create (my_perl=0x81ca000, obj=0x0,
>     classname=0x8312a08 "threads", init_function=0x81e417c, params=0x81e4194)
>     at threads.xs:386
> #29 0x283239d4 in XS_threads_new (my_perl=0x81ca000, cv=0x8377fec)
> ---Type <return> to continue, or q <return> to quit---
>     at threads.xs:557
> #30 0x80e437e in Perl_pp_entersub (my_perl=0x81ca000) at pp_hot.c:2773
> #31 0x80c21ad in Perl_runops_debug (my_perl=0x81ca000) at dump.c:1398
> #32 0x806364b in S_run_body (my_perl=0x81ca000, oldscope=1) at perl.c:1681
> #33 0x8063099 in perl_run (my_perl=0x81ca000) at perl.c:1600
> #34 0x805e247 in main (argc=3, argv=0xbfbff248, env=0xbfbff258)
>     at perlmain.c:85
> #35 0x805e0d9 in _start ()
> With perl-5.8.1-to-be, I get a similar error.
> Regards,
> Slaven
> --
> Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
>   Platform:
>     osname=freebsd, osvers=4.6-stable, archname=i386-freebsd-thread-multi-64int
>     uname='freebsd 4.6-stable freebsd 4.6-stable #15: sat jul 27 09:32:28 cest 2002 i386 '
>     config_args='-Doptimize=-g -DPERL_DEBUGGING_MSTATS -Dusemymalloc=y -D usethreads=define -Dprefix=/usr/perl5.8.0t -D
usedevel=define -Dusemorebits -de -D hintfile=myfreebsd'
>     hint=recommended, useposix=true, d_sigaction=define
>     usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
>     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
>     use64bitint=define use64bitall=undef uselongdouble=undef
>     usemymalloc=y, bincompat5005=undef
>   Compiler:
>     cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -DDEBUGGING -fno-strict-aliasing -I/usr/local/include',
>     optimize='-g -DPERL_DEBUGGING_MSTATS',
>     cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -DDEBUGGING -fno-strict-aliasing -I/usr/local/include'
>     ccversion='', gccversion='2.95.4 20020320 [FreeBSD]', gccosandvers=''
>     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
>     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
>     ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
>     alignbytes=4, prototype=define
>   Linker and Libraries:
>     ld='cc', ldflags ='-pthread -Wl,-E  -L/usr/local/lib'
>     libpth=/usr/lib /usr/local/lib
>     libs=-lgdbm -ldb -lm -lc_r -lcrypt -lutil
>     perllibs=-lm -lc_r -lcrypt -lutil
>     libc=, so=so, useshrplib=false, libperl=libperl.a
>     gnulibc_version=''
>   Dynamic Linking:
>     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
>     cccdlflags='-DPIC -fpic', lddlflags='-shared  -L/usr/local/lib'

Content-Type: application/x-compressed;
Content-Transfer-Encoding: base64
Content-Disposition: attachment;



------- End of forwarded message -------

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About