develooper Front page | perl.perl5.porters | Postings from March 2003

ithreads and usemymalloc

Thread Next
From:
Slaven Rezic
Date:
March 30, 2003 08:47
Subject:
ithreads and usemymalloc
Message ID:
87y92wdcqh.fsf@vran.herceg.de
Here's a mail from the Convert-Binary-C author regarding a problem
using ithreads and usemymalloc=y. I think a threads expert should
comment on this...

Regards,
	Slaven

-- 
Slaven Rezic - slaven@rezic.de

    Berlin Perl Mongers - http://berliner.pm.org
------- Start of forwarded message -------
Message-ID: <006001c2f6c1$62322650$8145eed9@R2D2>
From: "Marcus Holland-Moritz" <mhx-cpan@gmx.net>
To: <slaven@rezic.de>
Cc: <cpan-testers@perl.org>
References: <200303272347.h2RNlmdO085655@vran.herceg.de>
Subject: Re: FAIL Convert-Binary-C-0.12 i386-freebsd-thread-multi-64int 4.6-stable
Date: Sun, 30 Mar 2003 15:36:30 +0200
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_NextPart_000_005D_01C2F6D2.22C66E70"

This is a multi-part message in MIME format.

------=_NextPart_000_005D_01C2F6D2.22C66E70
Content-Type: text/plain;
	charset="Windows-1252"
Content-Transfer-Encoding: 7bit

Hi Slaven,

thanks for testing my module and for the information you supplied.

After looking at the stack trace, however, I didn't have any clue
how this problem could be related to my module (except of course,
that it occurs when my module's test executes ;-).

Fortunately, I was able to reproduce the segfault on my linux
laptop after building a 5.8.0 binary with your configuration.
The segfault happened about one in five times when I was running
the 802 test. Somehow the stack trace pointed me to perl's memory
allocator, so I built another binary with -Dusemymalloc=n, and I
didn't get a single segfault in over 1000 runs of the 802 test.

Running the 802 test under gdb for a couple of times, I always got
different backtraces, but they all came from either perl's or the
system's memory allocator. My guess was that the combination of

  * threads,
  * perl's memory allocator _and_
  * the system's memory allocator

wasn't that good. To prove it, I wrote a tiny XS extension called
Threads::Crasher (which I have attached). I guess this is as few
code as possible to reproduce the problem reliably. The extension
crashes in exactly the same manner as Convert::Binary::C does on
a binary with -Dusemymalloc=y, and runs fine when -Dusemymalloc=n.

So I guess the problem is that you cannot safely use XS code that
relies on sysmalloc in multithreaded applications running under a
perl binary built with -Dusemymalloc=y. I tend to call that a bug
(in perl) rather than a feature, because it may not always be
possible to tweak an XS extension not to use sysmalloc.

For example, my module uses the unmodified source code of a pre-
processor library. Although I could (and I've also tested that,
to prove my module is not the culprit) modify the code to use
perl's memory allocator (at least in cases where perl is built
with its own malloc), I would not want to do that for obvious
reasons.

If you think that this would be interesting for the porters
(I guess so, since I didn't find similar problems in the archive)
please tell me or feel free to forward my mail to the list.

Any other comments or suggestions are always really appreciated!

Regards,
Marcus


> This distribution has been tested as part of the cpan-testers
> effort to test as many new uploads to CPAN as possible.  See
> http://testers.cpan.org/
>
> Please cc any replies to cpan-testers@perl.org to keep other
> test volunteers informed and to prevent any duplicate effort.
>
> --
> make test fails with a threaded perl:
>
> $ perl5.8.0t -Mblib t/802_threads.t
> 1..4
> # parse_file (1) called
> # parse (2) called
> Segmentation fault(core dumped)
>
> The failures seem to happen at random subtests. Here's a stack trace
> from gdb:
>
> GNU gdb 4.18 (FreeBSD)
> Copyright 1998 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-unknown-freebsd"...
>
> warning: core file may not match specified executable file.
> Core was generated by `perl5.8.0'.
> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /usr/lib/libm.so.2...done.
> Reading symbols from /usr/lib/libc_r.so.4...done.
> Reading symbols from /usr/lib/libcrypt.so.2...done.
> Reading symbols from /usr/lib/libutil.so.3...done.
> Reading symbols from /usr/perl5.8.0t/lib/5.8.0/i386-freebsd-thread-multi-64int/auto/Cwd/Cwd.so...done.
> Reading symbols from /usr/local/src/CPAN/build/Convert-Binary-C-0.12/blib/arch/auto/Convert/Binary/C/C.so...done.
> Reading symbols from /usr/perl5.8.0t/lib/5.8.0/i386-freebsd-thread-multi-64int/auto/threads/threads.so...done.
> Reading symbols from /usr/libexec/ld-elf.so.1...done.
> #0  0x806a9b0 in morecore (bucket=10) at malloc.c:1548
> 1548 ovp->ov_next = (union overhead *)((caddr_t)ovp + siz);
> (gdb) bt
> #0  0x806a9b0 in morecore (bucket=10) at malloc.c:1548
> #1  0x806978b in Perl_malloc (nbytes=1020) at malloc.c:1073
> #2  0x80e696b in S_more_xpv (my_perl=0x8533000) at sv.c:740
> #3  0x80e690e in S_new_xpv (my_perl=0x8533000) at sv.c:715
> #4  0x80fa614 in Perl_sv_dup (my_perl=0x8533000, sstr=0x825866c,
>     param=0xbfbfefa4) at sv.c:9059
> #5  0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8243588,
>     param=0xbfbfefa4) at sv.c:9199
> #6  0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x82435a0,
>     param=0xbfbfefa4) at sv.c:9199
> #7  0x80fb7c1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x824357c,
>     param=0xbfbfefa4) at sv.c:9283
> #8  0x80fb892 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8259ee4,
>     param=0xbfbfefa4) at sv.c:9290
> #9  0x80f96fd in Perl_gp_dup (my_perl=0x8533000, gp=0x825b948,
>     param=0xbfbfefa4) at sv.c:8698
> #10 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8259f38,
>     param=0xbfbfefa4) at sv.c:9142
> #11 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x8234ee4, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #12 0x80d19ba in Perl_he_dup (my_perl=0x8533000, e=0x8234f50, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:109
> #13 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x82435e8,
>     param=0xbfbfefa4) at sv.c:9231
> #14 0x80fac73 in Perl_sv_dup (my_perl=0x8533000, sstr=0x82692b4,
>     param=0xbfbfefa4) at sv.c:9140
> #15 0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d94c,
>     param=0xbfbfefa4) at sv.c:9199
> #16 0x80fb1f1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d964,
>     param=0xbfbfefa4) at sv.c:9199
> #17 0x80fb7c1 in Perl_sv_dup (my_perl=0x8533000, sstr=0x833d940,
>     param=0xbfbfefa4) at sv.c:9283
> #18 0x80f96fd in Perl_gp_dup (my_perl=0x8533000, gp=0x8348608,
>     param=0xbfbfefa4) at sv.c:8698
> #19 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x8343444,
>     param=0xbfbfefa4) at sv.c:9142
> #20 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x83120b0, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #21 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x8310708,
>     param=0xbfbfefa4) at sv.c:9231
> #22 0x80f96a7 in Perl_gp_dup (my_perl=0x8533000, gp=0x8319848,
>     param=0xbfbfefa4) at sv.c:8696
> #23 0x80facb6 in Perl_sv_dup (my_perl=0x8533000, sstr=0x83106f0,
>     param=0xbfbfefa4) at sv.c:9142
> #24 0x80d1ad3 in Perl_he_dup (my_perl=0x8533000, e=0x831202c, shared=1 '\001',
>     param=0xbfbfefa4) at hv.c:118
> #25 0x80fb41f in Perl_sv_dup (my_perl=0x8533000, sstr=0x81cb0a4,
>     param=0xbfbfefa4) at sv.c:9231
> #26 0x80fac73 in Perl_sv_dup (my_perl=0x8533000, sstr=0x81dd444,
>     param=0xbfbfefa4) at sv.c:9140
> #27 0x80fdb0f in perl_clone (proto_perl=0x81ca000, flags=2) at sv.c:9932
> #28 0x28323191 in Perl_ithread_create (my_perl=0x81ca000, obj=0x0,
>     classname=0x8312a08 "threads", init_function=0x81e417c, params=0x81e4194)
>     at threads.xs:386
> #29 0x283239d4 in XS_threads_new (my_perl=0x81ca000, cv=0x8377fec)
> ---Type <return> to continue, or q <return> to quit---
>     at threads.xs:557
> #30 0x80e437e in Perl_pp_entersub (my_perl=0x81ca000) at pp_hot.c:2773
> #31 0x80c21ad in Perl_runops_debug (my_perl=0x81ca000) at dump.c:1398
> #32 0x806364b in S_run_body (my_perl=0x81ca000, oldscope=1) at perl.c:1681
> #33 0x8063099 in perl_run (my_perl=0x81ca000) at perl.c:1600
> #34 0x805e247 in main (argc=3, argv=0xbfbff248, env=0xbfbff258)
>     at perlmain.c:85
> #35 0x805e0d9 in _start ()
>
>
> With perl-5.8.1-to-be, I get a similar error.
>
> Regards,
> Slaven
>
> --
>
> Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
>   Platform:
>     osname=freebsd, osvers=4.6-stable, archname=i386-freebsd-thread-multi-64int
>     uname='freebsd vran.herceg.de 4.6-stable freebsd 4.6-stable #15: sat jul 27 09:32:28 cest 2002
root@vran.herceg.de:usrlocalsrcfreebsd-4srcsyscompilevran i386 '
>     config_args='-Doptimize=-g -DPERL_DEBUGGING_MSTATS -Dusemymalloc=y -D usethreads=define -Dprefix=/usr/perl5.8.0t -D
usedevel=define -Dusemorebits -de -D hintfile=myfreebsd'
>     hint=recommended, useposix=true, d_sigaction=define
>     usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
>     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
>     use64bitint=define use64bitall=undef uselongdouble=undef
>     usemymalloc=y, bincompat5005=undef
>   Compiler:
>     cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -DDEBUGGING -fno-strict-aliasing -I/usr/local/include',
>     optimize='-g -DPERL_DEBUGGING_MSTATS',
>     cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -DDEBUGGING -fno-strict-aliasing -I/usr/local/include'
>     ccversion='', gccversion='2.95.4 20020320 [FreeBSD]', gccosandvers=''
>     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
>     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
>     ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
>     alignbytes=4, prototype=define
>   Linker and Libraries:
>     ld='cc', ldflags ='-pthread -Wl,-E  -L/usr/local/lib'
>     libpth=/usr/lib /usr/local/lib
>     libs=-lgdbm -ldb -lm -lc_r -lcrypt -lutil
>     perllibs=-lm -lc_r -lcrypt -lutil
>     libc=, so=so, useshrplib=false, libperl=libperl.a
>     gnulibc_version=''
>   Dynamic Linking:
>     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
>     cccdlflags='-DPIC -fpic', lddlflags='-shared  -L/usr/local/lib'
>

------=_NextPart_000_005D_01C2F6D2.22C66E70
Content-Type: application/x-compressed;
	name="Threads-Crasher-0.01.tar.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="Threads-Crasher-0.01.tar.gz"

H4sICIjmhj4CA1RocmVhZHMtQ3Jhc2hlci0wLjAxLnRhcgDtWW1v2zYQ9tfyV1zTYnHSRJX8EgN2
HNSJ1dRbYgd+2RqgQCDLdMRFFl2RimME/u87UpLtpO42bJ2BDDrEkCgej0fynrvjpe+F1BmJw7PQ
ER4ND03DtN7nfiyZZsmslMv4NK1K2VRP0yqV9DOhnFk5Klnlo2LBwneraBVLOSjntkCRkE4IkJt4
D3/BR0OR+99Rf9P5u6phuD/s/C3TPIrPe+P5l4vF9PzNolVA/lKxcpQDMzv//5zesMD1oxGFYyFH
Phsa3gkh7/cJAEiPCcBPoRPOIeSRZAEF3AYBYi4mju9z9wBmHnM9EJROhB7DgQWShmMaUpgx6aEU
ClMa+jChE46C9DhH8pDsvycEN18yF+45G0FI3SgU7J7euLEx5pUs8GGPPKJszbOPUqAO8ex5sJT5
wF4Nu9k4j6wngM1NkvxDK+Ybh5TmlTKquSBEi12y6VY8n5qb6RFcaVI3a8CO1YQ1ePeOIRMud9NM
VqzRgrxc/CcNYzrZCv4L5dT/l0sFE+OEVTbLlQz/26Cp4945txQSO6hWk7OvEVwwNOeBc8GdUdoW
MmSujN/vnVDA11n+7a92t9fqtOFDq9dAw1cPBQ2EKfauJGDXkrUOu8rQdmtkyLlEqc70Gw0g5a4R
q0ZyGW0f/w9iK/gvFkop/o+KZY1/y8zwv934v2N/7tvdtuHtkNU3FbiffvncG5yqLxvjZo2Qy05z
cGEjwJ/D+dVV4+yXxvnGrleEXHU7/U7/+sruVcFuN04v7HgKkk4Rx1tFZ52mXV22UlryZb7iX+Nf
UiGNqb+1/N9a3v+wofP/UqmQ4X87y6eYomsbiMN6H88+eduYErg8wB3D3Lg9uLzpf+rajWYP6idQ
QvCf2uetNjzC1HcCUEYkVM864wK5+F0e3t4cnvzOWYDJOqbXmM9PcVhAZ6ku8OUn6Yi7A2SEBVhg
GOtiUIiIhqA4QKXqkzm8dcJbdC3CY2OpkvZpqPL3nTepX4C84tgDF28OdPQl2FFMz1dYra55ke/L
CKmMwmApxXoxyf7fxf+lc0fHzKfG1cUW8F8oW5UU/wWraOn4b1kZ/reF/7KBbtcsxvi2H+RAMh9B
oaxA/RLgn/FgzG4RejEsjo/tdhNb9Xpd/RBLTAD+OTCKJgjICR9FPoUR3voDleBLFtyCoLdjJ/Il
QosPfToRMPNoUF0KAdiHMAoCxevEZYMhC1T9YRgxHKYrCodNVGcyj2sA9fmTwZFQQxMnsqEH/dLn
HtAHSQPBODopz5HLmoakk2dVCsWSVD5W0lpjmPMofKIeLjzWUHmzeCqG3o/PAkgrJTgG3ROP/JGW
Qh9wPKOBS2Ml0mWreolynUBxreiSnixCRK5LKbrHPLrO0AnRHTuo/N5TpmST8TgCdUtjPBLKJbv0
6Y6gJx/rpeuFuHwy9amk/vy7wvURp+fKcWDApad0RinjyD+A4VyrP3PmBrTU8gOUNo3CKUf7wbGS
azkrm0jWrt7xDjqknnPP1NayQAuijmBqK1AgoAzB0GYMrQ5B4yMkCnwqBEaT2DYfUQ+WnP0CI8vj
0ocnxvr69Wv1g+vOABpdG9qdPgx6rfY5NCAOLXYTruzuBZy22o3u9XIAbh/qQkGJ1po5Q36PS1Kr
iFxlJsIwDM2uNQM8XibjyEBUZWpNw9RyF0C/wu58d5Oi6Y5fP7ezmSPWobC0sWe1NSMdnVjcurX1
7POPjcFFv4dgGMFZt9H7ZPc0Dr+xwbW9XhDq4wb8Y02VyI0Q26CsOpY1hVf2/CdKanvAzf4tZJKm
8SuP2rYbl/bykoDZyO7zoL97gFxJqeHmY7dzqblWJTDd3zn92T7rr0lJ+3mcHBgc2V7i1WNz/G+0
Wx/tXn9L+b9ZWdX/y/r+Xyrgpyz+b4FWdk5WVR+SGgBZywRJ8l8hktwOs2t2RhlllFFGGWWUUUYZ
vTT6A8e86/wAKAAA

------=_NextPart_000_005D_01C2F6D2.22C66E70--


------- End of forwarded message -------

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About