develooper Front page | perl.perl5.porters | Postings from September 2006

[perl #40416] threads creation memory leak

Thread Next
From:
Santeri Paavolainen
Date:
September 26, 2006 11:42
Subject:
[perl #40416] threads creation memory leak
Message ID:
rt-3.5.HEAD-31251-1159295346-1455.40416-75-0@perl.org
# New Ticket Created by  Santeri Paavolainen 
# Please include the string:  [perl #40416]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=40416 >


This is a bug report for perl from santtu@iki.fi,
generated with the help of perlbug 1.35 running under perl v5.8.8.


-----------------------------------------------------------------

The following simple program causes memory leak at about 134 kB/s -
with about 180 creates/second (on my machine) this equals about 850
bytes leaked per created, detached and destroyed thread.

    use strict;
    use threads;

    while (1) {
      threads->create(\&away)->detach;
    }

    sub away {
    }

This leak is simple to reproduce, but not so easy to detect. The
memory usage signal is very noisy (due to allocation patterns), but
with a long run a clear trend can be detected -- the leak is clearly
more than 0 bytes/loop, and less than 1 kB/loop. Statistics point to
loss of about 800 bytes/loop.

I have reproduced this on Fedora Core 5 stock perl (shown here in
perlbug report) as well as custom-built 5.8.8 interpreter with
DEBUGGING enabled. Both show approximately the same amount of memory
leaked per loop.

I also run valgrind on DEBUGGING perl (*not* the fc5 stock perl) with
the following script:

    use strict;
    use threads;
    use Time::HiRes qw(time);

    my $start = time;
    my $cnt = 0;
    my $max = 100;
    while ($cnt++ < $max) {
      threads->create(\&away)->detach;
    }
    my $end = time;
    my $diff = $end - $start;
    print "$max iters in $diff seconds, ", ($max / $diff), " iters/s\n";

    sub away {
    }

And the results are:

==13941== Memcheck, a memory error detector.
==13941== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==13941== Using LibVEX rev 1471, a library for dynamic binary translation.
==13941== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==13941== Using valgrind-3.1.0, a dynamic binary instrumentation framework.
==13941== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==13941== For more details, rerun with: -v
==13941==
==13941== My PID = 13941, parent PID = 2928.  Prog and args are:
==13941==    /l/perl/bin/perl
==13941==    scripts/memtest4.pl
==13941==
--13941-- WARNING: unhandled syscall: 311
--13941-- You may be able to write your own handler.
--13941-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
==13941==
==13941== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 22 from 1)
==13941== malloc/free: in use at exit: 37,739 bytes in 3,753 blocks.
==13941== malloc/free: 459,523 allocs, 455,770 frees, 61,144,783 bytes
allocated.
==13941== For counts of detected errors, rerun with: -v
==13941== searching for pointers to 3,753 not-freed blocks.
==13941== checked 31,761,632 bytes.
==13941==
==13941== 408 bytes in 3 blocks are possibly lost in loss record 7 of 9
==13941==    at 0x40045EB: calloc (vg_replace_malloc.c:279)
==13941==    by 0x4CDAD6A9: _dl_allocate_tls (in /lib/ld-2.4.so)
==13941==    by 0x4D054B13: pthread_create@@GLIBC_2.1 (in
/lib/libpthread-2.4.so)
==13941==    by 0x400BE03: Perl_ithread_create (threads.xs:532)
==13941==    by 0x400C647: XS_threads_new (threads.xs:698)
==13941==    by 0x80F3C6D: Perl_pp_entersub (pp_hot.c:2877)
==13941==    by 0x80CE72C: Perl_runops_debug (dump.c:1459)
==13941==    by 0x8065C81: S_run_body (perl.c:2366)
==13941==    by 0x80656B6: perl_run (perl.c:2283)
==13941==    by 0x80601EE: main (perlmain.c:99)
==13941==
==13941==
==13941== 35,456 bytes in 3,738 blocks are definitely lost in loss
record 9 of 9
==13941==    at 0x40051F9: malloc (vg_replace_malloc.c:149)
==13941==    by 0x80CED9B: Perl_safesysmalloc (util.c:78)
==13941==    by 0x80D0694: Perl_savepv (util.c:765)
==13941==    by 0x810DC2A: Perl_sv_dup (sv.c:10293)
==13941==    by 0x810B538: Perl_gp_dup (sv.c:9721)
==13941==    by 0x810CFE9: Perl_sv_dup (sv.c:10181)
==13941==    by 0x80E089D: Perl_he_dup (hv.c:156)
==13941==    by 0x810DB59: Perl_sv_dup (sv.c:10281)
==13941==    by 0x810CF87: Perl_sv_dup (sv.c:10179)
==13941==    by 0x8110D1F: perl_clone (sv.c:11074)
==13941==    by 0x400BB7A: Perl_ithread_create (threads.xs:438)
==13941==    by 0x400C647: XS_threads_new (threads.xs:698)
==13941==
==13941== LEAK SUMMARY:
==13941==    definitely lost: 35,456 bytes in 3,738 blocks.
==13941==      possibly lost: 408 bytes in 3 blocks.
==13941==    still reachable: 1,875 bytes in 12 blocks.
==13941==         suppressed: 0 bytes in 0 blocks.
==13941== Reachable blocks (those to which a pointer was found) are not
shown.
==13941== To see them, rerun with: --show-reachable=yes

-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
This perlbug was built using Perl v5.8.8 in the Red Hat build system.
It is being executed now by Perl v5.8.8 - Sun Jun  4 19:33:43 EDT 2006.

Site configuration information for perl v5.8.8:

Configured by Red Hat, Inc. at Sun Jun  4 19:33:43 EDT 2006.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.9-34.elsmp, archname=i386-linux-thread-multi
    uname='linux hs20-bc2-4.build.redhat.com 2.6.9-34.elsmp #1 smp fri
feb 24 16:56:28 est 2006 i686 i686 i386 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tables -Dversion=5.8.8 -Dmyhostname=localhost
-Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc.
-Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux
-Dvendorprefix=/usr -Dsiteprefix=/usr -Duseshrplib -Dusethreads
-Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly
-Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto
-Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto
-Ud_endservent_r_proto -Ud_setservent_r_proto -Dinc_version_list=5.8.7
5.8.6 5.8.5 5.8.4 5.8.3 -Dscriptdir=/usr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -Wdeclaration-after-statement -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386
-mtune=generic -fasynchronous-unwind-tables',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-Wdeclaration-after-statement -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='4.1.1 20060525 (Red Hat 4.1.1-1)',
gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.4.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:


---
@INC for perl v5.8.8:
    /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.4/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.3/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl/5.8.7
    /usr/lib/perl5/site_perl/5.8.6
    /usr/lib/perl5/site_perl/5.8.5
    /usr/lib/perl5/site_perl/5.8.4
    /usr/lib/perl5/site_perl/5.8.3
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.4/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.3/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl/5.8.7
    /usr/lib/perl5/vendor_perl/5.8.6
    /usr/lib/perl5/vendor_perl/5.8.5
    /usr/lib/perl5/vendor_perl/5.8.4
    /usr/lib/perl5/vendor_perl/5.8.3
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.8/i386-linux-thread-multi
    /usr/lib/perl5/5.8.8
    .

---
Environment for perl v5.8.8:
    HOME=/home/santtu
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
   
PATH=/usr/kerberos/bin:/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/pkg/bin:/usr/local/bin:/home/santtu/bin:/home/santtu/bin/Linux-i686:/usr/java/jdk1.5.0_06/bin:/home/santtu/bin
    PERL_BADLANG (unset)
    SHELL=/bin/zsh


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About