develooper Front page | perl.perl5.porters | Postings from October 2003

[perl #24255] shared hash ref memory leak in 5.8.1

From:
Jack Steadman
Date:
October 21, 2003 03:45
Subject:
[perl #24255] shared hash ref memory leak in 5.8.1
Message ID:
rt-24255-66283.10.88055546842@rt.perl.org
# New Ticket Created by  Jack Steadman 
# Please include the string:  [perl #24255]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt2/Ticket/Display.html?id=24255 >



This is a bug report for perl from perlbug@wallofcuriosities.com,
generated with the help of perlbug 1.34 running under perl v5.8.1.


-----------------------------------------------------------------
[Please enter your report here]

There are a couple of bug reports out there for shared array memory problems,
but I've got a major memory leak that I think is related to a leak with shared
hashes.  It seems that after a reference is assigned to a shared hash it simply
won't go out of scope.  Try this:

#!/usr/bin/perl

use strict;
use threads;
use threads::shared;
use Thread::Queue;

my $top = &share({});
$top->{mid} = &share({});

my $i = 0;
my $state = 1;
while (1) {
	
	## this can be any array/hash/object reference
	my $queue = new Thread::Queue;
	
	$top->{mid}->{$i} = $queue;
	
	$i++;
	if ($i > 10) {
		$i = 0;
	}
}

Running this will cause the program's memory usage to balloon to 100MB+ within
seconds.  Incidentally, the following code causes the program to hang at the point
of the first delete while monopolizing the CPU:

#!/usr/bin/perl

use strict;
use threads;
use threads::shared;
use Thread::Queue;

my $top = &share({});
$top->{mid} = &share({});

my $i = 0;
my $state = 1;
while (1) {
	
	my $queue = new Thread::Queue;
	
	if ($state) {
		print "adding key $i\n";
		$top->{mid}->{$i} = $queue;
	} else {
		print "deleting key $i\n";
		delete $top->{mid}->{$i};
	}
	
	$i++;
	if ($i > 10) {
		$i = 0;
		$state = ($state) ? 0 : 1;
	}
}

Please let me know if you need any more specific information about how I'm using
data structures similar to these.  Right now I'm faced with doing frequent restarts
to manage memory problems.



[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
Site configuration information for perl v5.8.1:

Configured by twells at Fri Oct 17 18:19:07 EDT 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.4.20-20.9smp, archname=i686-linux-thread-multi
    uname='linux mattapan 2.4.20-20.9smp #1 smp mon aug 18 11:32:15 edt 2003 i686 i686 i386 gnulinux '
    config_args='-d'
    hint=previous, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O3',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.1:
    /usr/local/lib/perl5/5.8.1/i686-linux-thread-multi
    /usr/local/lib/perl5/5.8.1
    /usr/local/lib/perl5/site_perl/5.8.1/i686-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.8.1
    /usr/local/lib/perl5/site_perl
    /usr/lib/perl5/site_perl/5.8.0/i383-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl/5.8.1/i383-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.1
    .

---
Environment for perl v5.8.1:
    HOME=/home/jack
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
    PERL_BADLANG (unset)
    SHELL=/bin/tcsh




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About