[perl #38384] Memory leak printing large hash-references

# New Ticket Created by  paul.boutros@utoronto.ca 
# Please include the string:  [perl #38384]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/rt3/Ticket/Display.html?id=38384 >


This is a bug report for perl from dlink@ccb690.ccb.utoronto.ca,
generated with the help of perlbug 1.35 running under perl v5.8.7.


-----------------------------------------------------------------
I appear to have found a memory leak in perl 5.8.7 and possibly 5.6.2
that occurs when printing out compound-hashes using tabs.  I realize
this sounds bizarre, but this has been confirmed by other uses on
PerlMonks:
http://perlmonks.org/?node_id=526595

Here is a minimal test-case (call it as test.pl > test.txt).  It
creates three 12.5-million element HoH's, then tries to print them
to file.  The program successfully creates the hashes, but crashes
about half-way through printing with an "out of memory!" error.
It is unclear why anything in the nested while loops should be using
substantial memory.

In addition, deleting keys manually after they've been printed
prevents crashing.

use strict;

my %hash1;
my %hash2;
my %hash3;

my @data = (0 .. 5000);

for (my $i = 0; $i < scalar(@data); $i++ ) {
        for (my $j = $i; $j < scalar(@data); $j++) {
                $hash1{ $data[$i] }{ $data[$j] } = 0;
                $hash2{ $data[$i] }{ $data[$j] } = 0;
                $hash3{ $data[$i] }{ $data[$j] } = 0;
                }
        }

print "Finished building hashes\n";

while ( my ($key1, $val1) = each %hash1 ) {
        while ( my ($key2, $val2) = each %$val1 ) {
                print join(
                        "\t",
                        $key1,
                        $key2,
                        $hash1{$key1}{$key2},
                        $hash2{$key1}{$key2},
                        $hash3{$key1}{$key2}
                        ), "\n";
                }
        }




[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.8.7:

Configured by dlink at Mon Jul  4 16:29:05 EDT 2005.

Summary of my perl5 (revision 5 version 8 subversion 7) configuration:
  Platform:
    osname=aix, osvers=5.2.0.0, archname=aix
    uname='aix ccb690 2 5 00251b3f4c00 '
    config_args='-Dcc=gcc -Dprefix=/db2blast/Paul/perl5.8.7'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef 
usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -
DUSE_NATIVE_DLOPEN -fno-strict-aliasing -pipe -I/usr/local/include -
D_LARGE_FILES',
    optimize='-O',
    cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN 
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='3.3.2', gccosandvers='aix5.2.0.0'
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -Wl,-brtl -Wl,-bdynamic -Wl,-bmaxdata:0x80000000 -L/
usr/local/lib -Wl,-b32'
    libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
    libs=-lbind -lnsl -ldbm -ldl -lld -lm -lcrypt -lc -lbsd
    perllibs=-lbind -lnsl -ldl -lld -lm -lcrypt -lc -lbsd
    libc=/lib/libc.a, so=a, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Xlinker -bE:/
db2blast/Paul/perl5.8.7/lib/5.8.7/aix/CORE/perl.exp'
    cccdlflags=' ', lddlflags='  -Wl,-bhalt:4 -Wl,-bexpall -Wl,-G -Wl,-bnoentry 
-lc -L/usr/local/lib'

Locally applied patches:


---
@INC for perl v5.8.7:
    /db2blast/Paul/perl5.8.7/lib/5.8.7/aix
    /db2blast/Paul/perl5.8.7/lib/5.8.7
    /db2blast/Paul/perl5.8.7/lib/site_perl/5.8.7/aix
    /db2blast/Paul/perl5.8.7/lib/site_perl/5.8.7
    /db2blast/Paul/perl5.8.7/lib/site_perl
    .

---
Environment for perl v5.8.7:
    HOME=/db2/dlink
    LANG=en_US
    LANGUAGE (unset)
    LC__FASTMSG=true
    LD_LIBRARY_PATH=:/db2/dlink/sqllib/lib
    LIBPATH=/usr/lib:/lib:/usr/local/lib:/db2/dlink/sqllib/lib
    LOGDIR (unset)
    PATH=/db2blast/gcc/bin:/db2blast/Paul/perl5.8.7/bin:/db2blast/Paul/bin:/usr/
local/lib:/usr/local/include:/usr/local/bin:/u
sr/bin:/etc:/usr/sbin:/usr/ucb:/db2/dlink/bin:/usr/vac/bin:/usr/bin/X11:/sbin:.:
/local/bin:/usr/java14/jre/bin:/usr/java131/bi
n:/db2blast/R/bin:/db2/blast/bin:/db2/dlink/sqllib/bin:/db2/dlink/sqllib/adm:/
db2/dlink/sqllib/misc
    PERL_BADLANG (unset)
    SHELL=/usr/bin/ksh

• Re: [perl #38384] Memory leak printing large hash-references by Unknown Sender
• [perl #38384] Memory leak printing large hash-references by paul . boutros @ utoronto . ca
• Re: [perl #38384] Memory leak printing large hash-references by Gisle Aas
• Re: [perl #38384] Memory leak printing large hash-references by Yitzchak Scott-Thoennes