[perl #38320] apparent memory leak in 5.8.1 while using subsitutions

# New Ticket Created by  jsalinas@cray.com 
# Please include the string:  [perl #38320]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/rt3/Ticket/Display.html?id=38320 >



This is a bug report for perl from jsalinas@cray.com,
generated with the help of perlbug 1.34 running under perl v5.8.1.

-----------------------------------------------------------------
[Please enter your report here]

This perl script demonstrates a memory leak problem 
when perl is using the subsitute operator.  If this
program is run long enough it can crash or hang a 
system because it will exhaust swap space when it should
not: 

#!/usr/bin/perl

# read hw spec to nid spec map
open S, "hw2nid";
while (<S>) {
        chomp;
        next if /^#/;
        ($hw,$nid) = split;
        $hw2nid{$hw} = $nid;
}
close S;

# sample input line
#[2005-07-07 16:10:40][c9-0c0s4n2]******* _cstart2(), yod_pid=2949 rank=146 lognid=146 physnid=0x912 pid=5

while (<>) {
        # convert hw nid spec to integer nid spec
        $_ =~ /\[(c\d+-\d+c\d+s\d+n\d+)\]/i;
        $hw_raw = $1;

        # mem leak
        $nidstr = sprintf "%+4s][$hw_raw", $nid;
        s/$hw_raw/$nidstr/;

        # no mem leak
        #@F = split /\]/;
        #$_ = sprintf "%s][%+4s]%s", $F[0], $nid, join "]",@F[1..$#F];
}

# sleep so can observe process characteristics
sleep 600;

exit 0

This program was run from a endless loop script:
#!/bin/sh
./do_translate.pl console.0601180954 &

while :; do
        ps -eo pid,ppid,pmem,cmd | egrep MEM\|do_translate 
        sleep 2
done

exit 0

Note this was not an issue under 5.8.0:
perl -V |more
Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.18-27.8.0, archname=i686-linux
    uname='linux steinmetz.psc.edu 2.4.18-27.8.0 #1 fri mar 14 07:36:43 est 2003
 i686 athlon i386 gnulinux '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=unde
f
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_B
ITS=64 -I/usr/include/gdbm',
    optimize='-O3',
    cppflags='-fno-strict-aliasing -I/usr/include/gdbm'
    ccversion='', gccversion='3.2 20020903 (Red Hat Linux 8.0 3.2-7)', gccosandv
ers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize
=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-L/usr/local/lib -L/usr/psc/lib -L/usr/lib -L/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/psc/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lc -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/usr/psc/lib -L/us
r/lib -L/lib'

Characteristics of this binary (from libperl): 
  Compile-time options: USE_LARGE_FILES
  Built under linux
  Compiled at Apr 18 2003 16:12:25
  @INC:
    /usr/psc/lib/perl5/5.8.0/i686-linux
 i686 athlon i386 gnulinux '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=unde
f
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_B
ITS=64 -I/usr/include/gdbm',
    optimize='-O3',
    cppflags='-fno-strict-aliasing -I/usr/include/gdbm'
    ccversion='', gccversion='3.2 20020903 (Red Hat Linux 8.0 3.2-7)', gccosandv
ers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize
=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-L/usr/local/lib -L/usr/psc/lib -L/usr/lib -L/lib'
    libpth=/usr/local/lib /lib /usr/lib /usr/psc/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lc -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib -L/usr/psc/lib -L/us
r/lib -L/lib'

Characteristics of this binary (from libperl): 
  Compile-time options: USE_LARGE_FILES
  Built under linux
  Compiled at Apr 18 2003 16:12:25
  @INC:
    /usr/psc/lib/perl5/5.8.0/i686-linux
    /usr/psc/lib/perl5/5.8.0
    /usr/psc/lib/perl5/site_perl/5.8.0/i686-linux
    /usr/psc/lib/perl5/site_perl/5.8.0
    /usr/psc/lib/perl5/site_perl
    .

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
This perlbug was built using Perl v5.8.1 - Mon Sep  1 20:48:29 UTC 2003
It is being executed now by  Perl v5.8.1 - Mon Sep  1 20:44:26 UTC 2003.

Site configuration information for perl v5.8.1:

Configured by root at Mon Sep  1 20:44:26 UTC 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.6.0-test2, archname=x86_64-linux-thread-multi
    uname='linux e69 2.6.0-test2 #1 sun aug 31 23:50:34 utc 2003 x86_64 x86_64 x86_64 gnulinux '
    config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=true -Doptimize=-O2 -Wall -pipe'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -Wall -pipe',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -fno-strict-aliasing'
    ccversion='', gccversion='3.3.1 (SuSE Linux)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib64'
    libpth=/lib64 /usr/lib64 /usr/local/lib64
    libs=-lm -ldl -lcrypt -lpthread
    perllibs=-lm -ldl -lcrypt -lpthread
    libc=/lib64//lib64/libc.so.6, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.1/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64'

Locally applied patches:
    RC4

---
@INC for perl v5.8.1:
    /usr/lib/perl5/5.8.1/x86_64-linux-thread-multi
    /usr/lib/perl5/5.8.1
    /usr/lib/perl5/site_perl/5.8.1/x86_64-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.1
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.1/x86_64-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.1
    /usr/lib/perl5/vendor_perl
    .

---
Environment for perl v5.8.1:
    HOME=/usr/users/1/jsalinas
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/usr/local/packages/globus/globus-4.0.1/lib:/opt/cray/lib:/opt/xt-os/1.3.10/lib:/opt/xt-libc/1.3.10/amd64/lib:/opt/xt-tools/papi/3.1.0/lib:/opt/xt-tools/papi/3.1.0/lib64:/opt/xt-tools/papi/3.1.0/lib/cnos64:/opt/xt-pe/1.3.10/lib:/opt/xt-mpt/1.3.10/mpich2-64/P2/lib:/opt/xt-libsci/1.3.10/pgi/lib:/opt/pgi/6.0.1/linux86-64/6.0/libso:/opt/pgi/6.0.1/linux86-64/6.0/lib:/opt/acml/2.6/pgi64/lib
    LOGDIR (unset)
    PATH=/usr/psc/bin:/usr/psc/gnu/bin:/usr/psc/krb5/bin:/usr/local/bin:/usr/local/packages/tg/bin:/usr/local/packages/globus/globus-4.0.1/bin:/usr/local/packages/globus/globus-4.0.1/sbin:/opt/kde3/bin:/opt/gnome/bin:/usr/games:/usr/bin/X11:/opt/xt-lustre-ss/1.3.10/usr/sbin:/opt/xt-lustre-ss/1.3.10/usr/bin:/opt/cray/bin:/opt/cray/etc:/opt/xt-boot/1.3.10/bin/snos64:/opt/xt-catamount/1.3.10/bin/snos64:/opt/xt-os/1.3.10/bin/snos64:/opt/xt-service/1.3.10/bin/snos64:/opt/xt-prgenv/1.3.10/bin:/opt/xt-tools/papi/3.1.0/bin:/opt/xt-tools/papi/3.1.0/share/papi/utils:/opt/xt-pe/1.3.10/cnos/linux/64/bin:/opt/xt-pe/1.3.10/bin/snos64:/opt/xt-mpt/1.3.10/mpich2-64/P2/bin:/opt/pgi/6.0.1/linux86-64/6.0/bin:/opt/xt-pbs/5.3.3xt_psc/bin:/bin:/usr/bin
    PERL_BADLANG (unset)
    SHELL=/bin/tcsh

• [perl #38320] apparent memory leak in 5.8.1 while using subsitutions by jsalinas @ cray . com
• Re: [perl #38320] apparent memory leak in 5.8.1 while using subsitutions by Joshua ben Jore