develooper Front page | perl.perl5.porters | Postings from May 2012

[perl #26785] mem leak in split

From:
James E Keenan via RT
Date:
May 26, 2012 19:08
Subject:
[perl #26785] mem leak in split
Message ID:
rt-3.6.HEAD-7788-1338084516-469.26785-15-0@perl.org
On Fri Mar 26 09:29:33 2004, davem@fdisolutions.com wrote:
> On Thu, Feb 19, 2004 at 06:25:48AM -0600, Ohlman, Jeff wrote:
> >
> > I am trying to split a record twice - the 2nd split takes an element
> of the first split and splits it again.  Trying to assign the results
> of the 2nd split to another array causes the OS to produce:
> >
> > Out of memory during request for 1008 bytes, total sbrk() is
> 64858324 bytes!
> >
> > I've reduced the program to the bare min in order to reproduce the
> error.  This is the offending code:
> >
> > perl -ne 'chomp;@x=split(/<!@>/,$_);@r=split(/~/,$x[1]);'
> reallybigfile.dat
> >
> > The work around is:
> >
> > perl -ne 'chomp;@x=split(/<!@>/,$_);($a, $b, $c, $d, $e)
> =split(/~/,$x[1]);' reallybigfile.dat
> >
> > Here are 2 typical records:
> >
> > 00040000000~BUBBAS BEER~~~A~~~~~~N~<!@>BILLING~~~~~<!@>
> > 00000443000~FOOBAR BROTHERS~~~A~~~~~~N~<!@>BILLING~~~~~<!@>TYPE1
> >
> >
> > The data file is 900 mb, but the out of mem occurs after about 300
> mb.
> 
> could you check that the file hasn't got any very long lines:
> 
> perl -ne 'die "length=".length if length >
> 10000;chomp;@x=split(/<!@>/,$_);@r=split(/~/,$x[1]);'
> reallybigfile.dat
> 
> 

Discussion in this ticket petered out eight years ago.  We didn't hear
back from the OP as to whether the file contained very long lines or not.

I was able to run the OP's code without incident:

$ perl -ne 'chomp;@x=split(/<!@>/,$_);($a, $b, $c, $d, $e)
=split(/~/,$x[1]);print join("|"=>($a,$b,$c,$d,$e)), "\n";'
reallybigfile.dat
BILLING||||
BILLING||||

How should we proceed with this RT?

Thank you very much.
Jim Keenan


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=26785



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About