develooper Front page | perl.perl5.porters | Postings from February 2004

Re: [perl #26785] mem leak in split

Thread Previous
Dave Mitchell
February 18, 2004 16:08
Re: [perl #26785] mem leak in split
Message ID:
On Tue, Feb 17, 2004 at 02:56:38PM -0000, Ohlman, Jeff wrote:
> # New Ticket Created by  "Ohlman, Jeff" 
> # Please include the string:  [perl #26785]
> # in the subject line of all future correspondence about this issue. 
> # <URL: >
> This is a bug report for perl from,
> generated with the help of perlbug 1.33 running under perl v5.6.1.
> I am trying to split a record twice - the 2nd split takes an element of the first split and splits it again.  Trying to assign the results of the 2nd split to another array causes the OS to produce:
> Out of memory during request for 1008 bytes, total sbrk() is 64858324 bytes!
> I've reduced the program to the bare min in order to reproduce the error.  This is the offending code:
> perl -ne 'chomp;@x=split(/< /,$_);@r=split(/~/,$x[1]);'">!@ <mailto:!@> >/,$_);@r=split(/~/,$x[1]);' reallybigfile.dat
> The work around is:
> perl -ne 'chomp;@x=split(/< /,$_);($a">!@ <mailto:!@> >/,$_);($a, $b, $c, $d, $e) =split(/~/,$x[1]);' reallybigfile.dat

I don't know whether there has been a cut+paste problem or something, but
neither of the two perl lines above look like like valid command-line
invocations. The first line has unbalanced single quotes, and the
second has unbalanced double quotes.

Without knowing what the real Perl code is that causing the problem its
difficult to comment further, but you may want to
a) try with a newer version of Perl than 5.6.1, eg 5.8.3, and see if the
problem goes away.
b) see whether any of the data lines in your file are excessively long.


> Here are 2 typical records:
> 00040000000~BUBBAS BEER~~~A~~~~~~N~< BILLING">!@ <mailto:!@> >BILLING~~~~~<!@>
> 00000443000~FOOBAR BROTHERS~~~A~~~~~~N~< BILLING~~~~~ <mailto:!@> TYPE1">!@>BILLING~~~~~<!@>TYPE1
> The data file is 900 mb, but the out of mem occurs after about 300 mb.
> We have ActiveState  perl, v5.6.1 built for 3446-svr4
> Our Unix box is NCR System V R4 V3 3446 Pentium Pro(TM)-EISA/PCI
> ---
> Flags:
>     category=core
>     severity=medium
> *****
> "The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential, proprietary, and/or privileged material.  Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.  If you received this in error, please contact the sender and delete the material from all computers."  113

Wesley Crusher gets beaten up by his classmates for being a smarmy git,
and consequently has a go at making some friends of his own age for a
    -- Things That Never Happen in "Star Trek" #18

Thread Previous Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About