develooper Front page | perl.perl5.porters | Postings from February 2014

Re: [perl #121274] sv arenas are never freeded

Thread Previous
Dave Mitchell
February 28, 2014 11:30
Re: [perl #121274] sv arenas are never freeded
Message ID:
On Thu, Feb 27, 2014 at 09:36:46AM -0800, bulk88 via RT wrote:
> On Thu Feb 27 09:09:47 2014, davem wrote:
> > On Wed, Feb 26, 2014 at 11:35:09PM -0800, bulk88 via RT wrote:
> > > To address what I think will be the response. Processing each data
> > > file/task in a fork is not a solution. It is a bandaid to a design bug
> > > in Perl. Perl should be able to goto 1 or 2 GB of memory to process a
> > > huge data set, then scale back down to, below, 150% of the memory usage
> > > before the task/data set job began. Not stay at 1 GB or 400% even though
> > > all variables/data belonging to the task were freed on a PP level.
> > 
> > But that is likely to be almost entirely down to the system's malloc()
> > implementation rather than perl's SV arena usage.
> No, AFAIK perl's SV arena usage never calls free on any arena blocks
> until interp exit. So this memory usage  has nothing to do with malloc
> implementation since free() was never called on it. Calling free() is
> perl's domain, not libc's.

I'm referring to the fact that in the real world, most code that uses
large amounts of memory have something having off each SV, like a PVX. So
in the real world, the issue of whether most of this memory is released to
the OS is dependent on whether the OS consolidates and unmmaps free()d
strings rather than whether perl free()s SV arenas.

> > Yes, perl will keep SV head and bodies around for future use
> Forever? That is a leak.

No, that's efficiency. Its a chosen trade-off. Like PADTMPs;

> This bug isn't about PVX buffers. None were allocated in the sample XS
> code in this ticket.

And I feel that the sample XS code isn't representative of typical large
memory footprint perl applications.

Look, if you can come up with a scheme that can detect completely
empty arenas and free them, that *doesn't* slow down the allocation or
freeing of each SV, and doesn't have un-perlish random pauses for GC
sweeps, then I would consider including in core. But I just think for the
reasons I and Zefram mentioned, the reality is that in practice users
won't see more memory reclaimed very often.

My Dad used to say 'always fight fire with fire', which is probably why
he got thrown out of the fire brigade.

Thread Previous Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About