develooper Front page | perl.perl5.porters | Postings from July 2013

[perl #117855] Excessive memory waste from duplicate CopFILE in threaded perl

Thread Previous | Thread Next
From:
Father Chrysostomos via RT
Date:
July 6, 2013 05:51
Subject:
[perl #117855] Excessive memory waste from duplicate CopFILE in threaded perl
Message ID:
rt-3.6.HEAD-2552-1373089870-719.117855-15-0@perl.org
On Sat May 04 04:56:48 2013, timbo wrote:
> A perl with ithreads enabled saves duplicate copies of the perl source
> file
> name for every CvCOPFILE. That's a lot of wasteful duplication.
> This isn't new. It goes back to at least 5.8.
> 
> To see how much this costs I modified a copy of Devel::SizeMe to do
> 
>     warn("%p cop_file %s\n", basecop->cop_file, basecop->cop_file);
> 
> and ran
> 
>     perl -Mblib -MDancer -MDevel::SizeMe=:all -e 'perl_size()' 2>x
> 
> then
> 
>     grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | wc
> -c
> 
> which reported 568813 bytes consumed by cop_file strings.
> 
> Then I ran the same but with an extra sort -u at the end:
> 
>     grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | sort
> -u | wc
> 
> which reports there there were only 9074 bytes of distinct cop_file
> strings
> (in 108 distinct cop_file strings).
> 
> So that's over 550KB wasted in duplicate by cop_file strings in a
> relatively
> small 'application'.
> 
> That's a lot of wasted memory. The scale of the waste is proportional
> to the
> length of the directory paths the modules are loaded from. E.g., mine
> were:
> /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-
> thr/lib/site_perl/5.8.9/HTTP/Body/XFormsMultipart.pm

Please test the attached patch and see whether it reduces memory usage
for you.  Testing it with Devel::Size will *not* work, as it will need
to be updated for this.  (It is needfully very intrusive into perl’s guts.)

-- 

Father Chrysostomos


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=117855

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About