develooper Front page | perl.perl5.porters | Postings from July 2013

[perl #117855] Excessive memory waste from duplicate CopFILE in threaded perl

Thread Previous | Thread Next
Father Chrysostomos via RT
July 6, 2013 05:51
[perl #117855] Excessive memory waste from duplicate CopFILE in threaded perl
Message ID:
On Sat May 04 04:56:48 2013, timbo wrote:
> A perl with ithreads enabled saves duplicate copies of the perl source
> file
> name for every CvCOPFILE. That's a lot of wasteful duplication.
> This isn't new. It goes back to at least 5.8.
> To see how much this costs I modified a copy of Devel::SizeMe to do
>     warn("%p cop_file %s\n", basecop->cop_file, basecop->cop_file);
> and ran
>     perl -Mblib -MDancer -MDevel::SizeMe=:all -e 'perl_size()' 2>x
> then
>     grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | wc
> -c
> which reported 568813 bytes consumed by cop_file strings.
> Then I ran the same but with an extra sort -u at the end:
>     grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | sort
> -u | wc
> which reports there there were only 9074 bytes of distinct cop_file
> strings
> (in 108 distinct cop_file strings).
> So that's over 550KB wasted in duplicate by cop_file strings in a
> relatively
> small 'application'.
> That's a lot of wasted memory. The scale of the waste is proportional
> to the
> length of the directory paths the modules are loaded from. E.g., mine
> were:
> /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-
> thr/lib/site_perl/5.8.9/HTTP/Body/

Please test the attached patch and see whether it reduces memory usage
for you.  Testing it with Devel::Size will *not* work, as it will need
to be updated for this.  (It is needfully very intrusive into perl’s guts.)


Father Chrysostomos

via perlbug:  queue: perl5 status: open

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About