develooper Front page | perl.perl5.porters | Postings from May 2013

[perl #117855] Excessive memory waste from duplicate CvCOPFILE in threaded perl

From:
Tim Bunce
Date:
May 4, 2013 11:57
Subject:
[perl #117855] Excessive memory waste from duplicate CvCOPFILE in threaded perl
Message ID:
rt-3.6.HEAD-28177-1367668608-1958.117855-75-0@perl.org
# New Ticket Created by  Tim Bunce 
# Please include the string:  [perl #117855]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=117855 >



This is a bug report for perl from timbo@timac.local,
generated with the help of perlbug 1.39 running under perl v5.8.9.


-----------------------------------------------------------------
[Please describe your issue here]

A perl with ithreads enabled saves duplicate copies of the perl source file
name for every CvCOPFILE. That's a lot of wasteful duplication.
This isn't new. It goes back to at least 5.8.

To see how much this costs I modified a copy of Devel::SizeMe to do

    warn("%p cop_file %s\n", basecop->cop_file, basecop->cop_file);

and ran

    perl -Mblib -MDancer -MDevel::SizeMe=:all -e 'perl_size()' 2>x

then

    grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | wc -c

which reported 568813 bytes consumed by cop_file strings.

Then I ran the same but with an extra sort -u at the end:

    grep 'cop_file' x | sort -u | perl -pe 's/^\S+ cop_file //' | sort -u | wc

which reports there there were only 9074 bytes of distinct cop_file strings
(in 108 distinct cop_file strings).

So that's over 550KB wasted in duplicate by cop_file strings in a relatively
small 'application'.

That's a lot of wasted memory. The scale of the waste is proportional to the
length of the directory paths the modules are loaded from. E.g., mine were:
/Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/lib/site_perl/5.8.9/HTTP/Body/XFormsMultipart.pm

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.8.9:

Configured by timbo at Thu Oct 11 16:23:44 IST 2012.

Summary of my perl5 (revision 5 version 8 subversion 9) configuration:
  Platform:
    osname=darwin, osvers=11.4.2, archname=darwin-thread-multi-2level
    uname='darwin timac.local 11.4.2 darwin kernel version 11.4.2: thu aug 23 16:25:48 pdt 2012; root:xnu-1699.32.7~1release_x86_64 x86_64 '
    config_args='-de -Dprefix=/Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr -Duseithreads -Aeval:scriptdir=/Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/bin'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -I/usr/local/include -I/opt/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple Clang 3.1 (tags/Apple/clang-318.0.61)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /usr/lib
    libs=-lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.9:
    /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/lib/5.8.9/darwin-thread-multi-2level
    /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/lib/5.8.9
    /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/lib/site_perl/5.8.9/darwin-thread-multi-2level
    /Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/lib/site_perl/5.8.9
    .

---
Environment for perl v5.8.9:
    HOME=/Users/timbo
    LANG=en_IE.UTF-8
    LANGUAGE (unset)
    LC_ALL=en_IE.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PERLBREW_BASHRC_VERSION=0.44
    PERLBREW_HOME=/Users/timbo/.perlbrew
    PERLBREW_MANPATH=/Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/man
    PERLBREW_PATH=/Users/timbo/perl5/perlbrew/bin:/Users/timbo/perl5/perlbrew/perls/perl-5.8.9-thr/bin
    PERLBREW_PERL=perl-5.8.9-thr
    PERLBREW_ROOT=/Users/timbo/perl5/perlbrew
    PERLBREW_VERSION=0.44
    PERLCRITIC=/Users/timbo/.setdev/perlcriticrc
    PERLTIDY=/Users/timbo/.setdev/perltidyrc
    SHELL=/bin/bash




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About