develooper Front page | perl.perl5.porters | Postings from June 2004

[perl #30509] use encoding and "eq" cause memory leak

Thread Next
pajas @ ufal . ms . mff . cuni . cz
June 28, 2004 08:56
[perl #30509] use encoding and "eq" cause memory leak
Message ID:
# New Ticket Created by 
# Please include the string:  [perl #30509]
# in the subject line of all future correspondence about this issue. 
# <URL: >

This is a bug report for perl from,
generated with the help of perlbug 1.34 running under perl v5.8.2.

[Please enter your report here]

If a subroutine defined before "use encoding" compares strings one of
which is flagged UTF8 (typically, this would be a string read from a
filehandle with an IO encoding layer), the comparison leaks terribly.
This doesn't happen if "use encoding" is used before the defining
subroutine or if it is replaced with "use utf8". On the other hand,
encoding mapage says, that

The best way to avoid ... oddity is to use this pragma RIGHT AFTER
other modules are loaded.

But this approach will leak, if the "other" modules compare UTF8
encoded strings (which is how I found this bug).

On my box the test case below uses >13Mb within 2 seconds and
grows until the system kills it.

#!/usr/bin/perl -w
use strict;
#use encoding "iso-8859-2"; #ok if here

use Encode;
sub leak {
    my $foo = "foo";
    Encode::_utf8_on($foo);  # or read utf8 string from a file
    while (1) {
        $foo eq "x" ? 1 : 0; # eq leaks

#use utf8; # this would be ok

use encoding "iso-8859-2"; # leak if here


[Please do not change anything below this line]
Site configuration information for perl v5.8.2:

Configured by pajas at Mon Dec  1 16:06:41 CET 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 2) configuration:
    osname=linux, osvers=2.4.20-20.9, archname=i386-linux-thread-multi
    uname='linux 2.4.20-20.9 #1 mon aug 18 11:45:58 edt 2003 i686 i686 i386 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dversion=5.8.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.2 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dinc_version_list=5.8.0/i386-linux-thread-multi 5.8.0 -Dpager=/usr/bin/less -isr'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -march=i386 -mcpu=i686',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/, so=so, useshrplib=true,
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.2/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -L/opt/local/lib'

Locally applied patches:

@INC for perl v5.8.2:

Environment for perl v5.8.2:
    LANGUAGE (unset)
    LOGDIR (unset)
    PERL_BADLANG (unset)

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About