[perl #30509] use encoding and "eq" cause memory leak

# New Ticket Created by  pajas@ufal.ms.mff.cuni.cz 
# Please include the string:  [perl #30509]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org:80/rt3/Ticket/Display.html?id=30509 >


This is a bug report for perl from pajas@ufal.mff.cuni.cz,
generated with the help of perlbug 1.34 running under perl v5.8.2.


-----------------------------------------------------------------
[Please enter your report here]

If a subroutine defined before "use encoding" compares strings one of
which is flagged UTF8 (typically, this would be a string read from a
filehandle with an IO encoding layer), the comparison leaks terribly.
This doesn't happen if "use encoding" is used before the defining
subroutine or if it is replaced with "use utf8". On the other hand,
encoding mapage says, that

The best way to avoid ... oddity is to use this pragma RIGHT AFTER
other modules are loaded.

But this approach will leak, if the "other" modules compare UTF8
encoded strings (which is how I found this bug).

On my box the test case below uses >13Mb within 2 seconds and
grows until the system kills it.

__________
#!/usr/bin/perl -w
use strict;
#use encoding "iso-8859-2"; #ok if here

use Encode;
sub leak {
    my $foo = "foo";
    Encode::_utf8_on($foo);  # or read utf8 string from a file
    while (1) {
        $foo eq "x" ? 1 : 0; # eq leaks
    }
}

#use utf8; # this would be ok

use encoding "iso-8859-2"; # leak if here

leak();
__END__

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
Site configuration information for perl v5.8.2:

Configured by pajas at Mon Dec  1 16:06:41 CET 2003.

Summary of my perl5 (revision 5.0 version 8 subversion 2) configuration:
  Platform:
    osname=linux, osvers=2.4.20-20.9, archname=i386-linux-thread-multi
    uname='linux sup.ms.mff.cuni.cz 2.4.20-20.9 #1 mon aug 18 11:45:58 edt 2003 i686 i686 i386 gnulinux '
    config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dversion=5.8.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.2 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dinc_version_list=5.8.0/i386-linux-thread-multi 5.8.0 -Dpager=/usr/bin/less -isr'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-O2 -g -pipe -march=i386 -mcpu=i686',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/opt/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.2/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -L/opt/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.2:
    /net/su/h/local2-rh8/lib/perl5/site_perl/5.8.2
    /net/su/h/local2-rh8/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/5.8.2
    /usr/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.2
    /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.2
    /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.0
    /usr/lib/perl5/vendor_perl
    /usr/lib/perl5/5.8.2/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/5.8.2/5.8.2
    /usr/lib/perl5/5.8.2/i386-linux-thread-multi
    /usr/lib/perl5/5.8.2
    .

---
Environment for perl v5.8.2:
    HOME=/home/pajas
    LANG=cs_CZ.UTF-8
    LANGUAGE (unset)
    LC_CTYPE=cs_CZ.UTF-8
    LD_LIBRARY_PATH=/lib:/usr/lib:/home/pajas/local2/lib:/home/pajas/lib:
    LOGDIR (unset)
    PATH=/home/pajas/bin:/net/su/h/local2-rh8/bin:/usr/ssh2/bin:/usr/bin:/bin:/usr/kerberos/bin:/usr/X11R6/bin:/home/pajas/bin:/usr/local/bin:/usr/local/exec:/home/pajas/treebank/perl:/home/pajas/treebank/rev:/home/pajas/jdk/bin
    PERLLIB=/net/su/h/local2-rh8/lib/perl5/site_perl/5.8.2:/net/su/h/local2-rh8/lib/perl5/site_perl/5.8.2/i386-linux-thread-multi
    PERL_BADLANG (unset)
    PERL_RL=Perl
    SHELL=/bin/bash

• [perl #30509] use encoding and "eq" cause memory leak by pajas @ ufal . ms . mff . cuni . cz
• Re: [perl #30509] use encoding and "eq" cause memory leak by Dave Mitchell