develooper Front page | perl.perl5.porters | Postings from December 2004

[perl #32964] utf-8 + void map problem in perl >= 5.8.3

From:
David Kamholz
Date:
December 8, 2004 14:25
Subject:
[perl #32964] utf-8 + void map problem in perl >= 5.8.3
Message ID:
rt-3.0.11-32964-102410.12.3768169349304@perl.org
# New Ticket Created by  David Kamholz 
# Please include the string:  [perl #32964]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org:80/rt3/Ticket/Display.html?id=32964 >


This is a bug report for perl from davekam@ext5.eva.mpg.de,
generated with the help of perlbug 1.35 running under perl v5.8.5.


-----------------------------------------------------------------
[Please enter your report here]

OK, after a long discussion on #perl on freenode we've narrowed this  
bug down.
It occurs only in 5.8.3 or later, on various architectures (tried on  
darwin, x86,
opteron, etc.). It happens only with use encoding 'utf8', NOT with use  
utf8. It
doesn't happen at all if neither are specified. The '<:utf8' in open is  
irrelevant.
Go to http://www.plover.com/~mjd/misc/perl/ningu/ to see the original  
script and
input file -- it returns 58018 lines instead of 58019. The crucial line  
is map.
If you change it to foreach, it works properly.

The following oneliners (contributed by Somni on #perl) also reproduce  
the problem:

perl -Mencoding=utf8 -e 'sub i { "" }  map { i join("", ""), "" }  
1..2042; "$i";'
perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'

These scripts typically produce segfault, memory wrap, or attempt to  
deallocate
unreferenced scalar errors. Different archs seem to have different  
magic numbers
required to trigger it, but typically a number somewhere in the 2040 to  
2043 range
will do it, and anything higher. But there are also some weird things  
where
sometimes higher numbers will work fine, or certain lower numbers will  
cause the
error.

Good luck tracking this one down!
Regards,
Dave (ningu)

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
     category=core
     severity=medium
---
Site configuration information for perl v5.8.5:

Configured by davekam at Thu Nov 18 18:00:57 CET 2004.

Summary of my perl5 (revision 5 version 8 subversion 5) configuration:
   Platform:
     osname=darwin, osvers=7.6.0, archname=darwin-2level
     uname='darwin ext5.eva.mpg.de 7.6.0 darwin kernel version 7.6.0:  
sun oct 10 12:05:27 pdt 2004; root:xnuxnu-517.9.4.obj~1release_ppc  
power macintosh powerpc '
     config_args='-des -Dprefix=/opt -Dccflags=-I'/opt/include'  
-Dldflags=-L/opt/lib -Dvendorprefix=/opt -Duseshrplib'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=undef use5005threads=undef useithreads=undef  
usemultiplicity=undef
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-I/opt/include -fno-common -DPERL_DARWIN  
-no-cpp-precomp -fno-strict-aliasing -pipe -I/usr/local/include',
     optimize='-Os',
     cppflags='-no-cpp-precomp -I/opt/include -fno-common -DPERL_DARWIN  
-no-cpp-precomp -fno-strict-aliasing -pipe -I/usr/local/include'
     ccversion='', gccversion='3.3 20030304 (Apple Computer, Inc. build  
1666)', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',  
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='-L/opt/lib  
-L/usr/local/lib'
     libpth=/usr/local/lib /usr/lib
     libs=-ldbm -ldl -lm -lc
     perllibs=-ldl -lm -lc
     libc=/usr/lib/libc.dylib, so=dylib, useshrplib=true,  
libperl=libperl.dylib
     gnulibc_version=''
   Dynamic Linking:
     dlsrc=dl_dyld.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
     cccdlflags=' ', lddlflags='-L/opt/lib -bundle -undefined  
dynamic_lookup -L/usr/local/lib'

Locally applied patches:


---
@INC for perl v5.8.5:
     /opt/lib/perl5/5.8.5/darwin-2level
     /opt/lib/perl5/5.8.5
     /opt/lib/perl5/site_perl/5.8.5/darwin-2level
     /opt/lib/perl5/site_perl/5.8.5
     /opt/lib/perl5/site_perl
     /opt/lib/perl5/vendor_perl/5.8.5/darwin-2level
     /opt/lib/perl5/vendor_perl/5.8.5
     /opt/lib/perl5/vendor_perl
     .

---
Environment for perl v5.8.5:
     DYLD_LIBRARY_PATH (unset)
     HOME=/Users/davekam
     LANG (unset)
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
      
PATH=/sw/bin:/opt/bin:/opt/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/bin: 
/sbin:/usr/X11R6/bin:/usr/local/teTeX/bin/powerpc-apple-darwin-current
     PERL_BADLANG (unset)
     SHELL=/bin/bash




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About