develooper Front page | perl.perl5.porters | Postings from May 2004

[perl #29841] utf8::decode doesn't work under -T

Thread Next
From:
Stas Bekman
Date:
May 25, 2004 02:20
Subject:
[perl #29841] utf8::decode doesn't work under -T
Message ID:
rt-3.0.9-29841-88083.1.56655856690257@perl.org
# New Ticket Created by  Stas Bekman 
# Please include the string:  [perl #29841]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org:80/rt3/Ticket/Display.html?id=29841 >



This is a bug report for perl from stas@rabbit.stason.org,
generated with the help of perlbug 1.35 running under perl v5.8.4.


-----------------------------------------------------------------
[Please enter your report here]

The bug seems to be in utf8::decode (and probably related functions) when 
running under -T. Please observe:

% perl -MDevel::Peek -le '$_ = shift; Dump $_; utf8::decode $_; Dump $_' \
  `perl -le 'binmode STDOUT, ":utf8"; print "\x{0432}\x{0430}"'`

SV = PV(0x804df00) at 0x804dbd4
   REFCNT = 1
   FLAGS = (POK,pPOK)
   PV = 0x8064570 "\320\262\320\260"\0
   CUR = 4
   LEN = 5
SV = PV(0x804df00) at 0x804dbd4
   REFCNT = 1
   FLAGS = (POK,pPOK,UTF8)
   PV = 0x8064570 "\320\262\320\260"\0 [UTF8 "\x{432}\x{430}"]
   CUR = 4
   LEN = 5

And now with -T:

% perl -T -MDevel::Peek -le '$_ = shift; Dump $_; utf8::decode $_; Dump $_' \
  `perl -le 'binmode STDOUT, ":utf8"; print "\x{0432}\x{0430}"'`

SV = PVMG(0x8087c10) at 0x804dbd4
   REFCNT = 1
   FLAGS = (GMG,SMG,pPOK)
   IV = 0
   NV = 0
   PV = 0x8064560 "\320\262\320\260"\0
   CUR = 4
   LEN = 5
   MAGIC = 0x806a618
     MG_VIRTUAL = &PL_vtbl_taint
     MG_TYPE = PERL_MAGIC_taint(t)
     MG_LEN = 1
SV = PVMG(0x8087c10) at 0x804dbd4
   REFCNT = 1
   FLAGS = (GMG,SMG,pPOK)
   IV = 0
   NV = 0
   PV = 0x8064560 "\320\262\320\260"\0
   CUR = 4
   LEN = 5
   MAGIC = 0x806a618
     MG_VIRTUAL = &PL_vtbl_taint
     MG_TYPE = PERL_MAGIC_taint(t)
     MG_LEN = 1

As you can see the first invocation w/o -T correctly decodes input,
the second does not.


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
     category=utilities
     severity=critical
---
Site configuration information for perl v5.8.4:

Configured by stas at Sat May  8 23:36:56 PDT 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
   Platform:
     osname=linux, osvers=2.6.3-9mdk, archname=i686-linux-thread-multi
     uname='linux rabbit.stason.org 2.6.3-9mdk #1 fri apr 23 16:41:09 edt 2004 
i686 unknown unknown gnulinux '
     config_args='-des -Dprefix=/home/stas/perl/5.8.4-ithread -Dusethreads 
-Doptimize=-g -Duseshrplib -Dusedevel -Accflags=-DDEBUG_LEAKING_SCALARS'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=define use5005threads=undef useithreads=define 
usemultiplicity=define
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
     optimize='-g',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS 
-DDEBUG_LEAKING_SCALARS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-I/usr/include/gdbm'
     ccversion='', gccversion='3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)', 
gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib'
     libpth=/usr/local/lib /lib /usr/lib
     libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
     perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
     libc=/lib/libc-2.3.3.so, so=so, useshrplib=true, libperl=libperl.so
     gnulibc_version='2.3.3'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E 
-Wl,-rpath,/home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi/CORE'
     cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:


---
@INC for perl v5.8.4:
     /home/stas/perl/5.8.4-ithread/lib/5.8.4/i686-linux-thread-multi
     /home/stas/perl/5.8.4-ithread/lib/5.8.4
     /home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4/i686-linux-thread-multi
     /home/stas/perl/5.8.4-ithread/lib/site_perl/5.8.4
     /home/stas/perl/5.8.4-ithread/lib/site_perl
     .

---
Environment for perl v5.8.4:
     HOME=/home/stas
     LANG=en_GB
     LANGUAGE=en_GB:en
     LC_ADDRESS=en_CA
     LC_COLLATE=en_GB
     LC_CTYPE=en_GB
     LC_IDENTIFICATION=en_CA
     LC_MEASUREMENT=en_CA
     LC_MESSAGES=en_GB
     LC_MONETARY=en_CA
     LC_NAME=en_CA
     LC_NUMERIC=en_CA
     LC_PAPER=en_CA
     LC_TELEPHONE=en_CA
     LC_TIME=en_GB
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
 
PATH=/usr//bin:/bin:/usr/bin:.:/usr/local/bin:/usr/X11R6/bin/:/usr/games:/home/stas/bin:/home/stas/bin:/usr/local/bin:/usr/X11R6/bin:/usr/java/j2re1.4.0/bin/
     PERLDOC_PAGER=less -R
     PERL_BADLANG (unset)
     SHELL=/bin/tcsh


-- 
__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About