Front page | perl.perl5.porters |
Postings from December 2012
[perl #63674] open() is not UTF-8-clean
Thread Previous
From:
Victor Efimov via RT
Date:
December 27, 2012 20:59
Subject:
[perl #63674] open() is not UTF-8-clean
Message ID:
rt-3.6.HEAD-17500-1356424488-1724.63674-15-0@perl.org
This code
perl -lwe '$a="\x{e3}"; utf8::downgrade($a); open(my $x, ">", "x$a");
utf8::upgrade($a); open(my $y, ">", "y$a"); opendir(my $d, ".");
while(defined($_ = readdir($d))) { print unpack("H*", $_) unless /\A[
-~]*\z/ }'
actually sends different octets to open().
see here:
perl -MDevel::Peek -lwe '$a="\x{e3}"; utf8::downgrade($a); print
Dump("x$a"); utf8::upgrade($a); print Dump("y$a");'
SV = PV(0x1b45c68) at 0x1b694e8
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK)
PV = 0x1b5fad0 "x\343"\0
CUR = 2
LEN = 8
SV = PV(0x1b45b58) at 0x1b730a0
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK,UTF8)
PV = 0x1b63b90 "y\303\243"\0 [UTF8 "y\x{e3}"]
CUR = 3
LEN = 8
On Fri Mar 06 03:03:13 2009, zefram@fysh.org wrote:
> This is a bug report for perl from zefram@fysh.org,
> generated with the help of perlbug 1.36 running under perl 5.10.0.
>
>
> -----------------------------------------------------------------
> [Please enter your report here]
>
> $ perl -lwe '$a="\x{e3}"; utf8::downgrade($a); open(my $x, ">",
> "x$a"); utf8::upgrade($a); open(my $y, ">", "y$a"); opendir(my $d,
> "."); while(defined($_ = readdir($d))) { print unpack("H*", $_)
> unless /\A[ -~]*\z/ }'
> 78e3
> 79c3a3
> $
>
> Apparently open() is using, for the filename, the octet sequence used
> to represent the string internally, rather than the character sequence
> that the string actually represents. This is a common problem with
> XS modules; I'm a bit surprised to see the core get it wrong too.
> (Not *very* surprised, though, because the way the SvUTF8 flag was
> injected invites this sort of mistake.)
>
> [Please do not change anything below this line]
> -----------------------------------------------------------------
> ---
> Flags:
> category=core
> severity=medium
> ---
> Site configuration information for perl 5.10.0:
>
> Configured by Debian Project at Thu Jan 1 12:43:38 UTC 2009.
>
> Summary of my perl5 (revision 5 version 10 subversion 0)
> configuration:
> Platform:
> osname=linux, osvers=2.6.26-1-686, archname=i486-linux-gnu-thread-
> multi
> uname='linux rebekka 2.6.26-1-686 #1 smp mon dec 15 18:15:07 utc
> 2008 i686 gnulinux '
> config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
> -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr
> -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10
> -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
> -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
> -Dsitelib=/usr/local/share/perl/5.10.0
> -Dsitearch=/usr/local/lib/perl/5.10.0 -Dman1dir=/usr/share/man/man1
> -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
> -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
> -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio
> -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib
> -Dlibperl=libperl.so.5.10.0 -Dd_dosuid -des'
> hint=recommended, useposix=true, d_sigaction=define
> useithreads=define, usemultiplicity=define
> useperlio=define, d_sfio=undef, uselargefiles=define,
> usesocks=undef
> use64bitint=undef, use64bitall=undef, uselongdouble=undef
> usemymalloc=n, bincompat5005=undef
> Compiler:
> cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN
> -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64',
> optimize='-O2 -g',
> cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing
> -pipe -I/usr/local/include'
> ccversion='', gccversion='4.3.2', gccosandvers=''
> intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
> d_longlong=define, longlongsize=8, d_longdbl=define,
> longdblsize=12
> ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
> lseeksize=8
> alignbytes=4, prototype=define
> Linker and Libraries:
> ld='cc', ldflags =' -L/usr/local/lib'
> libpth=/usr/local/lib /lib /usr/lib /usr/lib64
> libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
> perllibs=-ldl -lm -lpthread -lc -lcrypt
> libc=/lib/libc-2.7.so, so=so, useshrplib=true,
> libperl=libperl.so.5.10.0
> gnulibc_version='2.7'
> Dynamic Linking:
> dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
> cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib'
>
> Locally applied patches:
>
>
> ---
> @INC for perl 5.10.0:
> /etc/perl
> /usr/local/lib/perl/5.10.0
> /usr/local/share/perl/5.10.0
> /usr/lib/perl5
> /usr/share/perl5
> /usr/lib/perl/5.10
> /usr/share/perl/5.10
> /usr/local/lib/site_perl
> .
>
> ---
> Environment for perl 5.10.0:
> HOME=/home/zefram
> LANG (unset)
> LANGUAGE (unset)
> LD_LIBRARY_PATH (unset)
> LOGDIR (unset)
> PATH=/home/zefram/pub/i686-pc-linux-
>
gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/local/bin:/usr/games
> PERL_BADLANG (unset)
> SHELL=/usr/bin/zsh
---
via perlbug: queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=63674
Thread Previous