develooper Front page | perl.perl5.porters | Postings from April 2013

[perl #117429] UTF8 error messages ($!, $@); use open qw{:utf8 :std}

Thread Previous
From:
Dominic Hargreaves
Date:
April 1, 2013 17:43
Subject:
[perl #117429] UTF8 error messages ($!, $@); use open qw{:utf8 :std}
Message ID:
rt-3.6.HEAD-28177-1364838164-898.117429-75-0@perl.org
# New Ticket Created by  Dominic Hargreaves 
# Please include the string:  [perl #117429]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=117429 >



This is a bug report for perl from dom@earth.li,
generated with the help of perlbug 1.39 running under perl 5.17.10.


-----------------------------------------------------------------
[Please describe your issue here]

As reported in Debian's bugtracker at
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=409704>

Quoting from Joey on the bug report:

"
joey@kodama:~>LANG=fr_FR.UTF-8 perl -e 'use open qw{:utf8 :std};open(foo) || print STDERR "error: $!\n";'
error: Aucun fichier ou répertoire de ce type
                         ^^
This mojibake comes about because $! is a UTF-8 string in that locale, but it
is not decoded into perl's internal utf8 representation.

It's possible to work around the problem with the encoding pragma, but
not completely:

joey@kodama:~>LANG=fr_FR.UTF-8 perl -e 'use open qw{:utf8 :std}; use encoding 'utf8';open(foo) || print STDERR "error: $!\n";'    
error: Aucun fichier ou répertoire de ce type

joey@kodama:~>LANG=fr_FR.UTF-8 perl -e 'use open qw{:utf8 :std}; use encoding 'utf8';open(foo) || print STDERR "error: ",$!,"\n";'
error: Aucun fichier ou répertoire de ce type

The first example works because the encoding pragma converts the string
to utf8 during concacenation, but the second example shows that this is
not a solution because concacentation can't be relied on for all output.

The only solution if you want to use open qw{:utf8 :std} in a program
seems to be manually using Encode::decode_utf8 on every instance of $!
and $@ in the program. Which is exactly the kind of error-prone busywork
that IO layers and perl's unicode model are supposed to avoid.."

The test in question no longer works on my Debian system, because
the French error message no longer contains an accent, but the same
behaviour can be reproduced using eg ja_JP.UTF-8, and also on 5.17.10.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=low
---
Site configuration information for perl 5.17.10:

Configured by dom at Sun Mar 31 23:53:26 BST 2013.

Summary of my perl5 (revision 5 version 17 subversion 10) configuration:
   
  Platform:
    osname=linux, osvers=3.2.0-4-686-pae, archname=i686-linux
    uname='linux callisto 3.2.0-4-686-pae #1 smp debian 3.2.39-2 i686 gnulinux '
    config_args='-de -Dprefix=/home/dom/perl5/perlbrew/perls/perl-5.17.10 -Dusedevel'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.13'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    

---
@INC for perl 5.17.10:
    /home/dom/perl5/perlbrew/perls/perl-5.17.10/lib/site_perl/5.17.10/i686-linux
    /home/dom/perl5/perlbrew/perls/perl-5.17.10/lib/site_perl/5.17.10
    /home/dom/perl5/perlbrew/perls/perl-5.17.10/lib/5.17.10/i686-linux
    /home/dom/perl5/perlbrew/perls/perl-5.17.10/lib/5.17.10
    .

---
Environment for perl 5.17.10:
    HOME=/home/dom
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/dom/perl5/perlbrew/bin:/home/dom/perl5/perlbrew/perls/perl-5.17.10/bin:~/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERLBREW_BASHRC_VERSION=0.43
    PERLBREW_HOME=/home/dom/.perlbrew
    PERLBREW_MANPATH=/home/dom/perl5/perlbrew/perls/perl-5.17.10/man
    PERLBREW_PATH=/home/dom/perl5/perlbrew/bin:/home/dom/perl5/perlbrew/perls/perl-5.17.10/bin
    PERLBREW_PERL=perl-5.17.10
    PERLBREW_ROOT=/home/dom/perl5/perlbrew
    PERLBREW_VERSION=0.43
    PERL_BADLANG (unset)
    SHELL=/bin/bash


Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About