develooper Front page | perl.perl5.porters | Postings from September 2006

[perl #40401] PerlIO::encoding doesn't handle fallback modes correctly?

From:
Steve Hay
Date:
September 22, 2006 09:43
Subject:
[perl #40401] PerlIO::encoding doesn't handle fallback modes correctly?
Message ID:
rt-3.5.HEAD-31263-1158943160-1104.40401-75-0@perl.org
# New Ticket Created by  Steve Hay 
# Please include the string:  [perl #40401]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=40401 >


This is a bug report for perl from steveh@Mugwump.uk.radan.com,
generated with the help of perlbug 1.35 running under perl 5.9.4.


-----------------------------------------------------------------
[Please enter your report here]

I'm trying to use the FB_HTMLCREF fallback mode with PerlIO::encoding, 
but it isn't doing what I expect.

If I set $PerlIO::encoding::fallback to Encode::FB_HTMLCREF (or, 
equivalently, to Encode::HTMLCREF | Encode::LEAVE_SRC) and set the 
STDOUT encoding layer to 'encoding(windows-1252)' then any output to 
STDOUT seems to be repeated, regardless of whether the fallback 
mechanism comes into play or not:

 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::FB_HTMLCREF; binmode STDOUT, ':encoding(windows-1252)'; print 
qq[\x{0104}]"
&#260;&#260;
 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::HTMLCREF | Encode::LEAVE_SRC; binmode STDOUT, 
':encoding(windows-1252)'; print qq[\x{0104}]"
&#260;&#260;
 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::FB_HTMLCREF; binmode STDOUT, ':encoding(windows-1252)'; print 
qq[abc]"
abcabc
 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::HTMLCREF | Encode::LEAVE_SRC; binmode STDOUT, 
':encoding(windows-1252)'; print qq[abc]"
abcabc

Using the HTMLCREF bitmask flag without the LEAVE_SRC flag seems to fix 
this, but unless I'm missing something the above behaviour is not correct:

 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::HTMLCREF; binmode STDOUT, ':encoding(windows-1252)'; print 
qq[\x{0104}]"
&#260;
 >.\perl -MEncode -MPerlIO::encoding -e "$PerlIO::encoding::fallback = 
Encode::HTMLCREF; binmode STDOUT, ':encoding(windows-1252)'; print qq[abc]"
abc

Encode::encode() doesn't have this problem (and leaving out the 
LEAVE_SRC flag in that case, of course, causes trouble if the source 
string cannot be modified):

 >.\perl -MEncode -e "print encode('windows-1252', qq[\x{0104}], 
Encode::FB_HTMLCREF)"
&#260;
 >.\perl -MEncode -e "print encode('windows-1252', qq[\x{0104}], 
Encode::HTMLCREF | Encode::LEAVE_SRC)"
&#260;
 >.\perl -MEncode -e "print encode('windows-1252', qq[\x{0104}], 
Encode::HTMLCREF)"
Modification of a read-only value attempted at 
C:/p5p/bleadperl/lib/Encode.pm line 147.

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
     category=library
     severity=medium
---
Site configuration information for perl 5.9.4:

Configured by steveh at Fri Sep 22 16:42:49 2006.

Summary of my perl5 (revision 5 version 9 subversion 4) configuration:
   Platform:
     osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread
     uname=''
     config_args='undef'
     hint=recommended, useposix=true, d_sigaction=undef
     useithreads=define, usemultiplicity=define
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=undef, use64bitall=undef, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT 
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
     optimize='-MD -Zi -DNDEBUG -O1',
     cppflags='-DWIN32'
     ccversion='12.00.8804', gccversion='', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, 
Off_t='__int64', lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf 
-libpath:"c:\perl\lib\CORE"  -machine:x86'
     libpth=C:\PROGRA~1\MICROS~2\VC98\lib
     libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib 
  comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib 
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib 
odbc32.lib odbccp32.lib msvcrt.lib
     perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib 
winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib 
oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib 
version.lib odbc32.lib odbccp32.lib msvcrt.lib
     libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl59.lib
     gnulibc_version=''
   Dynamic Linking:
     dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
     cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug 
-opt:ref,icf  -libpath:"c:\perl\lib\CORE"  -machine:x86'

Locally applied patches:


---
@INC for perl 5.9.4:
     C:/perl/lib
     C:/perl/site/lib
     .

---
Environment for perl 5.9.4:
     HOME (unset)
     LANG (unset)
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
 
PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\perl\bin
     PERL_BADLANG (unset)
     SHELL (unset)


-- 


------------------------------------------------
Radan Computational Ltd.

The information contained in this message and any files transmitted with it are confidential and intended for the addressee(s) only. If you have received this message in error or there are any problems, please notify the sender immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Radan Computational Ltd. The recipient(s) of this message should check it and any attached files for viruses: Radan Computational will accept no liability for any damage caused by any virus transmitted by this email.




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About