[perl #29720] $PerlIO::encoding::fallback = FB_DEFAULT leads to duplicated output

# New Ticket Created by  Andrew Alakozow 
# Please include the string:  [perl #29720]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org:80/rt3/Ticket/Display.html?id=29720 >


To: perlbug@perl.org
Subject: $PerlIO::encoding::fallback = FB_DEFAULT leads to duplicated output
Reply-To: aa29@mail.ru
Message-Id: <5.8.4_2160_1084980780@INFORMED>

This is a bug report for perl from aa29@mail.ru,
generated with the help of perlbug 1.35 running under perl v5.8.4.

$PerlIO::encoding::fallback = FB_DEFAULT leads to duplicated output.

It is possible to change check-mode via $PerlIO::encoding::fallback:

use Encode qw(:fallback_all);
use encoding 'utf8';

$PerlIO::encoding::fallback = FB_DEFAULT;

binmode(STDERR, ":encoding(cp866)");
warn "foobar";

This code gives four messages instead of one:

foobar at 6.pl line 7.
foobar at 6.pl line 7.
foobar at 6.pl line 7.
foobar at 6.pl line 7.

And with redirection STDERR to file it gives three messages:

foobar at 6.pl line 7.
foobar at 6.pl line 7.
foobar at 6.pl line 7.

Further investigation shows that there is no duplication if
$PerlIO::encoding::fallback = FB_DEFAULT | FB_PERLQQ; # or FB_(HT|X)MLCREF

Looking into ext\Encode\Encode.xs I found such code
(ext\Encode\Encode.xs, line 229):

    if (check && !(check & ENCODE_LEAVE_SRC)){
 sdone = SvCUR(src) - (slen+sdone);
 if (sdone) {
     sv_setpvn(src, (char*)s+slen, sdone);
 }
 SvCUR_set(src, sdone);
    }

If check is set to FB_DEFAULT (which is 0) and no other fallback is
defined, then it behaves as if ENCODE_LEAVE_SRC is set, and buffer
does not became truncated, and then it will be flushed several times.
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.8.4:

Configured by aa29 at Mon May 17 17:59:46 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags
='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE
_DES_FCRYPT  -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL
_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags
'-nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"c:\perl\lib\CORE"  -ma
chine:x86'
    libpth=\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ',
lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf  -libpath:"c:\perl
\lib\CORE"  -machine:x86'

Locally applied patches:


---
@INC for perl v5.8.4:
    C:/Perl/lib
    C:/Perl/site/lib
    .

---
Environment for perl v5.8.4:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=C:\cygwin\bin;C:\Tcl\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\Syst
em32\Wbem;C:\Perl\bin;C:\Program Files\Support Tools;D:\src\lib;C:\Program
Files\Microsoft Visual Studio\Common\Tools\WinNT;C:\Program Files\Microsoft
Visual Studio\Common\MSDev98\Bin;C:\Program Files\Microsoft Visual
Studio\Common\Tools;C:\Program Files\Microsoft Visual
Studio\VC98\bin;C:\Arc;C:\Program Files\Utils;C:\Mysql\bin;C:\Program
Files\Debugging Tools for Windows;C:\Tcl\bin;D:\Linda\XML\fop;C:\Program
Files\GNU\WinCvs 1.2;D:\src\bin;C:\Program Files\Far
    PERL_BADLANG (unset)
    SHELL (unset)

aa29

• [perl #29720] $PerlIO::encoding::fallback = FB_DEFAULT leads to duplicated output by Andrew Alakozow