[perl #131115] Perl enters infinite loop with -d:Trace andbinmode(STDERR, encoding(UTF-8))

On Fri, 07 Apr 2017 06:13:38 -0700, alh wrote:
> Originally brought to my attention by tudor on #moose:
> 
> These are fine:
> 
> perl -d:Trace -e 'binmode(STDERR, ":encoding(UTF-8)");' 2>&1
> perl -d:Trace -e 'binmode(STDERR, ":encoding(UTF-8)");' | tee out.txt
> 
> These enter infinite loops:
> 
> perl -d:Trace -e 'binmode(STDERR, ":encoding(UTF-8)");' 2>&1 | tee
> out.txt
> perl -d:Trace -e 'binmode(STDERR, ":encoding(UTF-8)");' 2>&1 | grep
> thing
> 
> CPU is at 100%, and if I attach to perl with gdb and grab a backtrace:
> 
> (gdb) bt
> #0  0x00007fbb8a4a6780 in Perl_PerlIO_flush () from
> /usr/lib/libperl.so.5.18
> #1  0x00007fbb8a4a761c in PerlIOBuf_write () from
> /usr/lib/libperl.so.5.18
> #2  0x00007fbb8a4855f5 in Perl_do_print () from
> /usr/lib/libperl.so.5.18
> #3  0x00007fbb8a40dae7 in Perl_write_to_stderr () from
> /usr/lib/libperl.so.5.18
> #4  0x00007fbb8a468c9d in Perl_die_unwind () from
> /usr/lib/libperl.so.5.18
> #5  0x00007fbb8a40e5a9 in Perl_vcroak () from /usr/lib/libperl.so.5.18
> #6  0x00007fbb8a40efa7 in Perl_croak () from /usr/lib/libperl.so.5.18
> #7  0x00007fbb8a42b6ca in ?? () from /usr/lib/libperl.so.5.18
> #8  0x00007fbb8a433eea in Perl_pp_method () from
> /usr/lib/libperl.so.5.18
> #9  0x00007fbb8a42b266 in Perl_runops_standard () from
> /usr/lib/libperl.so.5.18
> #10 0x00007fbb8a3bc25c in Perl_call_sv () from
> /usr/lib/libperl.so.5.18
> #11 0x00007fbb88bf4ca0 in PerlIOEncode_flush ()
>    from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
> #12 0x00007fbb8a4a67bf in Perl_PerlIO_flush () from
> /usr/lib/libperl.so.5.18
> #13 0x00007fbb8a4a761c in PerlIOBuf_write () from
> /usr/lib/libperl.so.5.18
> #14 0x00007fbb8a4855f5 in Perl_do_print () from
> /usr/lib/libperl.so.5.18
> #15 0x00007fbb8a42d4cf in Perl_pp_print () from
> /usr/lib/libperl.so.5.18
> #16 0x00007fbb8a42b266 in Perl_runops_standard () from
> /usr/lib/libperl.so.5.18
> #17 0x00007fbb8a3bc25c in Perl_call_sv () from
> /usr/lib/libperl.so.5.18
> #18 0x00007fbb88bf3459 in PerlIOEncode_pushed ()
>     from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
> #19 0x00007fbb8a4a630e in PerlIO_push () from /usr/lib/libperl.so.5.18
> #20 0x00007fbb8a4a6449 in PerlIO_apply_layera () from
> /usr/lib/libperl.so.5.18
> ---Type <return> to continue, or q <return> to quit---
> #21 0x00007fbb8a4a6528 in PerlIO_apply_layers () from
> /usr/lib/libperl.so.5.18
> #22 0x00007fbb8a4a6591 in PerlIO_binmode () from
> /usr/lib/libperl.so.5.18
> #23 0x00007fbb8a472ba0 in Perl_pp_binmode () from
> /usr/lib/libperl.so.5.18
> #24 0x00007fbb8a42b266 in Perl_runops_standard () from
> /usr/lib/libperl.so.5.18
> #25 0x00007fbb8a3c3b44 in perl_run () from /usr/lib/libperl.so.5.18
> #26 0x0000000000400dd9 in main ()
> 
> Also tested with 5.24.1, and blead.

I don't have Devel::Trace installed (I'm working with an uninstalled blead tree), but I see an infinite loop with -Dt with a -DDEBUGGING build too.

I expect what's happening in your case is Devel::Trace is trying to trace the call to $encoding->renew() or $encoding->needs_lines() (both of which are done with G_EVAL), writing to STDERR, which tries to use the partly initialised PerlIO::encoding layer and dies, which tries to write to the stream calling PerlIO_flush().

This loops because PerlIO::encoding prevents recursive calls to $encoding->encode(), losing the output rather than recursving infinitely.

With -Dt (in blead at least) it seems to get stuck earlier in the call to Encode::find_encoding(), but you get a similar result.

The debugger doesn't seem to have a problem with this code, presumably because it either clones STDERR or opens the TTY.

Possible fixes:

- Devel::Trace could clone STDERR and write to that clone, this should work for older perls too.

- perl's -D switch could do something similar.

- perl (perhaps in PerlIO::encoding) could disable both -Dt tracing and debugger tracing when pushing a layer onto or writing to STDERR.  This might be a bit too magical though.

Tony

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=131115

• [perl #131115] Perl enters infinite loop with -d:Trace andbinmode(STDERR, encoding(UTF-8)) by Tony Cook via RT