develooper Front page | perl.perl5.porters | Postings from February 2018

[perl #132665] Always use STOP_AT_PARTIAL in PerlIO::encoding

From:
Leon Timmermans via RT
Date:
February 4, 2018 12:10
Subject:
[perl #132665] Always use STOP_AT_PARTIAL in PerlIO::encoding
Message ID:
rt-4.0.24-25182-1517746184-1493.132665-15-0@perl.org
On Fri, 29 Dec 2017 05:11:38 -0800, LeonT wrote:
> PerlIO::encoding has a $fallback variable that allows one to set the
> behavior on a encoding/decoding error, for example to make it throw an
> exception on error. What is not documented (actually the example in
> the
> documentation is even missing this) is that PerlIO::encoding needs the
> (equally undocumented) Encode::STOP_AT_PARTIAL flag to be set,
> otherwise
> a multi-byte character spanning buffer boundaries will be interpreted
> as
> two invalid byte sequences. I could have fixed the documentation, but
> instead I fixed the code to always pass this flag to Encode,
> simplifying
> the use and making the current documentation correct again.

It turned out that Encode allows one to pass a coderef instead of a set of flags to handle. This however doesn't allow one to pass STOP_AT_PARTIAL, which means it has always been buggy on buffer boundaries. With my new automatic STOP_AT_PARTIAL passing this would result in an unpredictable value (based on the pointer value of the CV), this is clearly undesirable.

Instead we now disallow it in PerlIO::encoding. This could be changed at some point in the future once Encode adds support for it.

Leon

---
via perlbug:  queue: perl5 status: new
https://rt.perl.org/Ticket/Display.html?id=132665



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About