develooper Front page | perl.perl5.porters | Postings from December 2018

[perl #132665] Always use STOP_AT_PARTIAL in PerlIO::encoding

James E Keenan via RT
December 6, 2018 20:31
[perl #132665] Always use STOP_AT_PARTIAL in PerlIO::encoding
Message ID:
On Sun, 04 Feb 2018 12:09:44 GMT, LeonT wrote:
> On Fri, 29 Dec 2017 05:11:38 -0800, LeonT wrote:
> > PerlIO::encoding has a $fallback variable that allows one to set the
> > behavior on a encoding/decoding error, for example to make it throw
> > an
> > exception on error. What is not documented (actually the example in
> > the
> > documentation is even missing this) is that PerlIO::encoding needs
> > the
> > (equally undocumented) Encode::STOP_AT_PARTIAL flag to be set,
> > otherwise
> > a multi-byte character spanning buffer boundaries will be interpreted
> > as
> > two invalid byte sequences. I could have fixed the documentation, but
> > instead I fixed the code to always pass this flag to Encode,
> > simplifying
> > the use and making the current documentation correct again.
> It turned out that Encode allows one to pass a coderef instead of a
> set of flags to handle. This however doesn't allow one to pass
> STOP_AT_PARTIAL, which means it has always been buggy on buffer
> boundaries. With my new automatic STOP_AT_PARTIAL passing this would
> result in an unpredictable value (based on the pointer value of the
> CV), this is clearly undesirable.
> Instead we now disallow it in PerlIO::encoding. This could be changed
> at some point in the future once Encode adds support for it.
> Leon

Would anyone like to comment on these two patches?

Thank you very much.
James E Keenan (

via perlbug:  queue: perl5 status: new Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About