develooper Front page | perl.perl5.porters | Postings from July 2014

Re: Encode vs. JSON

Thread Previous | Thread Next
David E. Wheeler
July 21, 2014 18:22
Re: Encode vs. JSON
Message ID:
On Jul 19, 2014, at 9:58 PM, David E. Wheeler <> wrote:

>> there is a ticket about that:
> Ah, interesting. I had not run into that warning. What I ran into with Encode I now think should be changed:
>    perl -MEncode -E 'say Encode::decode("UTF-8", "\xEF\xBF\xBF", Encode::FB_CROAK)' 
>    utf8 "\xFFFF" does not map to Unicode at /usr/local/lib/perl5/site_perl/5.20.0/darwin-thread-multi-2level/ line 175.
> In fact it *does* map to Unicode, IIUC Corrigendum 9 correctly. I’ll file a bug with Dan.

I did so, here:

Dan replied to report that it’s UTF8_DISALLOW_ILLEGAL_INTERCHANGE from the Perl core that’s at fault:

> If it were are a bug, it belongs to perl core because the strictness of UTF8 is #defined in the value of UTF8_DISALLOW_ILLEGAL_INTERCHANGE which is defined in perl core:
> In other words, Encode faithfully believes perl core with that respect.  And I want to leave Encode that way. If it is to be fixed, it should be fixed by redefining UTF8_DISALLOW_ILLEGAL_INTERCHANGE to exclude UTF8_DISALLOW_NONCHAR in perl core.

ISTM that, given the change in Corrigendum 9, UTF8_DISALLOW_ILLEGAL_INTERCHANGE should exclude UTF8_DISALLOW_NONCHAR.

Is this part of of the same issue as that described in RT-97358? Or should I start a new issue?



Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About