develooper Front page | perl.perl5.porters | Postings from July 2014

Re: Encode vs. JSON

Thread Previous | Thread Next
From:
David E. Wheeler
Date:
July 21, 2014 18:22
Subject:
Re: Encode vs. JSON
Message ID:
9621B74E-5DD5-446B-B12E-D33BA1D2994C@justatheory.com
On Jul 19, 2014, at 9:58 PM, David E. Wheeler <david@justatheory.com> wrote:

>> there is a ticket about that:
>> https://rt.perl.org/Public/Bug/Display.html?id=121937
> 
> Ah, interesting. I had not run into that warning. What I ran into with Encode I now think should be changed:
> 
>    perl -MEncode -E 'say Encode::decode("UTF-8", "\xEF\xBF\xBF", Encode::FB_CROAK)' 
>    utf8 "\xFFFF" does not map to Unicode at /usr/local/lib/perl5/site_perl/5.20.0/darwin-thread-multi-2level/Encode.pm line 175.
> 
> In fact it *does* map to Unicode, IIUC Corrigendum 9 correctly. I’ll file a bug with Dan.

I did so, here:

  https://rt.cpan.org/Ticket/Display.html?id=97358

Dan replied to report that it’s UTF8_DISALLOW_ILLEGAL_INTERCHANGE from the Perl core that’s at fault:

> If it were are a bug, it belongs to perl core because the strictness of UTF8 is #defined in the value of UTF8_DISALLOW_ILLEGAL_INTERCHANGE which is defined in perl core:
> 
> http://perldoc.perl.org/perlapi.html#Unicode-Support
> 
> In other words, Encode faithfully believes perl core with that respect.  And I want to leave Encode that way. If it is to be fixed, it should be fixed by redefining UTF8_DISALLOW_ILLEGAL_INTERCHANGE to exclude UTF8_DISALLOW_NONCHAR in perl core.


ISTM that, given the change in Corrigendum 9, UTF8_DISALLOW_ILLEGAL_INTERCHANGE should exclude UTF8_DISALLOW_NONCHAR.

Is this part of of the same issue as that described in RT-97358? Or should I start a new issue?

Best,

David


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About