develooper Front page | perl.perl5.porters | Postings from June 2004

Re: [perl #29841] utf8::decode doesn't work under -T

Thread Previous | Thread Next
From:
Nick Ing-Simmons
Date:
June 1, 2004 00:50
Subject:
Re: [perl #29841] utf8::decode doesn't work under -T
Message ID:
20040601075033.3608.1@llama.elixent.com
Stas Bekman <stas@stason.org> writes:
>SADAHIRO Tomoyuki wrote:
>> On Sun, 30 May 2004 21:43:38 -0700
>> Stas Bekman <stas@stason.org> wrote:
>> 
>> 
>>>So it's unclear whose fault it is, the code that populates sv with pv under -T 
>>>or Perl_sv_utf8_decode(), which may need to check pPOK instead of POK?
>> 
>> 
>> You can use <$_ = Encode::decode('utf8', $_)>
>> instead of <utf8::decode $_>.
>> Encode::decode('utf8') is based on Encode::utf8::decode_xs() and
>> it does not use "broken" sv_utf8_decode().
>> I think it should work well even if taint mode.
>
>Thank you, Sadahiro. It indeed fixes my simple test case (I haven't tested 
>that solution in the application yet).
>
>perl-5.8.0 -T -MEncode -MDevel::Peek -le '$_ = shift; Dump $_; $_ = 
>Encode::decode('utf8', $_); Dump $_' `perl -le 'binmode STDOUT, ":utf8"; print 
>"\x{0432}\x{0430}"'`
>
>Shouldn't sv_utf8_decode() (and potentially other utf8 functions) then be 
>replaced with Encode::utf8::decode_xs() (and equivalents from Encode?).

sv_utf8_decode is an internals thing that is exported to help out 
XS code at perl5.6 or so. We have discussed the pPOK vs POK thing before
and I forget the final outcome. 

Normally you have to do SvGETMAGIC() to get POK flag on, and I think 
internals routine assumes caller does that.

We could change that, but that won't help older perls.





Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About