develooper Front page | perl.perl5.porters | Postings from December 2012

[perl #109828] PerlIO::scalar does not handle UTF-8

Thread Previous | Thread Next
From:
Tony Cook via RT
Date:
December 28, 2012 22:07
Subject:
[perl #109828] PerlIO::scalar does not handle UTF-8
Message ID:
rt-3.6.HEAD-17500-1356732417-1200.109828-15-0@perl.org
On Fri Dec 28 07:45:25 2012, LeonT wrote:
> On Fri, Dec 28, 2012 at 8:34 AM, Peter Rabbitson
> <rabbit+p5p@rabbit.us> wrote:
> > Is there any word on this issue? I just hit this bug in reverse[1]
> and
> > while there is ample discussion about it being a problem I see the
> same
> > behavior under current blead. Is there a chance *at least* for a
> warning
> > to be added so that it lands in 5.18?
> 
> The process kind of fizzled somewhere. I'm in favor of a warning in
> 5.18, see attachment.

It should fail to open.  If you open a UTF8 flagged string for append
and write non-UTF8 bytes you will produce an invalidly encoded SvUTF8
string.

Your patch as written ignores the principle that the SvUTF8() flag only
controls the internal encoding, not other behaviour.  If the SV contains
only code point 0xFF or lower we should downgrade it and work with that
rather than failing (or producing a warning).

This should also be done for _read() and _write(), since the SV can be
modified between I/O operations.

There's an unrelated problem that _pushed() checks flags on both arg and
SvRV(arg) without calling SvGETMAGIC().

I'll take a look at these issues when I get home.

Tony


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=109828

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About