develooper Front page | perl.perl5.porters | Postings from January 2003

Re: [PATCH] enable locale-induced UTF-8 I/O only if explicitly asked (Was: Re: [perl #19743] implicit utf8ification causes action-at-distance bugs)

Thread Previous
From:
Benjamin Goldberg
Date:
January 14, 2003 14:47
Subject:
Re: [PATCH] enable locale-induced UTF-8 I/O only if explicitly asked (Was: Re: [perl #19743] implicit utf8ification causes action-at-distance bugs)
Message ID:
3E2495BE.E1E91768@earthlink.net
Jarkko Hietaniemi wrote:
> 
> In our previous episode we found out that there were two problems
> inherent in the implicit UTF-8-ification:
> 
> (1) The UTF-8 kicked in even when the user didn't ask for it.
>     Lots of people using RH 8.0 have been bitten by this because
>     the default locales are UTF-8.
[snip]
> So the issue (1) still would remain but the following patch attempts
> to rectify the situation, by making the UTF-8-ification explicit
> instead of implicit.

I've a foolish question -- if, on *nix, we're making handles binary by
default, and text mode only when asked, then ought we do the same on
windows?  No, I don't think we actually should, but it would be
logically consistant :-).

Perhaps more important -- on redhat8, if we write a file in latin1, or
any other non-utf8 mode, how will that file treated by other utilities?

On windows, for example, if we write a file in binary mode, and print
out mere "\n" chars between lines, then some utilities will break, due
to them expecting CRLF.

I fear (but don't know for certain) that if we produce files with bytes
whose high bits are set, and which aren't properly formed utf8, then at
least some text processing utilities will whine about malformed utf8
characters.

-- 
$..='(?:(?{local$^C=$^C|'.(1<<$_).'})|)'for+a..4;
$..='(?{print+substr"\n !,$^C,1 if $^C<26})(?!)';
$.=~s'!'haktrsreltanPJ,r  coeueh"';BEGIN{${"\cH"}
|=(1<<21)}""=~$.;qw(Just another Perl hacker,\n);

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About