develooper Front page | perl.perl5.porters | Postings from July 2011

normalization filters?

Thread Next
Tom Christiansen
July 20, 2011 11:02
normalization filters?
Message ID:
Is there any reason one couldn't, or even shouldn't, have 
normalization I/O layers available?

    open($fh_in,  "< :encoding(UTF-8) :normalize(NFD)", $input_file)
    open($fh_out, "> :encoding(UTF-8) :normalize(NFC)", $output_file)

Put another way, is there anything wrong with the general advice to NFD 
incoming data and to NFC outgoing data?  

Are there any particular provisos called for?

I know of various "services" that already seem to do this, 
and not too many people get *too* upset by it.

On another matter, even though I sometimes think of diddling $/ for it, 
I still wonder whether a \R-sensitive I/O layer is the best way to handle
input with unknown linebreaks.  But unless we convert those all to \n,
chomp() is going to be messy.


Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About