Front page | perl.perl5.porters |
Postings from November 2010
On 28 November 2010 23:36, Reverend Chip <rev.chip@gmail.com> wrote: > On 11/28/2010 2:31 PM, Nicholas Clark wrote: >> On Sun, Nov 28, 2010 at 02:26:51PM -0800, Reverend Chip wrote: >>> On 11/28/2010 2:34 AM, demerphq wrote: >>>> However if it means we have to validate the string every time we do a >>>> utf8 operation then I would say you are wrong. >>> Slippery slope fallacy. >> Why so? > > Because "validate" implies a level of caution that I have never > requested, and never will. At most I've requested "don't actually > crash" with a side order of "show me where I need to add protective > calls to utf8::validate", which would be satisfied by, for example, > avoiding invalid C pointers caused by bad utf8 and turning any > invalid-utf8-caused asserts to croaks. How else do you think it could work? If we don't do it in a single pass through the data we would end up having to do it every time we extract the length information from a high bit set start byte. In the regex engine that could be over and over and over.... The perl internals as general rule assume that you can read characters in utf8 just as you might with normal C strings. That is, that if the character you are reading starts at a safe position in memory that it can be read in its entirety. What you are saying is that this should not be the case, which essentially means we have to validate every character we read. Which brings us back to the "ok then lets just validate it once, and where do we store the information that we have done so", and right back with the points you so blithely dismissed. cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"Thread Previous | Thread Next