develooper Front page | perl.perl5.porters | Postings from March 2011

Re: (locales) Re: Smoke [5.13.10] v5.13.10-384-g3dfaac4 FAIL(F) openbsd4.8 (i386/1 cpu)

Thread Previous
From:
Karl Williamson
Date:
March 13, 2011 21:52
Subject:
Re: (locales) Re: Smoke [5.13.10] v5.13.10-384-g3dfaac4 FAIL(F) openbsd4.8 (i386/1 cpu)
Message ID:
4D7D9EB5.6090808@khwilliamson.com
On 03/13/2011 09:27 AM, Craig A. Berry wrote:
> On Sat, Mar 12, 2011 at 7:31 PM, Karl Williamson
> <public@khwilliamson.com>  wrote:
>> On 03/12/2011 11:12 AM, Steven Schubiger wrote:
>>>
>>> Automated smoke report for 5.13.10 patch
>>> 3dfaac447d030f911d146c3ae56b9dba63ce9dd4 v5.13.10-384-g3dfaac4
>>> p5openbsd: AMD Athlon(tm) Processor ("AuthenticAMD" 686-class, 256KB L2
>>> cache) (909 MHz) (i386/1 cpu)
>>>      on        openbsd - 4.8
>>>      using     cc version 4.2.1 20070719
>>>      smoketime 17 hours 37 minutes (average 2 hours 12 minutes)
>>>
>>> Summary: FAIL(F)
>>>
>>> O = OK  F = Failure(s), extended report at the bottom
>>> X = Failure(s) under TEST but not under harness
>>> ? = still running or test results not (yet) available
>>> Build failures during:       - = unknown or N/A
>>> c = Configure, m = make, M = make (after miniperl), t = make test-prep
>>>
>>> v5.13.10-384-g3dfaac4  Configuration (common) none
>>> ----------- ---------------------------------------------------------
>>> F F F F
>>> F F F F     -Duse64bitint
>>> F F F F     -Duseithreads
>>> F F F F     -Duseithreads -Duse64bitint
>>> | | | +----- PERLIO = perlio -DDEBUGGING
>>> | | +------- PERLIO = stdio  -DDEBUGGING
>>> | +--------- PERLIO = perlio
>>> +----------- PERLIO = stdio
>>>
>>>
>>> Locally applied patches:
>>>      uncommitted-changes
>>>      SMOKE3dfaac447d030f911d146c3ae56b9dba63ce9dd4
>>>
>>> Failures: (common-args) none
>>> [stdio/perlio]
>>> [stdio/perlio] -Duse64bitint
>>>      ../t/re/charset.t...........................................FAILED
>>>          1621-1624, 1629-1632, 1645-1648, 1657-1660
>>>          1905-1908, 1917-1920, 1941-1944, 1961-1964
>>
>>
>>
>> I have logged in to this box and it appears that the charset.t failures are
>> due to an improper C locale there!   I confirmed this by writing a tiny C
>> program and experimenting.
>>
>> I don't know how to proceed with this.  The C locale is supposed to be the
>> same everywhere, and so I thought it was safe to use it in testing, but
>> apparently not.  I see several options:
>
> FWIW, the exact same tests fail on VMS.  The details look like this:
>
> not ok 1621 - my $a = "\x{bf}"; $a !~ qr/ (?l: [_[:punct:]] ) /x;
> "\x{bf}" is not a [:punct:] in this locale under /l
> not ok 1622 - my $a = "\x{bf}" x 10; $a !~ qr/ (?l: [_[:punct:]]{10} )
> /x; "\x{bf}" is not a [:punct:] in this locale under /l
> not ok 1623 - my $a = "\x{bf}"; $a =~ qr/ (?l: [_[:^punct:]] ) /x;
> "\x{bf}" is a [:^punct:] in this locale under /l
> not ok 1624 - my $a = "\x{bf}" x 10; $a =~ qr/ (?l: [_[:^punct:]]{10}
> ) /x; "\x{bf}" is a [:^punct:] in this locale under /l
> not ok 1629 - my $a = "\x{f7}"; $a !~ qr/ (?l: [_[:print:]] ) /x;
> "\x{f7}" is not a [:print:] in this locale under /l
> not ok 1630 - my $a = "\x{f7}" x 10; $a !~ qr/ (?l: [_[:print:]]{10} )
> /x; "\x{f7}" is not a [:print:] in this locale under /l
> not ok 1631 - my $a = "\x{f7}"; $a =~ qr/ (?l: [_[:^print:]] ) /x;
> "\x{f7}" is a [:^print:] in this locale under /l
> not ok 1632 - my $a = "\x{f7}" x 10; $a =~ qr/ (?l: [_[:^print:]]{10}
> ) /x; "\x{f7}" is a [:^print:] in this locale under /l
> not ok 1645 - my $a = "\x{f7}"; $a !~ qr/ (?l: [_[:graph:]] ) /x;
> "\x{f7}" is not a [:graph:] in this locale under /l
> not ok 1646 - my $a = "\x{f7}" x 10; $a !~ qr/ (?l: [_[:graph:]]{10} )
> /x; "\x{f7}" is not a [:graph:] in this locale under /l
> not ok 1647 - my $a = "\x{f7}"; $a =~ qr/ (?l: [_[:^graph:]] ) /x;
> "\x{f7}" is a [:^graph:] in this locale under /l
> not ok 1648 - my $a = "\x{f7}" x 10; $a =~ qr/ (?l: [_[:^graph:]]{10}
> ) /x; "\x{f7}" is a [:^graph:] in this locale under /l
> not ok 1657 - my $a = "\x{88}"; $a !~ qr/ (?l: [_[:cntrl:]] ) /x;
> "\x{88}" is not a [:cntrl:] in this locale under /l
> not ok 1658 - my $a = "\x{88}" x 10; $a !~ qr/ (?l: [_[:cntrl:]]{10} )
> /x; "\x{88}" is not a [:cntrl:] in this locale under /l
> not ok 1659 - my $a = "\x{88}"; $a =~ qr/ (?l: [_[:^cntrl:]] ) /x;
> "\x{88}" is a [:^cntrl:] in this locale under /l
> not ok 1660 - my $a = "\x{88}" x 10; $a =~ qr/ (?l: [_[:^cntrl:]]{10}
> ) /x; "\x{88}" is a [:^cntrl:] in this locale under /l
> not ok 1905 - my $a = "\x{bf}"; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:punct:]] ) /x; "\x{bf}" is not a [:punct:] in this locale unde
> r /l
> not ok 1906 - my $a = "\x{bf}" x 10; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:punct:]]{10} ) /x; "\x{bf}" is not a [:punct:] in this lo
> cale under /l
> not ok 1907 - my $a = "\x{bf}"; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^punct:]] ) /x; "\x{bf}" is a [:^punct:] in this locale under
> /l
> not ok 1908 - my $a = "\x{bf}" x 10; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^punct:]]{10} ) /x; "\x{bf}" is a [:^punct:] in this loca
> le under /l
> not ok 1917 - my $a = "\x{f7}"; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:print:]] ) /x; "\x{f7}" is not a [:print:] in this locale unde
> r /l
> not ok 1918 - my $a = "\x{f7}" x 10; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:print:]]{10} ) /x; "\x{f7}" is not a [:print:] in this lo
> cale under /l
> not ok 1919 - my $a = "\x{f7}"; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^print:]] ) /x; "\x{f7}" is a [:^print:] in this locale under
> /l
> not ok 1920 - my $a = "\x{f7}" x 10; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^print:]]{10} ) /x; "\x{f7}" is a [:^print:] in this loca
> le under /l
> not ok 1941 - my $a = "\x{f7}"; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:graph:]] ) /x; "\x{f7}" is not a [:graph:] in this locale unde
> r /l
> not ok 1942 - my $a = "\x{f7}" x 10; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:graph:]]{10} ) /x; "\x{f7}" is not a [:graph:] in this lo
> cale under /l
> not ok 1943 - my $a = "\x{f7}"; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^graph:]] ) /x; "\x{f7}" is a [:^graph:] in this locale under
> /l
> not ok 1944 - my $a = "\x{f7}" x 10; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^graph:]]{10} ) /x; "\x{f7}" is a [:^graph:] in this loca
> le under /l
> not ok 1961 - my $a = "\x{88}"; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:cntrl:]] ) /x; "\x{88}" is not a [:cntrl:] in this locale unde
> r /l
> not ok 1962 - my $a = "\x{88}" x 10; utf8::upgrade($a); $a !~ qr/ (?l:
> [_[:cntrl:]]{10} ) /x; "\x{88}" is not a [:cntrl:] in this lo
> cale under /l
> not ok 1963 - my $a = "\x{88}"; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^cntrl:]] ) /x; "\x{88}" is a [:^cntrl:] in this locale under
> /l
> not ok 1964 - my $a = "\x{88}" x 10; utf8::upgrade($a); $a =~ qr/ (?l:
> [_[:^cntrl:]]{10} ) /x; "\x{88}" is a [:^cntrl:] in this loca
> le under /l
>
> So the problems are all with 0xbf, 0xf7, and 0x88.  These don't look
> to me like they're in the portable character set:
>
> http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap06.html
>
> so I think what happens to them in the C/POSIX locale is undefined.

I had thought the definition of all the functions was that anything not 
in in portable character set was supposed to return false.  But a closer 
inspection of the RFC of some version of the standard that I found 
on-line says that may be wrong.  This statement from the C Rationale is 
quite interesting:

"Defect report 19 against C90 clarified that isprint('[') need not be 
true, since it is implementation defined as to what is printable."


> I'm guessing these are continuation bytes of multi-byte sequences?  I
> didn't think the C locale knew anything about such things.

It's not supposed to.  Who knows what these vendors were thinking, but 
since it is on your machine too, it is not just a single outlier.  I 
have logged in to this smoke box, and, tried setting the locale to a 
german one; I think the box is in Germany, so I figured that might be 
set up correctly, but it wasn't.  I'm not sure what to do; I think I'm 
leaning to option 2, until that turns out to not work on some boxes.
>
>> 1) Skip this test on this platform, assuming the smoker is representative of
>> it.
>>
>> 2) Assume that locale is working well enough to do a a sanity check at the
>> beginning of the platform, and if it fails don't test locale
>>
>> 3) skip testing locale altogether.
>>
>> Ideas welcome
>>
>>
>


Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About