Front page | perl.perl5.porters |
Postings from June 2019
Re: [perl #134187] Turkic casing not working in freebsd
Thread Next
From:
E. Choroba
Date:
June 18, 2019 09:38
Subject:
Re: [perl #134187] Turkic casing not working in freebsd
Message ID:
alpine.LSU.2.20.1906181131320.4799@lenonovo.gooddata.com
On Mon, 17 Jun 2019, Tony Cook via RT wrote:
> On Tue, 11 Jun 2019 06:17:40 -0700, jkeenan wrote:
>> Re-arranging to try to compensate for top-posting.
>>
>>>>>>>> On Sun, 09 Jun 2019 17:00:56 GMT, choroba@cpan.org wrote:
>>>>>>>> This is a bug report for perl from choroba@cpan.org,
>>>>>>>> generated with the help of perlbug 1.39 running under perl
>>>>>>>> 5.18.2.
>>>>>>>>
>>>>>>>>
>>>>>>>> -----------------------------------------------------------------
>>>>>>>> [Please describe your issue here]
>>>>>>>>
>>>>>>>> The turkic casing doesn't work on freebsd. Running the
>>>>>>>> following
>>>>>>>> in
>>>>>>>> 5.30.0 works in Linux, but not FreeBSD:
>>>>>>>>
>>>>>>>>
>>>>>>>> #!/usr/bin/perl
>>>>>>>> use warnings;
>>>>>>>> use strict;
>>>>>>>>
>>>>>>>> use Test::More tests => 1;
>>>>>>>>
>>>>>>>> use locale;
>>>>>>>> use POSIX 'locale_h';
>>>>>>>> setlocale(LC_ALL, 'tr_TR.UTF-8');
>>>>>>>>
>>>>>>>> is lc 'I', "\N{LATIN SMALL LETTER DOTLESS I}";
>>>>>>>> __END__
>>>>>>>>
>>>>>>>> See also https://rt.cpan.org/Public/Bug/Display.html?id=129766
>>>>>>>>
>>>>>>>> Ch.
>>>>>>>
>>>>>>> Could you attach the 'perl -V' output for perl on the FreeBSD
>>>>>>> box
>>>>>>> where you noticed this problem?
>>>>>>>
>>>>>>> Thank you very much.
>>>>>>
>>>>>> I was the original reporter.
>>>>>> Attached 3x perl -V, from different freebsd versions.
>>>>>>
>>>>>> Regards,
>>>>>> Slaven
>>>>>>
>>>>>
>>>>> Thanks. I can confirm the problem on FreeBSD-11.2. My system has
>>>>> no
>>>>> problem printing LATIN SMALL LETTER DOTLESS I -- it's octal 304
>>>>> 261.
>>>>> So,
>>>>> does that mean that on this OS, the appropriate uc-to-lc mapping
>>>>> for
>>>>> this
>>>>> locale is incorrect?
>>>>>
>>>>> And, if so, that's a FreeBSD problem rather than a Perl problem --
>>>>> correct?
>>>>>
>>>>> Thank you very much.
>>>>> --
>>>>> James E Keenan (jkeenan@cpan.org)
>>>>>
>>>>> ---
>>>>> via perlbug: queue: perl5 status: open
>>>>> https://rt.perl.org/Ticket/Display.html?id=134187
>>>>
>>
>>> On Mon, 10 Jun 2019, Karl Williamson via RT wrote:
>>>> Does foldgrind_t.t say skipped. If so the os doesn’t provide a
>>>> working
>>>> Turkish locale
>>>>
>>
>> Yes, when I build blead on FreeBSD-11, I get:
>>
>> #####
>> re/fold_grind_T.t ..... skipped: Couldn't find a UTF-8 turkic locale
>> #####
>>
>> So the Perl 5 test suite takes appropriate evasive action in the
>> absence of a valid tr_TR.UTF-8 locale.
>>
>> On Tue, 11 Jun 2019 11:14:39 GMT, choroba@matfyz.cz wrote:
>>> The test that was originally failing should have been skipped if
>>> setting the
>>> locale failed. It's a bit convoluted, see
>>>
>>> https://metacpan.org/source/CHOROBA/Syntax-Construct-1.011/t/02-
>>> constructs.t#L39
>>>
>>> Ch.
>>>
>>
>> Karl,
>>
>> I think the upshot of this is that some of the guidance we have
>> provided for locale handling in perl-5.30 is misleading.
>>
>> In 'perldoc perllocale' in both 5.30 and in blead, we say:
>>
>> #####
>> Turkic languages and one for everything else. Starting in Perl v5.30,
>> Perl seamlessly handles both types; previously only the non-Turkic one
>> was supported.
>> #####
>>
>> In https://metacpan.org/source/CHOROBA/Syntax-Construct-1.011/t/02-
>> constructs.t, Choroba has updated test code to accommodate perl-5.30.
>> But this gets the test failure reported in
>> https://rt.cpan.org/Ticket/Display.html?id=129766.
>>
>> My inference is that in 'perllocale' the word "seamlessly" is
>> misleading. The Syntax::Construct test failure is evidence that
>> seamless this is not.
>>
>> I suspect we need some revision in the documentation to provide better
>> guidance for use of Turkic locales. Once we have that, Choroba could
>> revise his CPAN test accordingly.
>>
>> Thank you very much.
>
> I think the problem is how you define what is a "turkic locale".
>
> Choroba appears to assume that just because you can setlocale(LC_ALL, "tr_TR.UTF-8") you have a Turkic locale, but at least for NetBSD and Alpine Linux, this isn't true, both accept pretty much any locale name and return one of two locales.
>
> Perl on the other hand doesn't care about the name, only that the locale behaves like a Turkic locale, and the FreeBSD "tr_TR.UTF-8" doesn't.
OK, I understand. Now, if there were a script that someone wrote that depended
on turkic locale casing, how could they verify that it works? Something like
use POSIX 'locale_h';
use locale;
my $turkic_locale = 'tr_TR.UTF-8';
$turkic_locale eq setlocale(LC_ALL, $turkic_locale)
or die "Turkic locale not supported\n";
Does it work at all? And even if it does, is there a better way?
Thanks
Ch.
Thread Next