[perl #121111] is_locale_utf8 tests fail on VMS

# New Ticket Created by  Craig A. Berry 
# Please include the string:  [perl #121111]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=121111 >


A number of tests involving locales now fail on VMS since 31f05a37c4e9c37a72.  See

<http://www.nntp.perl.org/group/perl.daily-build.reports/2014/01/msg157574.html>

The failures boil down to this check for whether a particular locale chosen apparently at random from the list of available locales is UTF-8:

$ perl -"MPOSIX" -e "require 'test.pl'; require 'loc_tools.pl'; print is_locale_utf8('IW_IL_UTF-8');"
not ok 1 - Verify IW_IL_UTF-8 with UTF-8 in name is a UTF-8 locale
# Failed test 1 - Verify IW_IL_UTF-8 with UTF-8 in name is a UTF-8 locale at loc_tools.pl line 219
0

which in turn boils down to a check for whether the case folded form of the German eszett or sharp s is equal to ASCII 'ss'.  It isn't, as the following one-liners demonstrate:

$ perl -e "print CORE::fc(chr utf8::unicode_to_native(0xdf));"
ß

$ perl -"MPOSIX" -e "setlocale(&POSIX::LC_CTYPE(), 'IW_IL_UTF-8'); print CORE::fc(chr utf8::unicode_to_native(0xdf));"
ß

$ perl -e "print CORE::fc(chr utf8::unicode_to_native(0xdf)) ne 'ss';"
1

$ perl -"MPOSIX" -e "setlocale(&POSIX::LC_CTYPE(), 'IW_IL_UTF-8'); print CORE::fc(chr utf8::unicode_to_native(0xdf)) ne 'ss';"
1

This doesn't make much sense to me and is probably just my ignorance but I'll ask anyway.  

Why should a locale that I believe is Hebrew have any representation for a German-only letter?  

Since the upper case version of eszett was rejected for inclusion in the Unicode standard before Unicode 5.1, are we really just testing for Unicode 5.1 support in the locale rather than UTF-8-ness?


$ perl -V
Summary of my perl5 (revision 5 version 19 subversion 9 patch v5.19.8-211-ge12272f) configuration:
  Snapshot of: e12272fc5eb86388de50059ac74317661bf13377
  Platform:
    osname=VMS, osvers=V8.4, archname=VMS_IA64-thread-multi
    uname='VMS alma V8.4 HP rx2600  (1.50GHz/6.0MB)'
    config_args='-"des" -"Dusedevel" -"Duseithreads"'
    hint=none, useposix=false, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=undef, bincompat5005=undef
  Compiler:
    cc='CC/DECC', ccflags ='/Include=[]/Standard=Relaxed_ANSI/Prefix=All/Obj=.obj /NOANSI_ALIAS/float=ieee/ieee=denorm/NAMES=(SHORTENED)/Define=_USE_STD_STAT=1'
,
    optimize='/NoList',
    cppflags='undef'
    ccversion='70390020', gccversion='', gccosandvers='undef'
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='Link/nodebug', ldflags ='/NoTrace/NoMap'
    libpth=/sys$share /sys$library
    libs=
    perllibs=
    libc=(DECCRTL), so=exe, useshrplib=true, libperl=undef
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_vms.xs, dlext=exe, d_dlsymun=undef, ccdlflags=''
    cccdlflags='', lddlflags='/Share'


Characteristics of this PERLSHR image:
  Compile-time options: HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY
                        PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_EXTERNAL_GLOB PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV USE_IEEE
                        USE_ITHREADS USE_LARGE_FILES USE_LOCALE
                        USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
                        USE_REENTRANT_API VMS_DO_SOCKETS
                        VMS_SHORTEN_LONG_SYMBOLS
  Locally applied patches:
	SMOKEe12272fc5eb86388de50059ac74317661bf13377
  Built under VMS
  Compiled at Jan 28 2014 16:06:14
  %ENV:
    PERLSHR="perl_root:[000000]perlshr.exe"
    PERL_ROOT="MDA0:[CRAIG.smoke.blead.]"
  @INC:
    /perl_root/lib/site_perl/VMS_IA64-thread-multi
    /perl_root/lib/site_perl
    /perl_root/lib/VMS_IA64-thread-multi/5_19_9
    /perl_root/lib
    .
________________________________________
Craig A. Berry
mailto:craigberry@mac.com

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser

• Re: [perl #121111] is_locale_utf8 tests fail on VMS by Craig A. Berry
• Re: [perl #121111] is_locale_utf8 tests fail on VMS by Craig A. Berry
• [perl #121111] is_locale_utf8 tests fail on VMS by Karl Williamson via RT
• [perl #121111] is_locale_utf8 tests fail on VMS by Craig A . Berry