develooper Front page | perl.perl5.porters | Postings from July 2016

[perl #128734] tr/// with named Unicode chars broken since 5.23.3

Thread Previous
From:
slaven @ rezic . de
Date:
July 26, 2016 05:28
Subject:
[perl #128734] tr/// with named Unicode chars broken since 5.23.3
Message ID:
rt-4.0.18-29859-1469510889-1131.128734-75-0@perl.org
# New Ticket Created by  slaven@rezic.de 
# Please include the string:  [perl #128734]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=128734 >



This is a bug report for perl from slaven@rezic.de,
generated with the help of perlbug 1.40 running under perl 5.23.3.


-----------------------------------------------------------------
The following oneliner works for all perls until 5.23.2, and is broken
with 5.23.3 and later:

$ perl5.22.2 -e 'use Test::More "no_plan"; my $string = "\x{00e0}"; $string =~ tr/\N{U+00e0}/A/; is $string, "A";'
ok 1
1..1

$ perl5.24.0 -e 'use Test::More "no_plan"; my $string = "\x{00e0}"; $string =~ tr/\N{U+00e0}/A/; is $string, "A";'
not ok 1
#   Failed test at -e line 1.
#          got: '�'
#     expected: 'A'
1..1
# Looks like you failed 1 test of 1.

Running bisect delivers the following result:

HEAD is now at f424037 Refactor tr/// parsing to work on EBCDIC, fix other bug
bad - non-zero exit from ./perl -Ilib /tmp/a.pl
f424037916111e114a9c424abacc210aac685dff is the first bad commit
commit f424037916111e114a9c424abacc210aac685dff
Author: Karl Williamson <khw@cpan.org>
Date:   Fri Sep 4 11:32:26 2015 -0600

    Refactor tr/// parsing to work on EBCDIC, fix other bug
    
    This expands the concept introduced for regular expressions in v5.22 of
    a portable range, to the transliteration operators.  A portable range
    has at least one endpoint expressed as \N{} that indicates that the
    Unicode definition is desired, or has the endpoints expressed as both
    uppercase ASCII alphabetic letters or both lowercase ASCII alphabetics.
    
    The refactor fixes several EBCDIC problems, and it fixes the problem in
    all platforms wherein the first endpoint of a range was not checked to
    be <= the final endpoint in UTF-8 strings.
    
    There remains a bug in which if any transliterated code point is larger
    than IV_MAX, perl loops.

:040000 040000 b761e0d3d62cc2854153dd0a7560ea8094b5a3ee 1bee1201b49c33914abafaf8b0be4eba4f0e69d4 M      pod
:040000 040000 16dc0b2bc9fc101b3bfaea8455b5c87b86408f71 baf1219eac437e350f84f5b9ee61ccd745cb0b0a M      t
:100644 100644 70318a7609eefba7d4ad99cfaefeeb6115d8f323 27e6650b515a9ab1545e866c21dca2589aeab5dc M      toke.c
bisect run success
That took 566 seconds.


Regards,
    Slaven

-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
Site configuration information for perl 5.23.3:

Configured by eserte at Tue Sep 29 19:49:56 CEST 2015.

Summary of my perl5 (revision 5 version 23 subversion 3) configuration:
   
  Platform:
    osname=freebsd, osvers=9.2-release, archname=amd64-freebsd
    uname='freebsd cvrsnica.herceg.de 9.2-release freebsd 9.2-release #0 r255898: thu sep 26 22:50:31 utc 2013 root@bake.isc.freebsd.org:usrobjusrsrcsysgeneric amd64 '
    config_args='-ds -e -Dprefix=/usr/perl5.23.3sp -Dusedevel -Dusemallocwrap=no -Duseshrplib -Dcf_email=srezic@cpan.org'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_FORTIFY_SOURCE=2',
    optimize='-O',
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.2.1 20070831 patched [FreeBSD]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-pthread -Wl,-E  -fstack-protector -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/include/gcc/4.2 /usr/lib
    libs=-lpthread -lgdbm -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lm -lcrypt -lutil -lc
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -Wl,-R/usr/perl5.23.3sp/lib/5.23.3/amd64-freebsd/CORE'
    cccdlflags='-DPIC -fPIC', lddlflags='-shared  -L/usr/local/lib -fstack-protector'


---
@INC for perl 5.23.3:
    /usr/perl5.23.3sp/lib/site_perl/5.23.3/amd64-freebsd
    /usr/perl5.23.3sp/lib/site_perl/5.23.3
    /usr/perl5.23.3sp/lib/5.23.3/amd64-freebsd
    /usr/perl5.23.3sp/lib/5.23.3
    .

---
Environment for perl 5.23.3:
    HOME=/home/e/eserte
    LANG (unset)
    LANGUAGE (unset)
    LC_ALL=de_DE.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/e/eserte/bin/freebsd9.1:/home/e/eserte/bin/sh:/home/e/eserte/bin:/home/e/eserte/bin/pistachio-perl/bin:/usr/games:/home/e/eserte/devel
    PERLDOC=-MPod::Perldoc::ToTextOverstrike
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/zsh


Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About