[perl #129886] [BUG] Regexp skips (?{ eval }) block because ofcached (*FAIL).

# New Ticket Created by  Tomash Brechko 
# Please include the string:  [perl #129886]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=129886 >


Hello,

Please consider the following code (finding all possible matches with
/(*FAIL)/ (or /(?!)/)):

  # Correct, 6 results
  $ perl -e '"01234" =~ /^(.+)(.+)((?:.)+)$(?{ print "$1 $2 $3\n"
})(*FAIL)/'
  012 3 4
  01 23 4
  01 2 34
  0 123 4
  0 12 34
  0 1 234

  # Bug, "0 1 234" is missing (added /z?/ to last capture)
  $ perl -e '"01234" =~ /^(.+)(.+)((?:.z?)+)$(?{ print "$1 $2 $3\n"
})(*FAIL)/'
  012 3 4
  01 23 4
  01 2 34
  0 123 4
  0 12 34

  # Bug, "012 3 4" and "01 2 34" are missing (added /z?/ to last capture,
made first non-greedy)
  $ perl -e '"01234" =~ /^(.+?)(.+)((?:.z?)+)$(?{ print "$1 $2 $3\n"
})(*FAIL)/'
  0 123 4
  0 12 34
  0 1 234
  01 23 4

Using "use re 'debug'" shows that the number of missed results corresponds
to the number of

  whilem: (cache) already tried at this position...
  failed...

messages in the trace.  So my wild guess is that the engine caches the
effect of /(*FAIL)/ and hence subsequently doesn't call /(?{ eval })/
blocks that come before it.

The comment in http://www.perlmonks.org/?node_id=1174069 reports that the
bug is present since at least Perl 5.8.9, up to 5.22.2.

Thanks!

Summary of my perl5 (revision 5 version 22 subversion 2) configuration:

  Platform:
    osname=linux, osvers=4.6.3-300.fc24.x86_64,
archname=x86_64-linux-thread-multi
    uname='linux buildvm-05.phx2.fedoraproject.org 4.6.3-300.fc24.x86_64 #1
smp fri jun 24 20:52:41 utc 2016 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=none -Dccflags=-O2 -g -pipe -Wall
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
-m64 -mtune=generic -Dldflags=-Wl,-z,relro
-Dccdlflags=-Wl,--enable-new-dtags -Wl,-z,relro  -Dlddlflags=-shared
-Wl,-z,relro  -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.22.2
-Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat,
Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
-Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64
/usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
-Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
-Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio
-Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less
-isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto
-Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto
-Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
-m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='  -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall
-Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
-m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='6.1.1 20160621 (Red Hat 6.1.1-3)',
gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags ='-Wl,-z,relro  -fstack-protector-strong
-L/usr/local/lib'
    libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib /usr/lib
/lib/../lib64 /usr/lib/../lib64 /lib
    libs=-lpthread -lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
    perllibs=-lpthread -lresolv -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.23.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.23'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef,
ccdlflags='-Wl,--enable-new-dtags -Wl,-z,relro '
    cccdlflags='-fPIC', lddlflags='-shared -Wl,-z,relro  -L/usr/local/lib
-fstack-protector-strong'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
                        USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS
                        USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
                        USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME
                        USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API
                        USE_SITECUSTOMIZE
  Locally applied patches:
    Fedora Patch1: Removes date check, Fedora/RHEL specific
    Fedora Patch3: support for libdir64
    Fedora Patch4: use libresolv instead of libbind
    Fedora Patch5: USE_MM_LD_RUN_PATH
    Fedora Patch6: Skip hostname tests, due to builders not being network
capable
    Fedora Patch7: Dont run one io test due to random builder failures
    Fedora Patch15: Define SONAME for libperl.so
    Fedora Patch16: Install libperl.so to -Dshrpdir value
    Fedora Patch22: Document Math::BigInt::CalcEmu requires Math::BigInt
(CPAN RT#85015)
    Fedora Patch26: Make *DBM_File desctructors thread-safe (RT#61912)
    Fedora Patch27: Make PadlistNAMES() lvalue again (CPAN RT#101063)
    Fedora Patch28: Make magic vtable writable as a work-around for Coro
(CPAN RT#101063)
    Fedora Patch29: Fix duplicating PerlIO::encoding when spawning threads
(RT#31923)
    Fedora Patch30: Do not let XSLoader load relative paths (CVE-2016-6185)
    Fedora Patch31: Avoid loading optional modules from default .
(CVE-2016-1238)
    Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on
Linux
    Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux
  Built under linux
  Compiled at Aug  3 2016 13:39:13
  @INC:
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .


-- 
  Tomash Brechko

• [perl #129886] [BUG] Regexp skips (?{ eval }) block because ofcached (*FAIL). by Tomash Brechko