[perl #57244] crash with recursive regexp

# New Ticket Created by  Zefram 
# Please include the string:  [perl #57244]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt3/Ticket/Display.html?id=57244 >


This is a bug report for perl from zefram@fysh.org,
generated with the help of perlbug 1.35 running under perl v5.8.8.


-----------------------------------------------------------------
[Please enter your report here]

I have some horribly complicated regexps using the (??{}) facility
to parse a recursive-descent grammar.  (Actually the real application
is parsing a subset of Perl expression syntax.)  I'm getting crashes,
mostly in the form of libc's malloc detecting corruption and aborting.
Using valgrind I've narrowed things down, and here's my minimal test case:

$ cat x1.pl
use re "eval";
my $dqstring_rx = qr/"(?:[^"\\]+|\\(?:e|x\{([0-9a-fA-F]+)\}))*"/;
my $sqstring_rx = qr/'(?:aaa|bbb)'/;
my $string_rx = qr/$dqstring_rx|$sqstring_rx/o;
my $array_rx = qr/\[(??{$datum_rx}),*\]/o;
my $datum_rx = qr/$string_rx|$array_rx/o;
"['aaa','bbb',]" =~ /\A$array_rx\z/;
$ valgrind perl x1.pl
==5853== Memcheck, a memory error detector.
==5853== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==5853== Using LibVEX rev 1658, a library for dynamic binary translation.
==5853== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==5853== Using valgrind-3.2.1-Debian, a dynamic binary instrumentation framework.
==5853== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==5853== For more details, rerun with: -v
==5853== 
==5853== Invalid write of size 4
==5853==    at 0x810CFEE: (within /usr/bin/perl)
==5853==    by 0x810FB47: (within /usr/bin/perl)
==5853==    by 0x81125E9: (within /usr/bin/perl)
==5853==    by 0x81138CE: (within /usr/bin/perl)
==5853==    by 0x81179AC: Perl_regexec_flags (in /usr/bin/perl)
==5853==    by 0x80C223A: Perl_pp_match (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853==  Address 0x4249184 is 0 bytes after a block of size 4 alloc'd
==5853==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==5853==    by 0x80AC80E: Perl_safesysmalloc (in /usr/bin/perl)
==5853==    by 0x80A4CE8: Perl_pregcomp (in /usr/bin/perl)
==5853==    by 0x80F539B: Perl_pp_regcomp (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853== 
==5853== Invalid write of size 4
==5853==    at 0x810CFFC: (within /usr/bin/perl)
==5853==    by 0x810FB47: (within /usr/bin/perl)
==5853==    by 0x81125E9: (within /usr/bin/perl)
==5853==    by 0x81138CE: (within /usr/bin/perl)
==5853==    by 0x81179AC: Perl_regexec_flags (in /usr/bin/perl)
==5853==    by 0x80C223A: Perl_pp_match (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853==  Address 0x42491BC is 0 bytes after a block of size 4 alloc'd
==5853==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==5853==    by 0x80AC80E: Perl_safesysmalloc (in /usr/bin/perl)
==5853==    by 0x80A4D3D: Perl_pregcomp (in /usr/bin/perl)
==5853==    by 0x80F539B: Perl_pp_regcomp (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853== 
==5853== Invalid write of size 4
==5853==    at 0x810CFEE: (within /usr/bin/perl)
==5853==    by 0x8112618: (within /usr/bin/perl)
==5853==    by 0x81138CE: (within /usr/bin/perl)
==5853==    by 0x81179AC: Perl_regexec_flags (in /usr/bin/perl)
==5853==    by 0x80C223A: Perl_pp_match (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853==  Address 0x4249184 is 0 bytes after a block of size 4 alloc'd
==5853==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==5853==    by 0x80AC80E: Perl_safesysmalloc (in /usr/bin/perl)
==5853==    by 0x80A4CE8: Perl_pregcomp (in /usr/bin/perl)
==5853==    by 0x80F539B: Perl_pp_regcomp (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853== 
==5853== Invalid write of size 4
==5853==    at 0x810CFFC: (within /usr/bin/perl)
==5853==    by 0x8112618: (within /usr/bin/perl)
==5853==    by 0x81138CE: (within /usr/bin/perl)
==5853==    by 0x81179AC: Perl_regexec_flags (in /usr/bin/perl)
==5853==    by 0x80C223A: Perl_pp_match (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853==  Address 0x42491BC is 0 bytes after a block of size 4 alloc'd
==5853==    at 0x401D38B: malloc (vg_replace_malloc.c:149)
==5853==    by 0x80AC80E: Perl_safesysmalloc (in /usr/bin/perl)
==5853==    by 0x80A4D3D: Perl_pregcomp (in /usr/bin/perl)
==5853==    by 0x80F539B: Perl_pp_regcomp (in /usr/bin/perl)
==5853==    by 0x80BC378: Perl_runops_standard (in /usr/bin/perl)
==5853==    by 0x8063BFC: perl_run (in /usr/bin/perl)
==5853==    by 0x805FFD0: main (in /usr/bin/perl)
==5853== 
==5853== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 19 from 1)
==5853== malloc/free: in use at exit: 278,020 bytes in 1,150 blocks.
==5853== malloc/free: 1,774 allocs, 624 frees, 326,280 bytes allocated.
==5853== For counts of detected errors, rerun with: -v
==5853== searching for pointers to 1,150 not-freed blocks.
==5853== checked 523,656 bytes.
==5853== 
==5853== LEAK SUMMARY:
==5853==    definitely lost: 274,488 bytes in 1,143 blocks.
==5853==      possibly lost: 2,836 bytes in 2 blocks.
==5853==    still reachable: 696 bytes in 5 blocks.
==5853==         suppressed: 0 bytes in 0 blocks.
==5853== Use --leak-check=full to see details of leaked memory.
$

Note that on this minimal test script perl actually exits successfully,
and the memory error is only noticed by valgrind.  Longer test programs
produce more output from valgrind, and some abort with output such as

*** glibc detected *** /usr/bin/perl: munmap_chunk(): invalid pointer: 0x095c5618 ***

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.8.8:

Configured by Debian Project at Fri Apr 25 20:33:47 UTC 2008.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.24.4, archname=i486-linux-gnu-thread-multi
    uname='linux ninsei 2.6.24.4 #1 smp preempt fri apr 18 15:36:09 pdt 2008 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian 4.1.1-21)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.8:
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    /usr/local/lib/perl/5.8.4
    /usr/local/share/perl/5.8.4
    .

---
Environment for perl v5.8.8:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=en_GB
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/pub/i686-pc-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/local/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

• [perl #57244] crash with recursive regexp by Zefram
• Re: [perl #57244] crash with recursive regexp by demerphq
• Re: [perl #57244] crash with recursive regexp by Eric Brine
• Re: [perl #57244] crash with recursive regexp by Zefram