[perl #21574] Regexp::Common corrupts Filter::Simple

# New Ticket Created by  tim@teachmeperl.com 
# Please include the string:  [perl #21574]
# in the subject line of all future correspondence about this issue. 
# <URL: http://rt.perl.org/rt2/Ticket/Display.html?id=21574 >


This is a bug report for perl from tim@teachmeperl.com,
generated with the help of perlbug 1.33 running under perl v5.8.0.


-----------------------------------------------------------------
[Please enter your report here]

When Regexp::Common is used along with Filter::Simple,
even with a "null-filter" (that does no transformation whatsoever),
the source code gets corrupted in certain cases.  I've included below
a module called NullFilter.pm and a demo program that illustrates the bug.

I've tested and confirmed the bug under 5.8.0, with Regexp::Common 2.111
(latest) and Filter::Simple 0.78 (latest) on two different Linux machines
and one Solaris machine.  It's real!

#! /usr/bin/perl -w
package NullFilter;

# Tim Maher, tim@teachmeperl.com
# Fri Mar 14 06:19:31 PST 2003

# This module demonstrates a bug in the combination of
# Regexp::Common and Filter::Simple


use Filter::Simple;     # Tested under 0.78, with
use Regexp::Common;     # 2.110 and 2.111

FILTER_ONLY code => \&null_filter, all=>sub { print };

sub null_filter{        }       # shouldn't hurt anything!  8-}

# Comment-out Regexp::Common line above (whose resources are not
# even explicitly used), and no data loss!

# Leave it in, and 3rd line below gets translated into 4th below (it
# takes the extra space before the \( to show the bug). I know,
# because I modified Filter::Simple to see the data before & after
# replacement of control-chars that are used as placeholders for
# string data.

# IN:  @t_opt = ( = =, = = ) ;
# OUT: @t_opt = ( '', '-a' ) ;
# IN:  @t_opt =  ( = =, = = ) ;
# OUT: @t_opt =, '-a' ) ;

# Here's the input data line that causes the trouble:

# Adding <SP> before ( below causes null-filter to lose data
# @t_opt = ( '', '-a' ) ;

1;


This's test program illustrates the bug, with <SP> added where indicated

#! /usr/bin/perl -w
use NullFilter;

# Two <SP>s before \( below causes null-filter to corrupt data,
# (only) if Regexp::Common used. One <SP> is fine.

@t_opt = ( '', '-a' ) ;

print "OK\n";


This patch for Filter::Simple 0.78 helps capture the before/after data
97d96
<
116c115,126
<                       s/$extractor/${$pieces[unpack('N',$1)]}/g;
---
> if (defined $ENV{CELADOR} and $_ ne "") {
>       open OFILE, "> /tmp/celador$$.0"; print OFILE $_;       # -tfm
>       $result = s/$extractor/${$pieces[unpack('N',$1)]}/g; # - tfm
>       close OFILE;
>
>       open OFILE, "> /tmp/celador$$.1"; print OFILE $_;       # -tfm
>       close OFILE;
>       $result ; # Assert s/// return value, so becomes implicit return value -tfm
> }
> else {
>                        s/$extractor/${$pieces[unpack('N',$1)]}/g;
> }








[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
This perlbug was built using Perl v5.6.1 - Thu Sep 20 02:27:59 GMT 2001
It is being executed now by  Perl v5.8.0 - Sun Jul 21 01:19:20 PDT 2002.

Site configuration information for perl v5.8.0:

Configured by root at Sun Jul 21 01:19:20 PDT 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.16-4gb, archname=i686-linux
    uname='linux guru 2.4.16-4gb #1 wed dec 19 09:08:41 gmt 2001 i686 unknown '
    config_args='-de -Dprefix=/usr/local/perl5.8.0'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O3',
    cppflags='-fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.95.3 20010315 (SuSE)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lc -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    

---
@INC for perl v5.8.0:
    /usr/local/perl5.8.0/lib/5.8.0/i686-linux
    /usr/local/perl5.8.0/lib/5.8.0
    /usr/local/perl5.8.0/lib
    /usr/local/perl5.8.0/lib/site_perl/5.8.0/i686-linux
    /usr/local/perl5.8.0/lib/site_perl/5.8.0
    /usr/local/perl5.8.0/lib/site_perl/5.8.0/i686-linux
    /usr/local/perl5.8.0/lib/site_perl/5.8.0
    /usr/local/perl5.8.0/lib/site_perl
    /usr/lib/perl5/site_perl/5.6.1
    /root2/usr/local/LIB/perl5/5.6.1
    /root2/local/perl/lib/site_perl/5.6.1/i686-linux
    /root2/local/perl/lib/site_perl/5.6.1
    /usr/lib/perl5/site_perl/5.6.0
    /root2/usr/local/LIB/perl5/5.6.0/i686-linux
    /root2/usr/local/LIB/perl5/5.6.0
    /root2/local/perl/lib/site_perl/5.6.0/i686-linux
    /root2/local/perl/lib/site_perl/5.6.0
    /usr/local/perl5.8.0/lib/5.8.0/i686-linux
    /usr/local/perl5.8.0/lib/5.8.0
    /usr/local/perl5.8.0/lib/site_perl/5.8.0/i686-linux
    /usr/local/perl5.8.0/lib/site_perl/5.8.0
    /usr/local/perl5.8.0/lib/site_perl
    .

---
Environment for perl v5.8.0:
    HOME=/home/contix
    LANG=en_US
    LANGUAGE (unset)
    LC_COLLATE=POSIX
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/contix/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/lib/java/bin:/usr/games/bin:/usr/games:/opt/gnome/bin:/opt/kde2/bin:.:/usr/local/perl5.8.0/bin:/opt/kde2/bin:/usr/X11R6/bin:/local/bin:/local/linbin:/local/timbin:/local/astkit/bin:/home/contix/bin:/courses/perl/teacher/inst_bin:/class/teacher/inst_bin:/local/timbin:/linbin:/linbin:/local/office52:/local/office52/program:/opt/kde/bin:.
    PERL5LIB=/usr/local/perl5.8.0/lib:/usr/local/perl5.8.0/lib/site_perl/5.8.0:/usr/local/perl5.8.0/lib/site_perl:/usr/lib/perl5/site_perl/5.6.1:/root2/usr/local/LIB/perl5/5.6.1:/root2/local/perl/lib/site_perl/5.6.1/i686-linux:/root2/local/perl/lib/site_perl/5.6.1:/usr/lib/perl5/site_perl/5.6.0:/root2/usr/local/LIB/perl5/5.6.0:/root2/local/perl/lib/site_perl/5.6.0/i686-linux:/root2/local/perl/lib/site_perl/5.6.0
    PERL_BADLANG (unset)
    SHELL=/bin/ksh

• [perl #21574] Regexp::Common corrupts Filter::Simple by perlbug-followup