develooper Front page | perl.perl5.porters | Postings from September 2019

[perl #134409] Perl intepreter exception on expression substitution

Thread Next
From:
Chris Crook
Date:
September 7, 2019 22:19
Subject:
[perl #134409] Perl intepreter exception on expression substitution
Message ID:
rt-4.0.24-16334-1567715852-564.134409-75-0@perl.org
# New Ticket Created by  Chris Crook 
# Please include the string:  [perl #134409]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=134409 >


This is a bug report for perl from ccrook@linz.govt.nz,
generated with the help of perlbug 1.40 running under perl 5.22.1.


-----------------------------------------------------------------
[Please describe your issue here]

Perl raises exception on search and replace:

panic: sv_pos_b2u: bad byte offset, blen=20, byte=27 at lib/GDSearchSQLite.pm line 81

    75  sub SubstituteSql
    76  {
    77     my ($sql,$lookup)=@_;
    78     my $updated_sql={};
    79     while( my($k,$v)=each(%$sql))
    80     {
    81         $v =~ s/\{(\w+)\}/$lookup->{$1}/eg;
    82         $updated_sql->{$k}=$v;
    83     }
    84     return $updated_sql;
    85  }

Note: this is old code buried fairly deeply in CGI web source.  I was able dump the inputs
to this specific routine, below, but in isolation this did not trigger the error.  So unfortunately
I am not able to provide a useful reproducible test case.

I was able to work around this by replacing line 81 with the following lines (which
are equivalent in the context in which this is called - key is defined in code always
alpha, all strings {\w+} in $k have corresponding value in $lookup).

    80         while( my($k1,$v1)=each(%$lookup) )
    81         {
    82             $v =~ s/\{$k1\}/$v1/eg;
    83         };

Below: test code with offending input doesn't reproduces issue:

use Data::Dumper;

$sql = {
          'where' => 'gwdi{nword}.word {wordop} \'{word}\'',
          'table' => 'JOIN word_index gwdi{nword} ON gwdi{nword}.geodetic_code=ati.geodetic_code'
        };
$lookup = {
          'wordop' => '=',
          'word' => 'SMITH',
          'nword' => 0
        };


sub SubstituteSql
{
   my ($sql,$lookup)=@_;
   use Data::Dumper;
   open(my $tf, ">>/tmp/badstuff");
   print $tf Dumper($sql);
   print $tf Dumper($lookup);
   close($tf);
   my $updated_sql={};
   while( my($k,$v)=each(%$sql))
   {
       $v =~ s/\{(\w+)\}/$lookup->{$1}/eg;
       $updated_sql->{$k}=$v;
   }
   return $updated_sql;
}

print Dumper(SubstituteSql($sql,$lookup)),"\n";

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl 5.22.1:

Configured by Debian Project at Mon Nov 19 18:29:35 UTC 2018.

Summary of my perl5 (revision 5 version 22 subversion 1) configuration:

  Platform:
    osname=linux, osvers=3.16.0, archname=x86_64-linux-gnu-thread-multi
    uname='linux localhost 3.16.0 #1 smp debian 3.16.0 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dcc=x86_64-linux-gnu-gcc -Dcpp=x86_64-linux-gnu-cpp -Dld=x86_64-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.22 -Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.22 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.22 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.22.1 -Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.22.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -
 Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.22.1'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='x86_64-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='5.4.0 20160609', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='x86_64-linux-gnu-gcc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.23.so, so=so, useshrplib=true, libperl=libperl.so.5.22
    gnulibc_version='2.23'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
    DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
    DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of libperl.a under /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
    DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
    DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
    DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules
    DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
    DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository
    DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.22.1-9ubuntu0.6 in patchlevel.h
    DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD
    DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text
    DEBPKG:debian/perl5db-x-terminal-emulator.patch - http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs - http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected
    DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated git repositories
    DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 Pass LD settings through to subdirectories
    DEBPKG:fixes/pod_man_reproducible_date - http://bugs.debian.org/759405 Support POD_MAN_DATE in Pod::Man for the left-hand footer
    DEBPKG:debian/locale-robustness - http://bugs.debian.org/782068 [perl #124310] Make t/run/locale.t survive missing locales masked by LC_ALL
    DEBPKG:fixes/podman-utc - http://bugs.debian.org/780259 Make the embedded date from Pod::Man reproducible
    DEBPKG:fixes/podman-utc-docs - http://bugs.debian.org/780259 Documentation and test suite updates for UTC fix
    DEBPKG:fixes/podman-empty-date - http://bugs.debian.org/780259 Support an empty POD_MAN_DATE environment variable
    DEBPKG:fixes/podman-pipe - http://bugs.debian.org/777405 Better errors for man pages from standard input
    DEBPKG:debian/pod2man-customized - Update porting/customized.dat for pod2man modifications
    DEBPKG:debian/makemaker-manext - http://bugs.debian.org/247370 Make EU::MakeMaker honour MANnEXT settings in generated manpage headers
    DEBPKG:debian/makemaker_customized - Update t/porting/customized.dat for files patched in Debian
    DEBPKG:debian/do-not-record-build-date - [6baa8db] http://bugs.debian.org/774422 [perl #125830] Allow overriding the compile time in "perl -V" output
    DEBPKG:fixes/podman-source-date-epoch - http://bugs.debian.org/801621 Make Pod::Man honor the SOURCE_DATE_EPOCH environment variable
    DEBPKG:fixes/podman-source-date-epoch-cleanups - http://bugs.debian.org/801621 Coding style and documentation for SOURCE_EPOCH_DATE
    DEBPKG:fixes/podman-source-date-epoch-testfix - http://bugs.debian.org/807086 Guard for building with SOURCE_DATE_EPOCH or POD_MAN_DATE set
    DEBPKG:debian/devel-ppport-reproducibility - http://bugs.debian.org/801523 Sort the list of XS code files when generating RealPPPort.xs
    DEBPKG:fixes/encode-unicode-bom - http://bugs.debian.org/798727 [rt.cpan.org #107043] Address https://rt.cpan.org/Public/Bug/Display.html?id=107043
    DEBPKG:debian/encode-unicode-bom-doc - http://bugs.debian.org/798727 Document Debian backport of Encode::Unicode fix
    DEBPKG:debian/kfreebsd-softupdates - http://bugs.debian.org/796798 Work around Debian Bug#796798
    DEBPKG:fixes/autodie-scope - http://bugs.debian.org/798096 Fix a scoping issue with "no autodie" and the "system" sub
    DEBPKG:debian/debugperl-compat-fix - [perl #127212] http://bugs.debian.org/810326 Disable PERL_TRACK_MEMPOOL for debugging builds
    DEBPKG:fixes/CVE-2015-8607_file_spec_taint_fix - http://bugs.debian.org/810719 [perl #126862] ensure File::Spec::canonpath() preserves taint
    DEBPKG:fixes/mkstemp-umask - http://bugs.debian.org/810924 [perl #127322] [e57270b] Fix umask for mkstemp(3) calls
    DEBPKG:fixes/crosscompile-no-targethost - [perl #127234] Fix the Configure escape with usecrosscompile but no targethost
    DEBPKG:fixes/podlators-no-encode - [rt.cpan.org #111156] Degrade gracefully if utf8 is requested but Encode is not available
    DEBPKG:debian/cross-time-hires - [rt.cpan.org #111391] Add an environment variable to skip running configuration probes
    DEBPKG:fixes/encode-unicode-pod - Unicode.pm: Fix POD error
    DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in Memoize
    DEBPKG:fixes/ok-pod - Added encoding for pod.
    DEBPKG:fixes/CVE-2016-2381_duplicate_env - remove duplicate environment variables from environ
    DEBPKG:fixes/CVE-2017-12837.patch - [PATCH] regcomp [perl #131582]
    DEBPKG:fixes/CVE-2017-12883.patch - [PATCH] PATCH: [perl #131598]
    DEBPKG:fixes/CVE-2016-6185.patch - [PATCH] =?utf8?q?Don=E2=80=99t=20let=20XSLoader=20load=20relative?= =?utf8?q?=20paths?=
    DEBPKG:fixes/CVE-2017-6512-pre.patch - [PATCH] Correct the order of tests of chmod(). (#294)
    DEBPKG:fixes/CVE-2017-6512.patch - http://bugs.debian.org/863870 [rt.cpan.org #121951] Prevent directory chmod race attack.
    DEBPKG:fixes/CVE-2018-6797.patch - (perl #132227) restart a node if we change to uni rules within the node and encounter a sharp S
    DEBPKG:fixes/CVE-2018-6798-1.patch - [perl #132063]: Heap buffer overflow
    DEBPKG:fixes/CVE-2018-6798-2.patch - v5.24.3: fix TRIE_READ_CHAR and DECL_TRIE_TYPE to account for non-utf8 target
    DEBPKG:fixes/CVE-2018-6798-3.patch - (perl #132063) we should no longer warn for this code
    DEBPKG:fixes/CVE-2018-6913.patch - (perl #131844) fix various space calculation issues in pp_pack.c
    DEBPKG:fixes/CVE-2018-12015.patch - [PATCH] [PATCH] Remove existing files before overwriting them
    DEBPKG:fixes/CVE-2018-18311.patch - [PATCH] Perl_my_setenv(); handle integer wrap
    DEBPKG:fixes/CVE-2018-18312.patch - [PATCH 242/242] PATCH: [perl #133423] for 5.26 maint
    DEBPKG:fixes/CVE-2018-18313.patch - [PATCH] regcomp.c: Convert some strchr to memchr
    DEBPKG:fixes/CVE-2018-18314.patch - [PATCH] fix #131649 - extended charclass can trigger assert

---
@INC for perl 5.22.1:
    /etc/perl
    /usr/local/lib/x86_64-linux-gnu/perl/5.22.1
    /usr/local/share/perl/5.22.1
    /usr/lib/x86_64-linux-gnu/perl5/5.22
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.22
    /usr/share/perl/5.22
    /usr/local/lib/site_perl
    /usr/lib/x86_64-linux-gnu/perl-base
    .

---
Environment for perl 5.22.1:
    HOME=/home/ccrook
    LANG=en_NZ.UTF-8
    LANGUAGE=en_NZ:en
    LD_LIBRARY_PATH=/home/ccrook/apps/lib:
    LOGDIR (unset)
    PATH=:/usr/share/linz/snap:/usr/lib/ccache:/home/ccrook/bin:/home/ccrook/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/ccrook/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

________________________________

This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info@linz.govt.nz) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About