[perl #121562] index segfaults on 2G strings in 64bit perl

On Wed Apr 09 17:02:58 2014, ilmari wrote:
> "Tony Cook via RT" <perlbug-followup@perl.org> writes:
> 
> > On Thu Apr 03 15:44:40 2014, tonyc wrote:
> >> On Wed Apr 02 02:18:05 2014, mike.chamberlain@pirum.com wrote:
> >> > We're attemping to parse a large file over 2G in size, and it's
> >> > segfaulting. We are scanning
> >> > the file to identify key points within it. Ultimately the simplest
> >> > implementation of the bug is:
> >> >
> >> > chambm@wren9a ~/src/SBLREX> perl -e 'my $x = " " x ((2**31 )-1);
> >> > $x
> >> > .="\n"; my $end = index($x, "\n", 0); print "END: $end\n"'
> >> > Segmentation fault (core dumped)
> >> > chambm@wren9a ~/src/SBLREX> perl -e 'my $x = " " x ((2**31 )-2);
> >> > $x
> >> > .="\n"; my $end = index($x, "\n", 0); print "END: $end\n"'
> >> > END: 2147483646
> >>
> >> It looks like pp_index suffers from the I32 bug, assuming
> >> fbm_instr()
> >> and rninstr() are safe it should be easy to fix.
> >
> > Fix attached, for 5.21, though perhaps it should be in 5.20.
> 
> It needs to be switched to use sv_pos_(u2b|b2u)_flags as well, to
> handle
> UTF-8 strings properly:
> 
> pp.c: In function ‘Perl_pp_index’:
> pp.c:3284:6: warning: passing argument 2 of ‘Perl_sv_pos_u2b’ from
> incompatible pointer type [enabled by default]
>       sv_pos_u2b(big, &offset, 0);
>       ^
> In file included from perl.h:5012:0,
>                  from pp.c:28:
> proto.h:4266:20: note: expected ‘I32 * const’ but argument is of type
> ‘ssize_t *’
>  PERL_CALLCONV void Perl_sv_pos_u2b(pTHX_ SV *const sv, I32 *const
> offsetp, I32 *const lenp)
>                     ^
> pp.c:3301:6: warning: passing argument 2 of ‘Perl_sv_pos_b2u’ from
> incompatible pointer type [enabled by default]
>       sv_pos_b2u(big, &retval);
>       ^
> In file included from perl.h:5012:0,
>                  from pp.c:28:
> proto.h:4256:20: note: expected ‘I32 * const’ but argument is of type
> ‘ssize_t *’
>  PERL_CALLCONV void Perl_sv_pos_b2u(pTHX_ SV *const sv, I32 *const
> offsetp)

Thanks, I looked for warnings when I built it, but must have missed them.

> And adding utf8::upgrade($work) to t/bigmem/index.t, gives:
> 
> ilmari@nurket:~/src/perl/t$ PERL_TEST_MEMORY=4 ../perl -I../lib
> bigmem/index.t
> 1..2
> panic: sv_pos_b2u: bad byte offset, blen=2147483650,
> byte=18446744071562067968 at bigmem/index.t line 23.
> # Looks like you planned 2 tests but ran 0.

I should have tested the unicode path too.

Tony


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=121562

• [perl #121562] index segfaults on 2G strings in 64bit perl by Tony Cook via RT