develooper Front page | perl.perl5.porters | Postings from August 2019

[perl #127743] Storable loses information on large strings

From:
Tony Cook via RT
Date:
August 6, 2019 05:40
Subject:
[perl #127743] Storable loses information on large strings
Message ID:
rt-4.0.24-9652-1565070016-654.127743-15-0@perl.org
On Sun, 12 Feb 2017 15:44:19 -0800, tonyc wrote:
> On Sun, Feb 12, 2017 at 01:56:09PM -0800, Sawyer X via RT wrote:
> > On Sun, 17 Apr 2016 22:21:07 -0700, tonyc wrote:
> > > On Fri Mar 18 13:05:34 2016, jkeenan wrote:
> > > > My Perl version is v5.20.1 for x86_64-Linux. When I used Sortable
> > > > to
> > > > store
> > > > an array of large strings, the retrieved array would lost
> > > > information
> > > > of
> > > > strings longer than 1,699,160,188 bytes.
> > >
> > > Storable uses I32 and unsigned long internally for lengths,
> > > including
> > > both using it for the length of the scalar being stored and some
> > > intermediate lengths when calculating the new work buffer size for
> > > freeze()/thaw().
> > >
> > > The first of the attached patches simply rejects over-large
> > > scalars.
> > >
> > > The second handles scalars for which the size is too large for I32
> > > as
> > > new Storable tags that an older Storable will reject.
> >
> > Does anyone object merging this?
> 
> At the time I was put off merging this because of:
> 
> http://blogs.perl.org/users/rurban/2016/04/storable-security-problems-
> and-overlarge-data.html
> 
> (I don't recall him sending me the fixes.)
> 
> The second patch uses an op code the cperl changed skipped for some
> reason, and uses it only for 64-bit PV storage.[2]
> 
> cperl allocates a different op as a prefix op to mark the next op as
> taking a 64-bit size/count instead 32-bits for PVs, arrays, hashes.
> 
> So the issue for the second patch is file compatibility with cperl -
> are
> we worried about it?

I ended up merging much of the cperl Storable changes to blead, which was released as part of perl 5.28.0 and to CPAN as Storable 3.08.

Along with several other patches that fixed this issue and several others.

Closing.

Tony

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=127743



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About