develooper Front page | perl.perl5.porters | Postings from December 2017

TONYC TPF Grant 10 report 1

Thread Next
From:
Tony Cook
Date:
December 8, 2017 00:27
Subject:
TONYC TPF Grant 10 report 1
Message ID:
20171208002723.wk2y23yicyb3l45e@mars.tony.develop-help.com
[Hours]         [Activity]
2017/11/20      Monday
 2.02           #127743 work out a practical fix, work on implementation
 2.68           #127743 finish implementation (with some side trips –
                found a new bug), testing
=====
 4.70

2017/11/21      Tuesday
 2.48           #127743 commit, work on fix for network retrieves of large
                objects, fix some -DDEBUGGING build issues (amongst a maze
                of massive macro expansions)
 2.75           #127743 build issues, portability work, testing
=====
 5.23

2017/11/22      Wednesday
 2.22           #127743 run entire test suite, find out -DDEBUGGING builds
                of Storable are painfully slow for large arrays, hashes
 1.65           #127743 fix a huge.t failure I introduced, more testing,
                fix flagged hash bug
=====
 3.87

2017/11/23      Thursday
 1.73           #127743 rebase on blead (complicated by ad2ec6b54c),
                testing, rebase fixes
 2.45           #127743 cross platform testing, fixes
=====
 4.18

Which I calculate is 17.98 hours.

One ticket was worked on.

I've been continuing my work on Storable in blead.

The first part involved backporting Storable changes from cperl to
blead, to support scalars with more than 2GB of data, and
arrays/hashes with more than 2**32 elements.

Since then, and continuing this week I've been working on fixing bugs
in that implementation.

One problem not handled in the cperl work is large object ids.

Storable allocates an id to every scalar it freezes, so if a later
reference to that same scalar is duplicated when thawing.

I've added support for a 64-bit variant of SX_OBJECT so simple
references to scalars with large object ids can be transferred
correctly.

Hooks, as with STORABLE_freeze() can also return references which are
frozen as their object ids, which were also being stored as 32-bit
values, possibly corrupting the resulting data structure on thaw.

Network order writes of 64-bit sizes were being written as a 32-bit,
since a union of two 32-bit values was used instead of an array or
structure.

Flagged hashes (used for restricted hashes) ignored the hash flags
byte when reading.

Luckily, the booking.com provided dromedary has 96GB RAM, so I can
test most of this.

I'm going to need mental Drano to clear Storable out of my head after
I finish all of this.

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About