develooper Front page | perl.perl5.porters | Postings from September 2003

Strang Storable behavior

Lev Serebryakov
September 28, 2003 08:07
Strang Storable behavior
Message ID:
Hello, perl5-porters! How are you?

  I've asked this question in news:comp.lang.perl.misc and on my local
  newsgroups, but get no answer :(

  I write script, whcih process some data step-by-step. Each step
takes some time (typical time is from 1 to 10 minutes, depends on
input data) and each step could fail, but such failures could be fixed
by hints from command line.

  I store full state of processing after each step with `Storable'
module. Full state is describing by array of references: some
references to array and some references to hashes. After first step
only first array is not empty (it contains parsed input data), after
second step some data is in some hashes, etc. But always I pass to
Storable::store() same array of references.

  I see very strange timings of Storable::store() function calls. After
first 3 steps it is small (about 10 seconds for very big amount of
input data), after 4th step it is very big (about 240 seconds for
twice smaller data!), and after 5th step it is small again (12-14
seconds in any case).

  But each step only ADDS data to saved state. So, every next dump is
GREATER in size that previous, and already created structures doesn't
become simpler on next steps. And results of 4th step PRESENTS in
results of step 5. But results of 4th step are saved 20 times longer
than results of step 5!

  What does it mean and how this could be fixed? So long savings make
`dump-restore' idea useless -- Storable stores results of 4th step
longer, than other processing works :(

  I use Activestate's perl 5.8.0 on Win2K Pro.

               Lev Serebryakov Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About