develooper Front page | perl.perl5.porters | Postings from August 2012

Re: [PATCH] Module::CoreList delta support

Thread Previous | Thread Next
From:
David Golden
Date:
August 5, 2012 10:31
Subject:
Re: [PATCH] Module::CoreList delta support
Message ID:
CAOeq1c-WH6J+FMPBH-48wk5R3FbThbdyhZ3KVL=_j4yrhqkX6w@mail.gmail.com
On Sun, Aug 5, 2012 at 3:20 AM, Aristotle Pagaltzis <pagaltzis@gmx.de> wrote:
> That is a weakness in the scheme I outlined. Lines need to be fixed
> length for the unpack sleight of hand to work, which means adding a perl

(Without actually reading your code) my gut reaction is "don't do
unpack slight of hand" then.  Store lines in sorted module name order
like this:

    "$name $json\n" # where $json has no newlines in it

Then use Search::Dict to bisect to the right line, split the line into
two fields, decode the JSON part into a hash and dig into it as
needed.  The bisection means it will be faster than reading every line
and loading the whole thing into memory anyway.  Only in the rarest of
cases do you need to load the whole file -- most uses are still
"corelist Foo", which benefits hugely from bisection.

And if the JSON data for each module is in some delta format and only
changes when a module is updated, a line only gets touched when it
actually changes in a release, so the diff is annoying (a very long
line) but not horrible (every line changing).

Why JSON vs some other ascii delimited format?  Because it allows
structured data instead of just fields.  (Plus anyone with JSON:XS
installed gets a speed boost for free.)

-- David

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About