Front page | perl.perl5.porters |
Postings from October 2009
On 10/29/2009 07:04 AM, Yuval Kogman wrote: > 2009/10/28 Raphael Manfredi<Raphael_Manfredi@pobox.com> > > > There is no intrinsic reason that would make JSON faster than Storable. > > Yes there is, JSON expresses a tree, Storable expresses a graph. > > A while ago (1995?) we wrote an implementation to serialize/de-serialize data structures. I think we wrote it because Storable and Data::Dumper were relatively unknown at the time, or maybe the original developer didn't know about it. At some point after that (1998?), I noted the existed of Storable and Data::Dumper, and that they performed better as our implementation started out life as pure Perl. I wrote an XS implementation of our algorithm that was faster than both Storable and Data::Dumper. I do not recall the difference between my implementation and Storable - only that the XS version of our implementation was at least 20X faster than the pure-Perl version of our imlpementation - but that's not really surprising. I am pretty sure the problem isn't something wrong with Storable - the problem is that Storable has to handle more complicated situations (a "graph" as Yuval says, although I'm not familiar enough with the term to say whether it is perfectly accurate), whereas a simpler implementation could choose to limit itself to hash / array / scalars only with no support for circular references and no support for objects or per-object serialization functions. This is effectively what JSON is. For data structures with only a few elements, or scalars of only a few characters (very common), the loss of a things like a hash / array / scalar header that includes a length prefix can end up being a net gain, since the data being processed can all fit in one cache line, and the code doing the parsing or generation can also be quite tight. Of course, if you have strings with 1 Mbytes in length, length prefixing becomes preferred to parsing each of the characters - but this is unusual. Anyways - just providing some substantiating data to show that Storable may not be faster than the best JSON encoder/decoder. Not to say Storable is bad. I've used it successfully many times in the past. It's always a problem that the more generic the solution, that the theoretically best algorithm becomes necessarily more complicated, and necessarily slower. If the requirements can be kept simple and limited, more optimization capabilities exist. One can still hit situations where the author of the more complicated algorithm is simply a better designer that the author of the simple algorithm - but this isn't really relevant to the discussion. :-) JSON is a very good light-weight encoding format for simple data structures with short or simple scalars. These types of data structures are very common. Cheers, mark -- Mark Mielke<mark@mielke.cc>Thread Previous | Thread Next