develooper Front page | perl.perl5.porters | Postings from August 2013

Re: [perl #119445] performance bug: perl Thread::Queue is 20x slowerthan Unix pipe

Thread Previous | Thread Next
Dave Mitchell
August 26, 2013 15:10
Re: [perl #119445] performance bug: perl Thread::Queue is 20x slowerthan Unix pipe
Message ID:
On Sun, Aug 25, 2013 at 05:37:39PM -0700, James E Keenan via RT wrote:
> On Fri Aug 23 17:28:00 2013, wrote:
> > Why is Thread::Queue *so* slow?
> > 
> > I understand it has to do locking and be careful about data
> > structures, but it seems like it is about 20x slower than opening up a
> > Unix pipe, printing to that, reading it back and parsing the result.

Because it is nothing like a UNIX pipe.

A  UNIX pipe takes a stream of bytes, and read and writes chunks of them 
into a shared buffer.

A T::Q buffer takes a stream of perl "things", which might be objects or
other such complex structures, and ensures they they are accessible by
both the originating thread and any potential consumer thread. Migrating a
perl "thing" across a thread boundary is considerably more complex than
copying a byte across.

> > To speculate, I'm thinking the cost is in making all IPC data shared.
> > It would be great if one could have data that is sent over
> > Thread::Queue that is copied, not shared.

But T::Q is build upon a shared array, and is designed to handled shared

I think the performance you are seeing is the performance I would expect,
and that this is not a bug.

In England there is a special word which means the last sunshine
of the summer. That word is "spring".

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About