develooper Front page | perl.perl5.porters | Postings from August 2013

Re: [perl #119445] performance bug: perl Thread::Queue is 20x slowerthan Unix pipe

Thread Previous | Thread Next
From:
Dave Mitchell
Date:
August 26, 2013 15:10
Subject:
Re: [perl #119445] performance bug: perl Thread::Queue is 20x slowerthan Unix pipe
Message ID:
20130826151026.GA2177@iabyn.com
On Sun, Aug 25, 2013 at 05:37:39PM -0700, James E Keenan via RT wrote:
> On Fri Aug 23 17:28:00 2013, johnh@isi.edu wrote:
> > Why is Thread::Queue *so* slow?
> > 
> > I understand it has to do locking and be careful about data
> > structures, but it seems like it is about 20x slower than opening up a
> > Unix pipe, printing to that, reading it back and parsing the result.

Because it is nothing like a UNIX pipe.

A  UNIX pipe takes a stream of bytes, and read and writes chunks of them 
into a shared buffer.

A T::Q buffer takes a stream of perl "things", which might be objects or
other such complex structures, and ensures they they are accessible by
both the originating thread and any potential consumer thread. Migrating a
perl "thing" across a thread boundary is considerably more complex than
copying a byte across.


> > To speculate, I'm thinking the cost is in making all IPC data shared.
> > It would be great if one could have data that is sent over
> > Thread::Queue that is copied, not shared.

But T::Q is build upon a shared array, and is designed to handled shared
data. 

I think the performance you are seeing is the performance I would expect,
and that this is not a bug.




-- 
In England there is a special word which means the last sunshine
of the summer. That word is "spring".

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About