Front page | perl.perl5.porters |
Postings from August 2013
On Mon, Aug 26, 2013 at 5:58 PM, John Heidemann <johnh@isi.edu> wrote: > I understand that Thread::Queue and perl threads allow shared data, and > that > that's much more than a pipe. > > My concern is that Thread::Queue also *forces* shared data, even when > it's not rqeuired. If that sharing comes with a 20x performance hit, > that should be clear. > > From perlthrtut, the "Pipeline" model > > The pipeline model divides up a task into a series of steps, and > passes > the results of one step on to the thread processing the next. Each > thread does one thing to each piece of data and passes the results > to > the next thread in line. > > For the pipeline model, one does not need repeated sharing, just a > one-time hand-off. Each queue is FIFO with data touched by only one > thread at a time. That's exactly what my particular applications needs > to do. > > But one does not *want* sharing (for the pipeline model) there if it's a > 20x performance hit. > > If the statement is that queues should require shared data and the > corresponding performance hit, that's a design choice one could make. > Then I'd suggest the bug becomes: perlthrtut should say "don't use > Thread::Queue for the pipeline model if you expect high performance, > roll your own IPC". > Actually I did write a queue implementation for threads::lite that should be a lot faster for simple data structures, but I never released it as a separate module that could be used with threads.pm. > Alternatively, I'd love some mechanism to share data between threads > that allows a one-time handoff (not repeated sharing) with pipe-like > performance. One would *think* that shared memory should be able to be > faster than round-tripping through a pipe (with perl parsing and kernel > IO). It seems like a shame that perl is forcing full-on sharing since > it's slow and not required (in this case). > I don't think that would be faster than a queue, given perl's memory model (memory has to be owned by a thread, shared memory has to be be handled manually) a copy or two is necessary anyway. LeonThread Previous | Thread Next