Front page | perl.perl5.porters |
Postings from August 2013
On Fri Aug 23 17:28:00 2013, johnh@isi.edu wrote: > > This is a bug report for perl from johnh@isi.edu, > generated with the help of perlbug 1.39 running under perl 5.16.3. > > > ----------------------------------------------------------------- > > Why is Thread::Queue *so* slow? > > I understand it has to do locking and be careful about data > structures, but it seems like it is about 20x slower than opening up a > Unix pipe, printing to that, reading it back and parsing the result. > > Thread::Queue is correct, but I suggest that 20x slower is a > performance bug. > > One would think that IPC through memory would be at least as fast as a > pipe through the kernel, and ideally it should be faster. > > Here's timing of a test program that sends 500k integers between two > threads, > using Thread::Queue or pipe(2). > > $ ./thread_ipc_perf.pl -m queue > benchmark took 14 wallclock secs (14.71 usr + 2.51 sys = 17.22 CPU) @ > 0.06/s (n=1) > > $ ./thread_ipc_perf.pl -m pipe > benchmark took 0 wallclock secs ( 0.59 usr + 0.00 sys = 0.59 CPU) @ > 1.69/s (n=1) > > Here's a larger run (1M integers) with the same kind of results. > > $ ./thread_ipc_perf.pl -N 1000000 -m queue > benchmark took 30 wallclock secs (32.69 usr + 6.06 sys = 38.75 CPU) @ > 0.03/s (n=1) > > $ ./thread_ipc_perf.pl -N 1000000 -m pipe > benchmark took 1 wallclock secs ( 1.23 usr + 0.00 sys = 1.23 CPU) @ > 0.81/s (n=1) > > > Source code for the above simple benchmark is at > http://www.isi.edu/~johnh/SOFTWARE/FSDB/thread_ipc_perf.pl.txt > > > We can quibble over the exact multiplier (maybe it's only 15x slower), > but it's *really* slow. > > Any suggestions? I get similar results if I simplify Thread::Queue to > bare minimum code. > > To speculate, I'm thinking the cost is in making all IPC data shared. > It would be great if one could have data that is sent over > Thread::Queue that is copied, not shared. > > > Thanks for any suggestions, > -John Heidemann > > > > > [Please do not change anything below this line] > ----------------------------------------------------------------- > --- > Flags: > category=library > severity=medium > module=Thread::Queue > --- > Site configuration information for perl 5.16.3: > > Configured by Red Hat, Inc. at Tue Jun 18 09:17:09 UTC 2013. > > Summary of my perl5 (revision 5 version 16 subversion 3) > configuration: > > Platform: > osname=linux, osvers=2.6.32-358.2.1.el6.x86_64, archname=x86_64- > linux-thread-multi > uname='linux buildvm-05.phx2.fedoraproject.org 2.6.32- > 358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64 > x86_64 x86_64 gnulinux ' > config_args='-des -Doptimize=-O2 -g -pipe -Wall > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector > --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 > -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags > -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 > -fexceptions -fstack-protector --param=ssp-buffer-size=4 > -grecord-gcc-switches -m64 -mtune=generic -Wl,-z,relro > -DDEBUGGING=-g -Dversion=5.16.3 -Dmyhostname=localhost > -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. > -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local > -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 > -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl > -Darchlib=/usr/lib64/perl5 > -Dvendorarch=/usr/lib64/perl5/vendor_perl > -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 > /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads > -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db > -Ui_ndbm -Di_gdbm -Di_shadow -Di_sysl! > og -Dman3 > ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 > -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto > -Ud_endhostent_r_proto -Ud_sethostent_r_proto > -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto > -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin > -Dusesitecustomize' That's a lot of configuration options. While I don't doubt that you have a reason for all of them, I also doubt that many people are going to want to build a perl with all those options just for the purpose of testing your claim. Would it be possible for you to try this again with the absolute minimum number of configuration options required to build a threaded perl which manifests the problem? Thank you very much. Jim Keenan --- via perlbug: queue: perl5 status: new https://rt.perl.org:443/rt3/Ticket/Display.html?id=119445Thread Previous | Thread Next