develooper Front page | perl.perl5.porters | Postings from August 2013

Re: [perl #119445] performance bug: perl Thread::Queue is 20x slower than Unix pipe

Thread Next
From:
John Heidemann
Date:
August 26, 2013 20:39
Subject:
Re: [perl #119445] performance bug: perl Thread::Queue is 20x slower than Unix pipe
Message ID:
15905.1377488218@dash.isi.edu
On Sun, 25 Aug 2013 17:37:39 -0700, "James E Keenan via RT" wrote: 
>On Fri Aug 23 17:28:00 2013, johnh@isi.edu wrote:
>> 
>> This is a bug report for perl from johnh@isi.edu,
>> generated with the help of perlbug 1.39 running under perl 5.16.3.
>> 
>> 
>> -----------------------------------------------------------------
>> 
>> Why is Thread::Queue *so* slow?
...
>> 
>> $ ./thread_ipc_perf.pl -m queue
>> benchmark took 14 wallclock secs (14.71 usr +  2.51 sys = 17.22 CPU) @
>>    0.06/s (n=1)
>> 
>> $ ./thread_ipc_perf.pl -m pipe
>> benchmark took  0 wallclock secs ( 0.59 usr +  0.00 sys =  0.59 CPU) @
>>    1.69/s (n=1)
...

>> Source code for the above simple benchmark is at
>> http://www.isi.edu/~johnh/SOFTWARE/FSDB/thread_ipc_perf.pl.txt
...

>> Site configuration information for perl 5.16.3:
>> 
>> Configured by Red Hat, Inc. at Tue Jun 18 09:17:09 UTC 2013.
>> 
>> Summary of my perl5 (revision 5 version 16 subversion 3)
>>    configuration:
>> 
>>   Platform:
>>     osname=linux, osvers=2.6.32-358.2.1.el6.x86_64, archname=x86_64-
>>    linux-thread-multi
>>     uname='linux buildvm-05.phx2.fedoraproject.org 2.6.32-
>>    358.2.1.el6.x86_64 #1 smp wed feb 20 12:17:37 est 2013 x86_64
>>    x86_64 x86_64 gnulinux '
>>     config_args='-des -Doptimize=-O2 -g -pipe -Wall
>>    -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
>>    --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64
>>    -mtune=generic -Dccdlflags=-Wl,--enable-new-dtags
>>    -Dlddlflags=-shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
>>    -fexceptions -fstack-protector --param=ssp-buffer-size=4
>>    -grecord-gcc-switches  -m64 -mtune=generic -Wl,-z,relro
>>    -DDEBUGGING=-g -Dversion=5.16.3 -Dmyhostname=localhost
>>    -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc.
>>    -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local
>>    -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5
>>    -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
>>    -Darchlib=/usr/lib64/perl5
>>    -Dvendorarch=/usr/lib64/perl5/vendor_perl
>>    -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64
>>    /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads
>>    -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db
>>    -Ui_ndbm -Di_gdbm -Di_shadow -Di_sysl!
>>  og -Dman3
>>  ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005
>>    -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto
>>    -Ud_endhostent_r_proto -Ud_sethostent_r_proto
>>    -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto
>>    -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin
>>    -Dusesitecustomize'
>
>That's a lot of configuration options.  While I don't doubt that you
>have a reason for all of them, I also doubt that many people are going
>to want to build a perl with all those options just for the purpose of
>testing your claim.
>
>Would it be possible for you to try this again with the absolute minimum
>number of configuration options required to build a threaded perl which
>manifests the problem?
>
>Thank you very much.
>Jim Keenan

Thanks for the reply.

I don't build perl myself, those are the default configure options for
Fedora Linux.  (Presumably RHEL its derivatives uses similar builds.)

I can build perl if you really want, but let me suggest an alternative
if you don't mind:

I provided source code to my benchmark program at:

	http://www.isi.edu/~johnh/SOFTWARE/FSDB/thread_ipc_perf.pl.txt

and the two invocations that clearly show the difference on my platform:

>> $ ./thread_ipc_perf.pl -m queue
>> benchmark took 14 wallclock secs (14.71 usr +  2.51 sys = 17.22 CPU) @
>>    0.06/s (n=1)
>> 
>> $ ./thread_ipc_perf.pl -m pipe
>> benchmark took  0 wallclock secs ( 0.59 usr +  0.00 sys =  0.59 CPU) @
>>    1.69/s (n=1)

The benchmark is 293 lines long, but it's mostly POD documentation and
boilerplate.  Can I suggest you download the benchmark and try those two
invocations ("./thread_ipc_perf.pl" -m queue and "./thread_ipc_perf.pl
-m pipe") on whatever perl you prefer?

If some other platform or build has much different performance, I'll
take this up with my OS provider.

   -John
 

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About