Front page | perl.perl5.porters |
Postings from July 2011
[perl #94472] cygwin-thread-multi-64int is slow in multi-threaded code that relies heavily on split command
Thread Previous
From:
Alistair McGlinchy
Date:
July 11, 2011 02:06
Subject:
[perl #94472] cygwin-thread-multi-64int is slow in multi-threaded code that relies heavily on split command
Message ID:
rt-3.6.HEAD-30268-1310342432-1149.94472-75-0@perl.org
# New Ticket Created by Alistair McGlinchy
# Please include the string: [perl #94472]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=94472 >
This is a bug report for perl from amcglinchy@gmail.com,
generated with the help of perlbug 1.36 running under perl 5.10.0.
-----------------------------------------------------------------
When running a threaded perl program in Cygwin that relies heavily on the
split command, a massive decrease in performance has been noted over the
single threaded version. 100_000 splits of a 250 byte string goes from 2.6
seconds to 12.5 seconds when spread across multiple threads. The expected
result of a small increase in performance can be seen with the same code
against the Activestate build.
The following code can reproduce the effect.
## Test code to show split slow when used in muiltiple threads is slow in
cygwin perl
use strict;
use warnings;
use threads;
use Thread::Queue;
use Time::HiRes();
my $string = "12356789 "x25;
my $TASK_COUNT =10;
print( (`$^X -v`)[1]);
thread_process(1);
thread_process(4);
sub thread_process {
my $parallel = shift;
my $start = Time::HiRes::time();
my @threads;
my $queue = Thread::Queue->new();
for my $p ( 1 .. $parallel ) {
push @threads, threads->create( sub {
while( my $task_no = $queue->dequeue) {
for my $n (1..10_000) {
my @x= split / /, $string;
}
}})
}
$queue->enqueue( 1 .. $TASK_COUNT , (undef) x $parallel );
$_->join for @threads;
my $end= Time::HiRes::time();
printf "Processing %d tasks in %d threads completed in %fsecs\n",
$TASK_COUNT, $parallel, $end-$start;
}
__END__
# Here is an example of the output on my dual core XP machine
$ ~/localperl/bin/perl.exe example.split.pl
This is perl 5, version 14, subversion 1 (v5.14.1) built for
cygwin-thread-multi-64int
Processing 10 tasks in 1 threads completed in 2.640625secs
Processing 10 tasks in 4 threads completed in 12.468637secs
$ perl example.split.pl
This is perl, v5.10.0 built for cygwin-thread-multi-64int
Processing 10 tasks in 1 threads completed in 4.578125secs
Processing 10 tasks in 4 threads completed in 25.437345secs
# The defect does not occur with ActivePerl. Instead the expected slightly
faster result occurs
$ c:/perl/bin/perl example.split.pl
This is perl 5, version 12, subversion 3 (v5.12.3) built for
MSWin32-x86-multi-thread
Processing 10 tasks in 1 threads completed in 2.062500secs
Processing 10 tasks in 4 threads completed in 1.671790secs
-----------------------------------------------------------------
---
Flags:
category=core
severity=medium
---
Site configuration information for perl 5.10.0:
Configured by rurban at Mon Jun 30 16:03:19 GMT 2008.
Summary of my perl5 (revision 5 version 10 subversion 0 patch 34065)
configuration:
Platform:
osname=cygwin, osvers=1.5.25(0.15642),
archname=cygwin-thread-multi-64int
uname='cygwin_nt-5.1 reini 1.5.25(0.15642) 2008-06-12 19:34 i686 cygwin
'
config_args='-de -Dmksymlinks -Dusethreads -Dmad=y -Dusedevel'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=undef, uselongdouble=undef
usemymalloc=y, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__
-fno-strict-aliasing -pipe -I/usr/local/include',
optimize='-O3',
cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fno-strict-aliasing
-pipe -I/usr/local/include'
ccversion='', gccversion='3.4.4 (cygming special, gdc 0.12, using dmd
0.125)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='g++', ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols
-Wl,--stack,8388608 -Wl,--enable-auto-image-base -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib /lib
libs=-lgdbm -ldb -ldl -lcrypt -lgdbm_compat
perllibs=-ldl -lcrypt
libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags=' --shared -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608 -Wl,--enable-auto-image-base
-L/usr/local/lib'
Locally applied patches:
MAINT34065
CYG11 no-bs
CYG12 no archlib in otherlibdirs
CYG14 Dynaloader
CYG15 static-Win32CORE
Bug#55162 File::Spec::case_tolerant performance
---
@INC for perl 5.10.0:
/usr/lib/perl5/5.10/i686-cygwin
/usr/lib/perl5/5.10
/usr/lib/perl5/site_perl/5.10/i686-cygwin
/usr/lib/perl5/site_perl/5.10
/usr/lib/perl5/vendor_perl/5.10/i686-cygwin
/usr/lib/perl5/vendor_perl/5.10
/usr/lib/perl5/vendor_perl/5.10
/usr/lib/perl5/site_perl/5.8
/usr/lib/perl5/vendor_perl/5.8
.
---
Environment for perl 5.10.0:
CYGWIN_ROOT=\cygwin
HOME=/home/Awie
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:.:/usr/bin:/cygdrive/c/bats/:/cygdrive/c/Program
Files/ActiveState Perl Dev Kit
7.0/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/Perl/site/bin:/cygdrive/c/Perl/bin:/cygdrive/c/Program
Files/TortoiseSVN/bin:/cygdrive/c/bats:/cygdrive/c/Parrot-2.5.0/bin:/cygdrive/c/MinGW/bin:/cygdrive/c/MySQL5.1/bin:/cygdrive/c/Program
Files/QuickTime/QTSystem/:/cygdrive/c/WINDOWS/system32/WindowsPowerShell/v1.0:/usr/bin:/usr/bin:/usr/lib/lapack
PERL_BADLANG (unset)
SHELL (unset)
Thread Previous