develooper Front page | perl.perl5.porters | Postings from July 2011

[perl #94472] cygwin-thread-multi-64int is slow in multi-threaded code that relies heavily on split command

Thread Previous
From:
Alistair McGlinchy
Date:
July 11, 2011 02:06
Subject:
[perl #94472] cygwin-thread-multi-64int is slow in multi-threaded code that relies heavily on split command
Message ID:
rt-3.6.HEAD-30268-1310342432-1149.94472-75-0@perl.org
# New Ticket Created by  Alistair McGlinchy 
# Please include the string:  [perl #94472]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=94472 >


This is a bug report for perl from amcglinchy@gmail.com,
generated with the help of perlbug 1.36 running under perl 5.10.0.


-----------------------------------------------------------------


When running a threaded perl program in Cygwin that relies heavily on the
split command, a massive decrease in performance has been noted over the
single threaded version. 100_000 splits of a 250 byte string goes from 2.6
seconds to 12.5 seconds when spread across multiple threads.  The expected
result of a small increase in performance can be seen with the same code
against the Activestate build.

The following code can reproduce the effect.

## Test code to show split slow when used in muiltiple threads is slow in
cygwin perl
use strict;
use warnings;
use threads;
use Thread::Queue;
use Time::HiRes();

my $string = "12356789 "x25;
my $TASK_COUNT =10;

print( (`$^X -v`)[1]);
thread_process(1);
thread_process(4);

sub thread_process {
    my $parallel = shift;
    my $start = Time::HiRes::time();
    my @threads;
    my $queue = Thread::Queue->new();
    for my $p ( 1 .. $parallel ) {
        push @threads, threads->create( sub {
            while( my $task_no = $queue->dequeue) {
                for my $n (1..10_000) {
                    my @x= split / /, $string;
                }
        }})
    }
    $queue->enqueue( 1 .. $TASK_COUNT  , (undef) x $parallel );
    $_->join for @threads;
    my $end= Time::HiRes::time();
    printf "Processing %d tasks in %d threads completed in %fsecs\n",
$TASK_COUNT, $parallel, $end-$start;
}
__END__

# Here is an example of the output on my dual core XP machine

$ ~/localperl/bin/perl.exe example.split.pl
This is perl 5, version 14, subversion 1 (v5.14.1) built for
cygwin-thread-multi-64int
Processing 10 tasks in 1 threads completed in 2.640625secs
Processing 10 tasks in 4 threads completed in 12.468637secs

$ perl example.split.pl
This is perl, v5.10.0 built for cygwin-thread-multi-64int
Processing 10 tasks in 1 threads completed in 4.578125secs
Processing 10 tasks in 4 threads completed in 25.437345secs

# The defect does not occur with ActivePerl. Instead the expected slightly
faster result occurs
$ c:/perl/bin/perl example.split.pl
This is perl 5, version 12, subversion 3 (v5.12.3) built for
MSWin32-x86-multi-thread
Processing 10 tasks in 1 threads completed in 2.062500secs
Processing 10 tasks in 4 threads completed in 1.671790secs



-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl 5.10.0:

Configured by rurban at Mon Jun 30 16:03:19 GMT 2008.

Summary of my perl5 (revision 5 version 10 subversion 0 patch 34065)
configuration:
  Platform:
    osname=cygwin, osvers=1.5.25(0.15642),
archname=cygwin-thread-multi-64int
    uname='cygwin_nt-5.1 reini 1.5.25(0.15642) 2008-06-12 19:34 i686 cygwin
'
    config_args='-de -Dmksymlinks -Dusethreads -Dmad=y -Dusedevel'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__
-fno-strict-aliasing -pipe -I/usr/local/include',
    optimize='-O3',
    cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fno-strict-aliasing
-pipe -I/usr/local/include'
    ccversion='', gccversion='3.4.4 (cygming special, gdc 0.12, using dmd
0.125)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols
-Wl,--stack,8388608 -Wl,--enable-auto-image-base -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib
    libs=-lgdbm -ldb -ldl -lcrypt -lgdbm_compat
    perllibs=-ldl -lcrypt
    libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' --shared  -Wl,--enable-auto-import
-Wl,--export-all-symbols -Wl,--stack,8388608 -Wl,--enable-auto-image-base
-L/usr/local/lib'

Locally applied patches:
    MAINT34065
    CYG11 no-bs
    CYG12 no archlib in otherlibdirs
    CYG14 Dynaloader
    CYG15 static-Win32CORE
    Bug#55162 File::Spec::case_tolerant performance

---
@INC for perl 5.10.0:
    /usr/lib/perl5/5.10/i686-cygwin
    /usr/lib/perl5/5.10
    /usr/lib/perl5/site_perl/5.10/i686-cygwin
    /usr/lib/perl5/site_perl/5.10
    /usr/lib/perl5/vendor_perl/5.10/i686-cygwin
    /usr/lib/perl5/vendor_perl/5.10
    /usr/lib/perl5/vendor_perl/5.10
    /usr/lib/perl5/site_perl/5.8
    /usr/lib/perl5/vendor_perl/5.8
    .

---
Environment for perl 5.10.0:
    CYGWIN_ROOT=\cygwin
    HOME=/home/Awie
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:.:/usr/bin:/cygdrive/c/bats/:/cygdrive/c/Program
Files/ActiveState Perl Dev Kit
7.0/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/Perl/site/bin:/cygdrive/c/Perl/bin:/cygdrive/c/Program
Files/TortoiseSVN/bin:/cygdrive/c/bats:/cygdrive/c/Parrot-2.5.0/bin:/cygdrive/c/MinGW/bin:/cygdrive/c/MySQL5.1/bin:/cygdrive/c/Program
Files/QuickTime/QTSystem/:/cygdrive/c/WINDOWS/system32/WindowsPowerShell/v1.0:/usr/bin:/usr/bin:/usr/lib/lapack
    PERL_BADLANG (unset)
    SHELL (unset)
Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About