Front page | perl.perl5.porters |
Postings from May 2016
[perl #128227] vfork should be used for spawning external processes
Thread Next
From:
perlbug-followup
Date:
May 24, 2016 13:21
Subject:
[perl #128227] vfork should be used for spawning external processes
Message ID:
rt-4.0.18-24908-1464047651-1524.128227-75-0@perl.org
# New Ticket Created by
# Please include the string: [perl #128227]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt.perl.org/Ticket/Display.html?id=128227 >
This is a bug report for perl from e@80x24.org,
generated with the help of perlbug 1.40 running under perl 5.24.0.
-----------------------------------------------------------------
Perl currently uses fork + exec for spawning processes with
system(), pipe open(), and `backtick` operators.
Despite the implementation of copy-on-write (CoW) under Linux,
fork performance degrades as process size grows. vfork avoids
the issue by pausing the parent thread and sharing the heap
until execve() or _exit() is called.
In my attached example (vfork.perl) using Inline::C, it only takes
around 3 seconds to vfork+execve+waitpid "/bin/true" 10000 times
on my system. With the Perl CORE::system() function, it takes
11-12 seconds(!).
This is with a 10MB string in memory. Increasing the size of the
$mem string in the attached script degrades performance further.
Keep in mind vfork is tricky to use and some of the caveats
are documented at: https://ewontfix.com/7/
The mainline Ruby implementation has been using vfork since
Ruby 2.2 (released December 2014) and AFAIK we have not had
major problems related to it; even in multi-threaded (pthreads)
environments.
Disclaimer: I'm a member of the ruby-core team, but I probably
use Perl more than Ruby :)
Unfortunately my knowledge of Perl internals is weak at this
point, but I will try my best to help answer questions related
to implementing vfork.
Thank you for the many years of Perl 5!
-----------------------------------------------------------------
---
Flags:
category=core
severity=wishlist
---
Site configuration information for perl 5.24.0:
Configured by ew at Mon May 23 22:06:43 UTC 2016.
Summary of my perl5 (revision 5 version 24 subversion 0) configuration:
Commit id: be2c0c650b028f54e427f2469a59942edfdff8a9
Platform:
osname=linux, osvers=4.5.3-x86_64-linode67, archname=x86_64-linux-64int
uname='linux dcvr 4.5.3-x86_64-linode67 #3 smp tue may 10 10:22:44 edt 2016 x86_64 gnulinux '
config_args='-des -Dprefix=/home/eee/p -Duse64bitint'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
optimize='-O2',
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678, doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12, longdblkind=3
ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/i586-linux-gnu/4.9/include-fixed /usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib /usr/lib/i386-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
---
@INC for perl 5.24.0:
/home/eee/p/lib/perl5/site_perl/5.24.0/x86_64-linux-64int
/home/eee/p/lib/perl5/site_perl/5.24.0
/home/eee/p/lib/perl5/5.24.0/x86_64-linux-64int
/home/eee/p/lib/perl5/5.24.0
.
---
Environment for perl 5.24.0:
HOME=/home/eee
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/eee/p/bin:/usr/bin:/bin
PERL_BADLANG (unset)
SHELL=/bin/bash
Thread Next
-
[perl #128227] vfork should be used for spawning external processes
by perlbug-followup