develooper Front page | perl.perl5.porters | Postings from May 2016

Re: [perl #128227] vfork should be used for spawning externalprocesses

Thread Previous | Thread Next
From:
Leon Timmermans
Date:
May 25, 2016 08:36
Subject:
Re: [perl #128227] vfork should be used for spawning externalprocesses
Message ID:
CAHhgV8ik3puLjUk4=5i9KGJ4t4uHM3cE=r_Lwfquasq8Yezx+A@mail.gmail.com
On Tue, May 24, 2016 at 1:54 AM, via RT <perlbug-followup@perl.org> wrote:

> Perl currently uses fork + exec for spawning processes with
> system(), pipe open(), and `backtick` operators.
>
> Despite the implementation of copy-on-write (CoW) under Linux,
> fork performance degrades as process size grows.  vfork avoids
> the issue by pausing the parent thread and sharing the heap
> until execve() or _exit() is called.
>
> In my attached example (vfork.perl) using Inline::C, it only takes
> around 3 seconds to vfork+execve+waitpid "/bin/true" 10000 times
> on my system.  With the Perl CORE::system() function, it takes
> 11-12 seconds(!).
>
> This is with a 10MB string in memory.  Increasing the size of the
> $mem string in the attached script degrades performance further.
>
> Keep in mind vfork is tricky to use and some of the caveats
> are documented at: https://ewontfix.com/7/
>
> The mainline Ruby implementation has been using vfork since
> Ruby 2.2 (released December 2014) and AFAIK we have not had
> major problems related to it; even in multi-threaded (pthreads)
> environments.
>
> Disclaimer: I'm a member of the ruby-core team, but I probably
> use Perl more than Ruby :)
>
> Unfortunately my knowledge of Perl internals is weak at this
> point, but I will try my best to help answer questions related
> to implementing vfork.
>
> Thank you for the many years of Perl 5!
>

It appears we removed vfork support 15 years ago in 52e18b1f. I think the
concept of it is out-of-date, and using it correctly is fickle. I don't
think it's a good idea.

It may actually be sane to implement system on top of posix_spawn though
(which can use vfork internally). That at least can be reasoned about, and
seems to generally do what we want (I've never actually used it, there may
be pitfalls that aren't obvious yet).

Not sure either option is really worth the development effort though, any
program where fork is a serious bottleneck is likely to be a mess anyway
unless you're writing a shell or make.

Leon

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About