Front page | perl.perl5.porters |
Postings from April 2021
From: B. Estrade
April 8, 2021 00:50
Message ID: firstname.lastname@example.org
On 4/7/21 7:13 PM, Dan Book wrote:
> On Wed, Apr 7, 2021 at 7:55 PM B. Estrade <email@example.com
> <mailto:firstname.lastname@example.org>> wrote:
> Yes, thank you, Dan. I am aware of these. I also use fork a lot and am
> happy exploiting machine CPUs that way. But I really (really) want
> personally - and think it'd be a highly compelling capability for Perl
> itself - is to be able to exploit SMP natively, even if "unsafely".
> I'm not sure I understand. How does "exploit SMP natively" with a
> "fork/join model" differ from what these modules offer?
Sorry, I was not clear. I meant "natively" within Perl.
Just to provide contrast, MCE provides fork process management and
communication among forked processes. Most of my experience with
threading is from OpenMP, which is supported by every major compiler out
there. Granted it's high level and declarative via the use of pragmas.
So I reference this with the risk that that point gets misconstrued.
It's the threading model (fork/join) or (spawn/join) that I am talking
I am not sure how else to explain it without sending you a bunch of
google links and risking the insulting of your knowledge.
So I can put it in terms of what I think would be great.
0. upon reaching a "threaded block", lightweight threads (e.g.,
pthreads) are forked from the parent process
1. each thread is aware of it's id (0 .. N-1); parent is tid 0; all
threads sibling threads have access to parent process's memory
2. each thread has the ability to create private variables
3. communication is done so implicitly through the use of shared memory
and various forms of synchronization constructs
4. at the end of the block, child private memorys are destroyed
(including parent thread tid 0)
5. primary process memory reachable by child processes during the
threaded activity is available, potentially as affected during the
threaded fork (come what may)
I believe the essence of what characterizes what I am thinking comes
down to 2 things:
a. the exploitation of lightweight threads spawned from the parent
b. the availability of regions of directly shared memory to the child
I don't know squat about internals, but I do believe I have a handle on
the ideal that a "perl" process is a singular line of execution. What I
am suggesting would take that process; and for a moment in the execution
spawn it into threads whereby the threads may operate independently over
the same process space (here's the 'shared memory' portion).
It is very important to know that I understand this is not a "fork". The
process is not "cloned". There is a period of time where it is executing
a very precise piece of code using multiple threads with access to the
parent process's memory. The distinction in terms of it versus the
"fork" in Perl is that it's not a bunch of cloned perl processes that
operate in their own space as if there was no interruption. There is
only ever one process and the part that the threads operate over are
fully encapsulated inside of the original process. They communicate
through the developer carefully coordinating the reading and writing of
these areas of shared memory - most of which was initialized in the main
process pre-spawn. In OpenMP lingo, it's a "fork/join"; but since we use
that word "fork" to mean something very specific as it relates to how
the operating system treats it, it's probably better to call it a
"spawn/join" (not to be further confused with the "join" keyword).
Note, while I keep saying "OpenMP"; I do so as it relates to the
simplified model it presents for those who use it. Most compilers that
implement it (pretty much all relevant ones) facilitate it by injecting
pthread calls during source transformation phases of the compilation. If
I am worried about any unforeseen complication with what I am
suggesting, it's the concern that what I am asking for would necessarily
require Perl to do something crazy like that. I hope that's not the
case. In my mind, what I am suggesting looks like:
0. perl hits spawn block
1. perl calls pthreads init to generate threads
2. threads do stuff (again, if this idea is "impossible" it's here)
3. threaded section ends, threads die; original parent process memory
remains as affect
All solutions I have seen have employed a combination of fork+provide
communication layer or are asynchronous "fire and forget" - none of
which are SMP.
I must also be careful to make sure it's clear I am not asking for
phtread "scripting" support. This is what Qore is, essentially - albeit
presented with a very perlish syntax.
My main objective is to ask, "what would it look like in perl to provide
some regions of shared memory over which lightweight threads may operate".
Hope that helps.
by B. Estrade
by Dan Book
by B. Estrade