develooper Front page | perl.perl5.porters | Postings from April 2021

Re: threading and stuff (was Re: Perl’s leaky bucket)

Thread Next
From:
B. Estrade
Date:
April 7, 2021 23:55
Subject:
Re: threading and stuff (was Re: Perl’s leaky bucket)
Message ID:
ed6d3961-2ebd-56e7-39b1-e87e30992e4e@cpanel.net


On 4/7/21 6:46 PM, Dan Book wrote:
> On Wed, Apr 7, 2021 at 7:34 PM B. Estrade <brett@cpanel.net 
> <mailto:brett@cpanel.net>> wrote:
> 
>     I appreciate the reply!
> 
>     On 4/7/21 2:40 PM, Nicholas Clark wrote:
>      > On Wed, Apr 07, 2021 at 10:49:05AM -0500, B. Estrade wrote:
>      >
>      >> Here's some food for thought. When was was the last time anyone
>     had a
>      >> serious discussion of introducing smp thread safety into the
>     language? Or
>      >> easy access to atomics? These are, IMO, 2 very important and
>     serious things
>      >
>      > Retrofitting proper multicore concurrency to perl would imply
>     that it's
>      > possible at all.
> 
>     Just to be clear, that's not what I think is prudent, either. Just
>     having the conversation about exposing it where possible would be
>     really
>     beneficial. E.g., where is it possible and where does it makes sense.
> 
>     When I think about it, I don't think about anything beyond a simple
>     fork/join model (e.g., what OpenMP exposes at a high level) + some
>     synchronization primitives (barriers), basic memory initialization
>     controls, basic scheduling. and some built-in data reduction operations.
> 
>     And still, I don't expect that. Redis takes that approach (just as an
>     example). It's single process, except where it's safe and beneficial to
>     thread-out.
> 
>     What does even limit exposure to SMP give us? I do not know. But, we
>     would probably be pleasantly surprised; and from what little access
>     we're given, some might get pretty creative in using it.
> 
>     Going off of OpenMP's fork/join model (which has a richly developed
>     fork/join model); having a block that provides a restricted "threaded"
>     environment needs:
> 
>     * access to thread id
>     * some ability to use private variables
>     * access to a subset of memory from the parent process pre-fork
>     * a synchronization primitives (barriers to force serialization if
>     needed)
> 
>     Just some thoughts. But would it be useful to have a highly restricted
>     "threaded" block, even with some basic abilities? I think so, 100%.
> 
>     my $sum = 0;
>     my $num_threads = 5;
>     smp_fork($num_threads) {
> 
>         # all threads do this using a private variable
>         local $tid = tfork_id();
> 
>         # one thread a time may enter here, in no particular order
>         critical {
>           $sum += $tid;
>         }
>     }
>     # end of block is an implicit "join" where threads die, private memory
>     # is reaped
>     say "sum is $sum"; # here $sum is "15" - 0+1+2+3+4+5
> 
>     There's nothing academic or pretty about the above; but it would
>     open up
>     a tiny window to exploit the host machines power. And if, somehow, Perl
>     had this; it'd provide a starting point to do other things. It'd likely
>     get a whole lot of folks very excited; not to mention open the doors
>     for
>     new applications. Users who enter into this area would be required to
>     heed all the normal caveats of shared memory programming.
> 
>     To put it another way, I'd rather Perl have a highly restricted
>     "threaded" environment (or "sandbox") than none at all. What that looks
>     like is any one's guess.
> 
>     With all that said, is this so off the wall or wrong that it's a
>     non-starter? If so, what would a dip in the SMP pond look like in terms
>     of exposing it (somehow) in Perl - even if it is just enough to get
>     module authors some reach that would then facilitate the bulk of any
>     capability to be implemented externally.
> 
> 
> For what it's worth, check out MCE and IO::Async::Function on CPAN for 
> existing art of this sort of thing (they both even leverage threads.pm 
> <http://threads.pm> on Windows, where forks aren't great for this).
> 
> -Dan

Yes, thank you, Dan. I am aware of these. I also use fork a lot and am 
happy exploiting machine CPUs that way. But I really (really) want 
personally - and think it'd be a highly compelling capability for Perl 
itself - is to be able to exploit SMP natively, even if "unsafely".

On my list of ways to "future proof" Perl, this is #1 without any 
doubts. Beyond that, anything I said or suggested is largely uniformed 
and comes from how I think I would personally use it. Even if only a 
robust conversation about it comes from this (and I believe it has 
started, so thank you) I will be happy.

Brett

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About