develooper Front page | perl.perl5.porters | Postings from August 2021

Re: concurrency list (was Re: Relinquishing Maintenance of CoreModules)

Thread Previous | Thread Next
Paul "LeoNerd" Evans
August 3, 2021 13:17
Re: concurrency list (was Re: Relinquishing Maintenance of CoreModules)
Message ID:
On Mon, 2 Aug 2021 14:12:18 -0700
David Christensen <> wrote:

> Have you thought ahead to distributed/ network programming?

We discussed it at great length in Amsterdam, 2019, yes.

I have so far avoided wading into this "concurrency" chat, but at this
point I sortof feel I must. I'm not going to specifically respond to
individual points, but rather address the entire tone of the thread as
a whole.

The thread started with two points, one about Perl's (perceived) lack
of support for "concurrency", and a thought about starting a mailing
list about it.

## On Perl's Concurrency

Yuki Kimoto made a good summary here; I shall quote:

> I think we need to properly classify concurrency(fork, thread, I/O,
> CPU(OpenMP), GPU, coroutine, async/await syntax) before we say we want
> to add concurrency.

That list suggests the usual way I like to stare at these things - are
we talking concurrency of communication, or concurrency of computation?
I.e. are the tasks we're performing IO-bound or CPU-bound?

If the former, then I really don't buy the statement that Perl is
lacking in this area. We have tonnes of good stuff - as well as a
variety of "older" event systems, we also have a growing collection of
Future-based asynchronous code management. For example, if you want to
fetch a bunch of HTTP URLs by making 10 concurrent connections at once,
you can (eliding the `use` statements for brevity):

  my @urls = (lots of URLs here);

  my $http = Net::Async::HTTP->new;
  IO::Async::Loop->add( $http );

  my @responses = await Future::Utils::fmap {
    $http->GET( $_ )
  } foreach => \@urls, concurrent => 10;

Similarly any other sorts of "large amounts of concurrent IO" problems
are equally simple with a lot of these modern solutions.

Or perhaps you meant the latter kind of problem; those CPU-bound ones.
Thing is a whole heap of those are equally trivial with this sort of
system. Perhaps we want to find which English word gives us the lowest
sha256 sum when concatenated with some payload data, by splitting the
task across 10 CPU cores (again eliding `use` statements):

  my @words = map { chomp $_; $_ }

  my $payload = (much data here);

  my $result = List::UtilsBy::minstr_by { $_->[1] }
               Parallel::Map::pmap_scalar {
                 [ $_, Crypt::Digest::SHA256::sha256($payload . $_) ]
               } foreach => \@words, forks => 10;

  my $best_word = $result->[0];

It's hard to see what is fundamentally missing from the language here.
Perhaps the async/await keywords could be core syntax. Or perhaps some
of the modules involved could be shipped by core. But is that
necessary? After all, core doesn't ship a CSV parser, or a database
connector, or an HTML template system, and yet people don't say Perl is
lacking in these, they just grab them from CPAN.

In terms of the innermost details of implementation, it's perhaps
possible to find more performant ways that the insides can be
implemented, perhaps by using some shared memory structure instead of
the marshalling overhead of using some pipes, but overall that doesn't
fundamentally change the shape of the solution, nor the way the code is
written, it just makes it burn slightly fewer CPU cycles while it runs.
(I believe there are various projects underway looking at exactly this
problem, by the way).

Overall though I really don't accept the statement that "Perl can't do
concurrency". I think I demonstrate above that it clearly can do both
concurrency-of-communication and concurrency-of-computation.

## On Needing A Mailing List

If you really feel it necessary, sure go set one up. I don't feel that
a "perl + concurrency" mailing list is a fundamentally different beast
to a "perl + databases" or "perl + electronics" list. Here's some stuff
we could do in Perl, via the help of some CPAN modules, and we can
discuss that subject. 

Paul "LeoNerd" Evans      |  |

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About