develooper Front page | perl.perl6.users | Postings from November 2021

Re: hope we have the distributed computing perl6

Thread Previous | Thread Next
Piper H
November 29, 2021 06:35
Re: hope we have the distributed computing perl6
Message ID:
William, I didn't use SparkR. I use R primarily for plotting.

Spark's basic API is quite simple, it does the distributed computing of
map, filter, group, reduce etc, which are all covered by perl's map, sort,
grep functions IMO.

for instance, this common statistics on Spark:

>>> fruit.take(5)
[('peach', 1), ('apricot', 2), ('apple', 3), ('haw', 1), ('persimmon', 9)]
>>> fruit.filter(lambda x:x[0] == 'apple').reduceByKey(lambda
[('apple', 86)]

Which is easily implemented by perl's grep and map functions.
But we need a distributed computing framework of perl6.

Yes there is already the perl-spark project:
Which didn't get updated for many years. I don't think it's still in active

So I asked the original question.

Thank you.

On Mon, Nov 29, 2021 at 1:44 PM William Michels <>

> Hi Piper!
> Have you used SparkR (R on Spark)?
> I'm encouraged by the data-type mapping between R and Spark. It
> suggests to me that with a reasonable Spark API, mapping data types
> between Raku and Spark should be straightforward:
> Best Regards,
> Bill.
> On Sat, Nov 27, 2021 at 12:16 AM Piper H <> wrote:
> >
> > I use perl5 everyday for data statistics.
> > The scripts are running on a single server for the computing tasks.
> > I also use R, which has the similar usage.
> > When we face very large data, we change to Apache Spark for distributed
> computing.
> > Spark's interface languages (python, scala, even ruby) are not flexible,
> but their computing capability is amazing, due to the whole cluster
> contributing the computing powers.
> > Yes I know perl5 is somewhat old, but in perl6 why won't we make that a
> distributed computing framework like Spark? Then it will help a lot to the
> data programmer who already knows perl.
> > I expect a lot from this project.
> >
> > Thanks.
> > Piper

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About