develooper Front page | perl.perl6.users | Postings from November 2021

Re: hope we have the distributed computing perl6

Thread Previous | Thread Next
From:
William Michels via perl6-users
Date:
November 29, 2021 05:44
Subject:
Re: hope we have the distributed computing perl6
Message ID:
CAA99HCx3DRsXKM+W1MN_9Szak0xv2xPDhOoapnKmsrS_53LJEA@mail.gmail.com
Hi Piper!

Have you used SparkR (R on Spark)?

https://spark.apache.org/docs/latest/sparkr.html

I'm encouraged by the data-type mapping between R and Spark. It
suggests to me that with a reasonable Spark API, mapping data types
between Raku and Spark should be straightforward:

https://spark.apache.org/docs/latest/sparkr.html#data-type-mapping-between-r-and-spark

Best Regards,

Bill.


On Sat, Nov 27, 2021 at 12:16 AM Piper H <potthua@gmail.com> wrote:
>
> I use perl5 everyday for data statistics.
> The scripts are running on a single server for the computing tasks.
> I also use R, which has the similar usage.
> When we face very large data, we change to Apache Spark for distributed computing.
> Spark's interface languages (python, scala, even ruby) are not flexible, but their computing capability is amazing, due to the whole cluster contributing the computing powers.
> Yes I know perl5 is somewhat old, but in perl6 why won't we make that a distributed computing framework like Spark? Then it will help a lot to the data programmer who already knows perl.
> I expect a lot from this project.
>
> Thanks.
> Piper

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About