develooper Front page | perl.perl5.porters | Postings from August 2023

Re: PPC Elevator Pitch for Perl::Types

Oodler 577 via perl5-porters
August 22, 2023 18:43
Re: PPC Elevator Pitch for Perl::Types
Message ID:
* Ovid <> [2023-08-22 16:10:00 +0200]:

> On Tue, Aug 22, 2023 at 12:03?PM Dave Mitchell <> wrote:
> > > Regarding your `my number $x; $x = 'foo';` example, this will
> > > require the use of a `tie` or similar mechanism
> > > * increased performance
> >
> > So how would you get increased performance? I've already pointed out that
> > magic tends to slow things down, and the cost of calling a check routine
> > will slow things down even further.

We're going to continuing to do the best we can given we perl gives us only
so much access to internals on a higher level; and the fact that we don't
have "core" developers in our pocket or a monetary budget to speak of. We
our time and our zeal, outside of that we can only deal with the cards perl
gives us. I wish there were better ways to do this, which IMO is exactly
the kind of things any future perl work should entail. Giving us better and
more performant access to empower new and more powerful CPAN modules.

> >
> For some context: RPerl is a transpiler which converts a limited subset of
> Perl-like syntax to C++ <> and
> then runs it at native speed.
> integers and floats are "unboxed" and the native C++ code can run at
> lightning fast speed.

Mostly True. RPerl is a real subset (R = "restricted"), not "Perl-like".
"Perl-like" is something like Qore(.org), which has actual threads
and REAL C++ data types. And really does look like Perl. I encourage
others to check it out, and always have. The only reason I don't use
it is because it's not the One True Interpreter^TM (and never will

Will even has a full book for free, in PDF form on the website,

All for vaporware, tho.

> Perl wasn't designed with software performance in mind and this often has
> not been an issue. Much of our program overhead is I/O, network latency,
> etc. When we get CPU-bound, however, we struggle. Perl's optimized well for
> all sorts of crazy string manipulation and that was *perfect* for the early
> web. In today's data-heavy world, along with AI, lots of math is needed and
> Perl's not very fast at that.

I don't think it is fair to say what Perl was or was not designed
for performance; that's just not a reasonable thing to claim for
people who work on computers most of the day/night. But I know it
wasn't designed for exposing OpenMP, and this is a direction I am
leading and has a nice tie-in to Perl::Types.

Will is an absolute expert with the Perl API and I am far from it,
as much as one can be without being in some club (speaking of

I do have a lot of experience in both HPC and Perl, and can say
that Perl most definitly is an amazing driver for all kinds of
performance related things - yes, via C bindings, but also for a
lot of pure Perl things, provided you do the yoeman's work of
actually profiling with the excellent Devel::NYTProf toolset.

Perl's API is untapped by most, but that's shame and the real place
of most improvement would to make it easier to expose is in a mid-
level way; and I see Perl::Types in may ways as a good example
of this. There are many such benefits to building up support for
this underserved are (not pure Perl but not Perl API). 

The fact PDL exists leads to your point. However, there is a
historical obsession with optimizing perl and perl code. Why does
Devel::NYTProf exist? While we can't say perl (lower "p") is intended
to complete with C in many cases, it is also demonstrably false
that there are constant attempts to optimize it out the ying-yang
for what it does, ruthlessly so. So rutheless that one can't even
ensure a "read only" operation on a SV/AV/HV is thread safe. That's
pretty extreme.

Performance refers to a lot of things. We all want fast and efficient
code, so please just stop trying to convince us that performance
is not an issue.  That's simply false, I don't care what blog post
says it is not.

> It really is a problem with Perl and Perl adoption and I don't know how it
> can easily solved.
> But Perl was designed for *developer* efficiency and the Oshun data check
> project leans into that, hard.

Ironically, what you are doing is extremely similar to Ada's
approach, part of which is called "design by contract". Ada has
full specifications on that and it's done by actual language
designers on US tax payer's dime. It's not bad, just not novel. It
seems a first pass on understanding how to apply Ada's approach in
describing the kinds of constraints you wish to describe would be
well worth the effort - and maybe a few academic papers. Were you
aware that Oshun is heading decidely into a space that has been
well specified by Ada?

The last time I used Ada was in college, and it was for 1 semester.
It is an interesting languge, used even in real time systems like
avionics. So clearly it can be performant, but it also doens't have
the limitations perl does.

Anyway, I suggestion you suggest you call Oshun what it is, "design
by contract", not only would more people understand at what level
it sits; but it would assist in the internal and external designs
by the mountains of existing implementations and specifications.
It might be novel in Perl, but it is not novel.

> As a developer, I want to know that I can write this (hypothetical syntax,
> but very similar to our actual code):

DARPA funded a spate of "High Productivity Computing System"
languages in the 2000s, and it came up with some interesting results.
Some of them could apply to Perl, especially Chapel (originally
ZPL, Ada is something that influenced it).

Again, you should look up "design by contract" and Ada.


> We do not make promises of code performance. We make promises of developer
> performance and data correctness. The amount of code you would have to
> write to manually verify all of these assumptions is ridiculous, so many
> developers skip that entirely.

Ignoring actual performance unwise. I am not suggesting pre-mature
optimizing the tricky stuff, but the obvious stuff, yes. It also
immediately loses most of your potential users. So what's the point
in hedging AGAINST performance? Makes me think you're not really

> We offer:
>    1. Working code (albeit alpha and NOT appropriate for production use)
>    2. Almost 200,000 passing tests
>    3. Documentation (though limited)
>    4. A full, draft spec which needs to be updated when we agree on syntax
>    5. We handle bad assignments to the variable well, but assignments deep
>    inside complex data structures are not handled because of performance. This
>    is a source of contention.
>    6. Trivial to downgrade data checks to warnings, or disable them for
>    performance

How is this not vaporware and somehow RPerl and Perl::Types is
getting sniped on the sidelines for being vapor? This is getting
stupid and we're both looking stupid.

>    7. We have an open process where anyone is allowed to participate. I am
>    a doorman, not a gatekeepr.

We more structured, who cares. We have a gatekeeping process and
are formal. We'll build our in cathedral, you build your bazaar.

> When I say "developer performance" instead of "code performance," consider
> the following C function:

You keep saying "developer performance". See,

And start talking about prior art, nothing you are doing is novel.
At least we don't make that claim.

I do think Chapel could really offer some ideas on how to extend
Perl in actual useful ways, but it's unrelated to your DbC efforts
(which btw do not conflict with Perl::Types, a point we've made).
Perhaps you can show where Perl::Types, as we've demonstrated would
conflict with Oshun. I am not suggesting they are directly composable,
but Perl::Types could definitely assist you where otherwise you'd
just be using regular expressions everywhere (behind some sugar,
of course).

>     #include <stdarg.h>
>     double get_average_temp(int n, ...)
>     {
>         double average = 0.0;
>         va_list ptr;
>         va_start(ptr, n);
>         for (int i = 0; i < n - 1; i++) {
>             average += va_arg(ptr, double);
>         }
>         va_end(ptr);
>         return average / (n - 1);
>     }
> That takes a variadic list of doubles and computes the "average

Because I had no idea and it seems "important",

  variadic := it can take a varying number of arguments

> temperature" for them. It's written in C, so type checks are done at
> compile-time and then thrown out at runtime. Very fast for the computer.
> For the developer, if those temperatures are accidentally a mixture of
> Celcius and Fahrenheit, there will be no error. There will be no warning.
> You might not notice something is wrong. This is not a theoretical
> problem. This
> is a $125 million problem
> <>.
> (billions, maybe trilions, if you could count all the times this mistake
> has been made).

I have no doubt lives and lots of money hinge on Perl, but I assure
you lives and a lot more dollars do hinge on Ada, as it is in
avionics and other real time control systems. DbC is a good thing,
and I don't believe Perl::Types steps in your wading pool here. It
sits lower, and the whole point of what we are doing is providing
a basis for things like DbC, rather than it being a way to apply
a bunch of regular expressions. If someone wants that, they can
integrate Validate::Tiny quite nicely into their project.

> So Oshun assumes we want correct code *first*. (The following assumes
> celsius is a user-defined check):
>     sub get_average_temp(celsius @temps) returns celsius {
>         return Celsius->new( temp => sum0(@temps) / @temps );
>     }
> Later, if you want polymorphism via multi subs, we *might* be able to
> consider something like:
>     multi sub get_average_temp(celsius @temps) returns celsius {
>         return Celsius->new( temp => sum0(@temps) / @temps );
>     }
>     multi sub get_average_temp(fahrenheit @temps) returns fahrenheit {
>         return Fahrenheit->new( temp => sum0(@temps) / @temps );
>     }
> But now we're dispatching on what the arguments mean, not just on primitive
> value types without semantic content.
> This is *Perl*. If you want  CPU performance, you're not going to get it by
> attaching a few labels on things. The RPerl transpiler approach literally
> *cannot* work here. You can't have unboxed ints and floats flying around
> perl's twisty-little maze of macros and even if you did, you lose the
> *semantic* meaning of them. There's no such thing as a "4". It's "4 *of
> something*" and that's what Oshun does.

The "R" in "RPerl" means "restricted". It is for people interested
in the tradeoff between leveraging a real type system (from underlying
C or C++) into their code for performance and type safety. It's
not and never was offered as a design by contract approach. It sits
lower than Oshun, and would benefit your project by giving you
access in a lot of cases to the underlying data structures in the
perl interpreter, both at compile time and at runtime - and varying
degress of abstraction from the high level perl SV/AV/HV to the
actual underlying C or C++ (particular in the case of C++ supported
objects and derived data types).

> I have no idea if the Oshun project will ever get accepted, either. While
> the general syntax and intent seems favorable to many (because it fits what
> Perl already does), the fact that it would be hard to make it performant
> might be its death knell. And when that time comes, I'll point out that
> these checks can be disabled for production, but if that's not enough, I
> will accept that decision.

It seems that the same barrier of entry exists for Oshun as
Perl::Types. They are different things.

Oshun is "Design by Contract" in Perl. Call it what it is. It'll
help you and anyone interested in it clarify the vision and actually
deliver. You can then focus on the UX of DbC and hedge the performance
but of it to whatever abstraction you're currently sitting atop - be
it regular expressions, something sane like Validate::Tiny, or even
Perl::Types provides access to real type systems in perl and below
in the C/C++ compilers. So it sits at a lower level and provides
some compelling access to internal data structures that are otherwise
the realm of witches, "high magic" as it is called.

One thing I will assert is Perl::Types and RPerl is about as much
vaporware as Oshun, so it would be benefit both projects to quit
this "vaporware" BS that your cheerleaders seems to be pushing on
here and on the darker depths of the internet echo chambers. It's not
helpful, and it is clearly not a majority position as evidenced by
the pushback they are receiving.


> --
> Curtis "Ovid" Poe
> --
> CTO, All Around the World
> World-class software development and consulting

SDF-EU Public Access UNIX System - #openmp #pdl #native Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About