develooper Front page | perl.perl5.porters | Postings from August 2023

Re: PPC Elevator Pitch for Perl::Types

Dave Mitchell
August 22, 2023 10:03
Re: PPC Elevator Pitch for Perl::Types
Message ID:
On Tue, Aug 22, 2023 at 04:19:04AM +0000, Oodler 577 via perl5-porters wrote:
> Yes you are correct, the current mechanism for enabling type-checking
> for subroutine calls is essentially a type of source filter.  As
> mentioned in our first reply:
> > > The Perl compiler currently supports type enforcement for subroutine calls, so that is our starting point for Perl::Types.
> This source-filter-like functionality is currently contained in
> the Perl compiler's file ``

So just for the avoidance of doubt, is the proposal that Perl::Types,
when running in a perl (rather than RPerl) environment, will use perl's
actual source filter mechanism, or that it will use some other
"essentially a type of source filter" thing?

If the latter, please expand.

If the former, note that we generally regard the perl source filter
mechanism as deeply flawed, and wouldn't recommend its use in production
code. The basic problem comes down to the old mantra that "only perl can
parse Perl". A source filter may correctly parse the source 99.9% of the
time, but on those 0.1% occasions, it will get something wrong. Maybe it
will get the start of a // operator confused with the start of a pattern.
Or maybe it will trip over some complex code embeded into a pattern:
/...(?{  $x = '/'; }) .../. Or whatever. At that point, code will get
injected at the wrong point into the source, and the end user gets a
compile error from code that isn't even in their source, which is
completely mystifying and hard to debug. used source filters, and it was the cause of an endless stream
of tickets. It was eventually deprecated and removed from core in 5.14.0.

> Regarding your `my number $x; $x = 'foo';` example, this will
> require the use of a `tie` or similar mechanism

Again, this is very vague. Do you actually mean perl's tie / magic
mechanism? If not, what "similar mechanism" are you proposing to use?

The problem with using actual ties, is that they make the variable very
slow, even if the actual tie methods such as FETCH() are XS code. That's
because a lot of code paths in the perl interpreter have optimisations for
non-magic values. For example, look at pp_add() in pp_hot.c in the perl
source. This implements perl's '+' operator. Obviously in general the two
operands can be anything - ints, ,nums, strings, magic values, overloaded
objects etc, and mixtures thereof. But in pp_add(), the first thing it
does it check the flags of the two operands, and says:

    If they are both non magical and non ref, and if both are either
    simple ints or simple nums, then just add the two values, check they
    haven't overflowed, set the return value and return. Otherwise go
    through the complex 200-lines-of-code path which handles magic, mixed
    types etc.

So if, for example, an integer variable was marked with get magic, it
would become a lot slower when being added. Similar considerations apply
in many places.

Also, once tie magic starts being applied to arrays and hashes, you start
to hit edges cases. It's very had to make a tied aggregate behave *exactly*
like a plain array/hash in all circumstances, even ignoring the slowdown.

> So, as mentioned in the original Elevator Pitch, we can "utilize
> Perl data types to achieve a number of benefits including but not
> limited to":
> * increased performance

So how would you get increased performance? I've already pointed out that
magic tends to slow things down, and the cost of calling a check routine
will slow things down even further.

> * memory safety (bounds checking)

Can you give an example of where currently perl isn't memory safe, but
Perl::Types would make it so?

> * potential for polymorphism

Please explain further.

Ok, that was the technical side of things. Now on to the policy side.
20 years or so ago, perl went through a phase of adding lots of 'useful'
modules to the perl core. This was widely seen as a mistake.

First, it made a particular module seem to be officially endorsed.
Hypothetically, "among several XML parser modules, we chosen this one as
the best and the one you should use". When design flaws in XML::Parser
(say) are discovered, and the general recommendation is to use
XML::BetterParser instead, XML::Parser is still sitting in the perl core
getting an unfair endorsement. And removing it from core is hard, because
now lots of people used it, and they're using it because, well, we
endorsed it!

Second, if modules stop being actively maintained by their author(s) for
whatever reason, then suddenly we become responsible for maintaining it.
It's enough work just maintaining the core perl interpreter, without
taking on extra responsibilities.

Third, in these days of package managers etc, it's relatively easy for
people to install the CPAN modules they want. They will also get the
newest and best version by default, rather than using the old buggy version
bundled with perl. So there is less good reason to bundle a package with

So these days the policy is not to bundle CPAN modules with perl unless
they are necessary, usually because they are needed as part of the tool
chain. There have been very few new modules added to the perl core in the
last 15 years or so that are capable of acting as standalone CPAN
distributions instead.

So the short answer is that even if Perl::Types really appealed to us, it
is very unlikely that we would agree to bundle it. And I think we're 
far from a consensus yet that it appeals.

On a purely individual note, I am finding Perl::Types very unappealing so

I took leave and went to hear Mrs Turner's daughter play on the harpsicon,
but Lord, it was enough to make any man sick to hear her; yet I was forced
to commend her highly.
    -- The Diary of Samuel Pepys, 1 May 1663 Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About