develooper Front page | perl.perl5.porters | Postings from January 2022

Re: arity checking (was Re: PSC #049 2022-01-07)

Thread Previous | Thread Next
Nicholas Clark
January 24, 2022 10:09
Re: arity checking (was Re: PSC #049 2022-01-07)
Message ID:
So, up front, I'm not sure if I should write this message because I'm not sure
how much value I'm adding, and I failed to write a short message...

On Fri, Jan 21, 2022 at 05:49:40PM +0000, Dave Mitchell wrote:

> Now you can of course argue that signatures make it *easier* to
> inadvertently write subroutines which have arity checking when with
> hindsight it would have been easier to migrate to a new version of the
> framework if you had disabled arity checking. But its not a fundamental
> design problem with signatures.

I agree. With caveats.

It's that comment I made previously:

    There is only one perl core, so it can only have one default.
    Whereas different use cases often benefit from different defaults.

because in the case of signatures:

On Fri, Jan 21, 2022 at 12:33:49PM -0500, Dan Book wrote:
> On Fri, Jan 21, 2022 at 11:26 AM Dave Mitchell <> wrote:

> > But this is a general problem, not just one with signatured subs. A
> > "normal" callback sub could be written as:
> >
> >     sub callback {
> >         croak unless @_ == 2;
> >         (my $self, $foo) = @_;
> >         ...
> >     }
> >
> > How would the caller (say an event handler module), which has been upgraded
> > to pass an optional next arg, know that the callback sub it wanted to call
> > would croak on that extra arg?
> >
> The difference is a practical one: no one would pass a callback like your
> example because it's asking for breakage and adds nothing, but most would
> use a signature like ($self, $foo) in a callback documented to accept those
> arguments.

I'm assuming (and I think that I'm correct in assuming) that most folks
writing code will see this:

sub foo {
    my ($bar, $baz) = @_;

and think that they can rewrite it as

sub foo ($bar, $baz) {

when (of course) the latter approach creates the strict arity checking.
*And* for nearly cases, the strict arity checking is the right default.

So this combined with how callbacks typically evolve is creating a trap.
And this isn't a "with hindsight, we ..." trap. It's foreseeable.

On Fri, Jan 21, 2022 at 04:26:18PM +0000, Dave Mitchell wrote:

> I think in the past that strong opinions have been expressed against
> allowing a signature API which a caller can use to inspect a sub, on the
> grounds that really the sub signature is an implementation detail - just a
> fancy and fast way of writing a @_ unpacker, and in general, doesn't form
> a promised API.

I don't remember *this* specifically. (I might have missed it)

I *do* remember a desire to inspect signatures as strings, and even treat
them roughly as such, and pushback that signatures are *code*, not flat
strings, and so can't be handed round as strings. But the objection I
remember (and agreed with) was the "strings", not fundamentally the

I think it important to say that, as signatures can contain arbitrary code
(and I think it important that they can, for enabling setting defaults
without trying to instead create a Domain Specific Language that morphs into
a *different* Turing complete programming language) then they can't be 100%
introspectable. And then that obviously means that there's going to be some
level of discussion on where are the "rigidly defined areas of doubt an
uncertainty". In that:

On Sat, Jan 22, 2022 at 02:20:21AM +0800, Tom Molesworth via perl5-porters wrote:

> I ended up writing
> as a proof-of-concept API, but would hope that core Perl will expose some
> sort of arity inspection in due course. Ignoring this library-extensibility
> use-case while simultaneously promoting signatures as a complete
> replacement for @_-based subs seems less than ideal to me.

Aside - I didn't think that we *were* promoting signatures as a complete
replacement for @_-based subs. I always thought of signatures as the "easy
things easy" part, where @_ remains for "hard things possible". In that, I'm
predicting that the syntax of signatures will get too complex to be easily
readable, if it's extended to cover all rarely used things that one can do
with @_.

Thanks for the module. This is useful because you chose a different line for
"doubt" than I was thinking, and that's useful. In that, your API can
distinguish between these two:

    sub foo($bar, @args) { my ($baz) = @args; say "$bar: $baz" }

    sub foo(@args) { my ($bar, $baz) = @args; say "$bar: $baz" }

and these two:

    sub foo($bar, $baz = "world", ,@) { say "$bar: $baz" }

    sub foo($bar, @args) { my $baz = shift @args // "world"; say "$bar: $baz" }

whereas I thought that I'd view these choices as "implementation details",
and so I'd limit things to roughly:

1) min count of positional arguments
2) max count of positional arguments (might be infinite)

and not expose

1) where the named arguments stop
2) whether "infinite" is a named array, or just discarded

but then...

I realised that introspecting "does it end with a slurpy hash" gets sort of
interesting too, because that means that not all large parameter counts are
valid. The simple and wrong answer would be `$m + 2 * $n`. But optional
positional parameters followed by a slurpy hash...

./perl -Ilib -wE 'use feature "signatures"; sub foo($bar, $baz = "world", ,%) { say "$bar: $baz" } foo("Mandatory", "Optional", "Whoops!")'
The signatures feature is experimental at -e line 1.
Odd name/value argument for subroutine 'main::foo' at -e line 1


1) I don't know exactly what is the published interface, and what is
2) And this is only positional parameters. It's not future-proofed for also
   describing named parameters (mandatory and optional. eg does one even
   need to "announce" the names of optional named parameters? Is that
   implementation details? Do I want "your" code to object to my callback
   because I have an optional named parameter you don't use, but you wrote
   your validator to reject any sub reference with non-zero optional named

On Fri, Jan 21, 2022 at 03:15:31PM +0100, demerphq wrote:
> On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <> wrote:
> > Really you want to know you have a problem *before* you call the callback
> > (ie what Yves wrote - you want to be able to flag a problem at object
> > construction time, which might well be program startup. Not fail at
> > runtime,
> > unattended, into a log file or even /dev/null)
> >
> So am i correct in understanding that you would be ok with exposing an API
> to check the subs arity signature? If so then I'll try to think up
> one (others could too).

Yes, I'd be OK with it. But as above

1) I don't know what it should expose
2) and I don't think I (alone) get to say "we" want this
3) and I really can't promise to be around in a timely fashion to resolve
   design questions or review anything (which makes me feel bad about saying
   "yes", because I feel that "yes" implies "and I have time to stay involved")

On Fri, Jan 21, 2022 at 06:28:19PM +0000, Paul "LeoNerd" Evans wrote:

> What would be lovely would be if we could encode a position in a
> function call site, to say "from this point onwards, don't worry if the
> invoked sub does not handle this argument. They are optional". I wonder
> about allowing the keyword `optional` in the middle of a function or
> method call argument list:
>   $self->$callback($item, optional $idx);
> A function/method call with one of these 'optional' words inside it is
> then handled slightly differently. If the invoked sub fails to handle
> at least all of the arguments before the 'optional' then that's fatal
> as it is now, but the caller is saying "I don't mind if the callee
> ignores any of the values after here". Thus $callback could be defined
> by any of

It bugs me that implementing this efficiently would be a right pain.

Currently a signature sub looks like this:

$ ./perl -Ilib -MO=Concise,foo,-exec -wE 'use feature "signatures"; sub foo($bar, $baz = "world", ,%) { say "$bar: $baz" }'
The signatures feature is experimental at -e line 1.
1  <;> nextstate(main 6 -e:1) v:%,us,fea=15
2  <+> argcheck(2,1,%) v
3  <;> nextstate(main 4 -e:1) v:%,us,fea=15
4  <+> argelem(0)[$bar:4,6] v/SV
5  <;> nextstate(main 5 -e:1) v:%,us,fea=15
6  <|> argdefelem(other->7)[1] sK
7      <$> const(PV "world") s
8  <+> argelem(1)[$baz:5,6] vKS/SV
9  <;> nextstate(main 6 -e:1) v:%,us,fea=15
a  <0> padrange[$bar:4,6; $baz:5,6] /range=2
b  <+> multiconcat(": ",-1,2,-1)[t5] sK/STRINGIFY
c  <@> say sK
d  <1> leavesub[1 ref] K/REFC,1
-e syntax OK

Dave suggests that for common cases all of that argument setup can be reduced
to a single op (with data for a finite state machine). Maybe not with my
default value shown. But generally, I guess we get to this:

1  <;> setup[arm waving]
2  <;> nextstate(main 6 -e:1) v:%,us,fea=15
3  <0> padrange[$bar:4,6; $baz:5,6] /range=2
4  <+> multiconcat(": ",-1,2,-1)[t5] sK/STRINGIFY
5  <@> say sK
6  <1> leavesub[1 ref] K/REFC,1
-e syntax OK

as the optree of the subroutine. And nothing in pp_entersub *or* the optrees
of the calling code needs to know the difference between the two, and always
passes flat arguments just like perl 5.000 to blead.

Hence the implementation is free to swap to the latter whenever it needs to.
But more importantly, that "setup" OP only takes the list of arguments.  It
doesn't care what the *callee* is doing, so the only logic it needs to
encode is the (known at compile time)(single) logic of the signature author.

Whereas if the *callee* is also allowed to influence what is optional, then

1) a side channel has to exist for the *callee* to pass that onward too
2) the signature processing code must handle it (more branches

Yes, one *could* them optimise this by having two optree entry points for
each CV - one with no callee extra info (fast path) and one for with,
both of which have ->op_next land at the first (real) nextstate op of the
body, but this is (also) getting more complex.

> Now I'm not hugely fixed on this particular syntax as a solution to the
> problem, but I feel that the principle of "caller specifies API" is one
> we should explore.


I like the idea of a (limited) signature introspection API better as a
solution to the problem.

Nicholas Clark

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About