develooper Front page | perl.perl5.changes | Postings from October 2021

[Perl/perl5] 65b7cc: for CORE::my $var (...) {} is legal syntax,hence ...

From:
Nicholas Clark via perl5-changes
Date:
October 19, 2021 06:12
Subject:
[Perl/perl5] 65b7cc: for CORE::my $var (...) {} is legal syntax,hence ...
Message ID:
Perl/perl5/push/refs/heads/blead/7b1885-bd09be@github.com
  Branch: refs/heads/blead
  Home:   https://github.com/Perl/perl5
  Commit: 65b7cc064468ea58fd900e8293dfb0fdd1de8092
      https://github.com/Perl/perl5/commit/65b7cc064468ea58fd900e8293dfb0fdd1de8092
  Author: Nicholas Clark <nick@ccl4.org>
  Date:   2021-10-18 (Mon, 18 Oct 2021)

  Changed paths:
    M t/op/for.t

  Log Message:
  -----------
  for CORE::my $var (...) {} is legal syntax, hence test it

We had tests for CORE::state, but not CORE::my or CORE::our. It happens that
they are all legal syntax, but seemingly more be accident than design. They
are only accepted by yyl_foreach() as a quirk of the current implementation.

Note that for my Dog $spot (...) {} is legal, but not CORE::my. our Dog is
legal, CORE::our is not. Neither state Dog nor CORE::state are legal. These
are all emergent behaviour of the parser - do not take these tests as
"correct", merely as verifying that current behaviour doesn't change unless
we intended it to.


  Commit: bd09be5b1b65f68dd41433db47acee101aa69f7b
      https://github.com/Perl/perl5/commit/bd09be5b1b65f68dd41433db47acee101aa69f7b
  Author: Nicholas Clark <nick@ccl4.org>
  Date:   2021-10-19 (Tue, 19 Oct 2021)

  Changed paths:
    M t/comp/parser.t
    M t/comp/parser_run.t
    M t/op/for-many.t
    M toke.c

  Log Message:
  -----------
  `for my($k, $v) (%hash)` should not be a syntax error

No-one had thought to test for this explicitly. After all, both
`for my $foo ...` and `for my$foo ...` are valid syntax, so tweaking the
C lexer code to add an optional '(' should work, surely?

The problem was that, *as was*, the lexer code didn't "accept" those two
syntax variants the way the comments would suggest.

`for my $foo ...` was treated as

1) we saw 'my '
2) we saw the dollar sign
3) success!

but `for my$foo ...` was treated as

0) we didn't see 'my ' or 'our '
1) we saw the literal string 'my' which is valid as a package name
2) we saw the dollar sign
3) success!

ie some sort of mangled variant of `for my Dog $spot ...` without 'my'

*but* as the lexer was happy with what it saw, it returned that the input
stream was valid for a "for" token, and control continues to the grammar.
The grammar, of course, didn't make these mistakes, so parsed everything
properly and built the correct optree.

(And, if presented with `for Dog $spot (...)` the grammar wouldn't match,
so all invalid code was correctly rejected)

However, all this came unstuck with `for my($k` because that didn't
mis-tokenise as some crazy package name 'my(', so it reported a syntax error.

Hence rewrite yyl_foreach() to actually tokenise everything correctly.

"Correctly", to be clear, is bug-for-bug compatible with the current emergent
behaviour for various corner cases for "parses and runs" vs "syntax error".

We don't always report identical error messages for certain syntax errors,
where the precise message reported was itself emergent behaviour from the
bugs in the previous implementation.


Compare: https://github.com/Perl/perl5/compare/7b1885ae53fa...bd09be5b1b65



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About