Front page | perl.perl5.porters |
Postings from January 2017
The thing that pops the save stack seems to be the end of the qq{}
construct, as handled in toke.c. The yacc parser doesn't realise that
any construct is ending there, because it's busy discarding tokens (up
to the semicolon) to recover from the syntax error. So the lexer and
parser get out of synch.
This problem is partly due to the way the code generated by a qq gets
embedded in the parse. The lexer generates tokens corresponding to
some builtin function calls (to join() and a notional stringification
function), and passes through the tokens of embedded ${} and @{}
interpolations within that sequence. This works OK on correct code, but
it's fragile, and the present ticket is an illustration of the problems
that can arise. Fundamentally, delimiting the nested construct using
ordinary tokens doesn't work when it can contain arbitrary ordinary
tokens which may come in an ill-formed sequence.
In principle, a better way to handle the nested parses of qq constructs
would be to make a recursive call to the parser for the interpolations,
giving it a completely separate input stream. It would be impossible
to screw up the nesting in this case. Presumably this wasn't originally
done because the yacc structure does not lend itself to parsing different
top-level symbols. But that is a problem that I solved a few years ago,
to permit procedural calls into the parser from parse-time plugins.
Maybe it would be a good idea to reimplement qq constructs that way.
It's a bit too risky for this phase of the development cycle, though.
Incidentally, in debugging this I tried adding an assertion to
LEAVE_SCOPE(), asserting that the current stack height is at least at the
level the caller is requesting to pop to. It turns out that this isn't
a viable assertion; many exceptions have to be made. The parser's stack
clearing is heedless of the order in which it tries to leave scopes,
but there are dodgy scope-leaving calls from other places too. I was
surprised to discover this.
-zefram
Thread Previous