develooper Front page | perl.perl5.porters | Postings from August 2010

[PATCH] Damian will probably hate me

Thread Next
August 24, 2010 13:24
[PATCH] Damian will probably hate me
Message ID:
Attached patch fixes the mess around qw syntax.  "make regen regen_perly"
required.  Patch also available at git://
branch zefram/qw_strong.

As Damian points out in one of his talks, qw is currently parsed in a
very ugly way.  The lexer, on seeing a qw expression, generates three
tokens: a THING containing the list of strings, surrounded by parens.
This works fine within an expression, where the parens do what they
were intended to do.  But a side effect is that in some cases where the
grammar calls for parens outside an expression, a qw expression can supply
phantom parens.  Whether the lexer is in the right state to interpret a qw
this way is a bit random, and as this violates the lexer's expectations
about token sequences it's also a bit random whether the lexer is left
in a proper state afterwards.

For example, "foreach $_ qw(a b c) { print $_ }" will parse happily,
and print out three letters.  The lexer injects parens such that the
parser sees "foreach $_ (qw(a b c)) { print $_ }".  But "foreach qw(a b
c) { print $_ }" doesn't parse, because the lexer doesn't interpret qw
as the quoting keyword in that situation.  And while "foreach $_ qw(a b
c) { print }" does parse, it doesn't print anything, because the lexer
interpreted "{ print }" as a hash constructor rather than a code block.

I propose to fix this by removing the phantom parens.  The main effect
of this patch is to fix the way qw expressions are parsed, so that a
qw list becomes a single token, of a new type, that the parser accepts
in the appropriate places in the expression grammar.  In the long run
I think that's the right way to go.

Some people have suggested that rather than eliminating the unintended
uses of qw we should make them work and make them a deliberate part of
the grammar, along the lines of the Perl 6 things that don't require
parens where Perl 5 requires them.  Those who think this is a good idea
are welcome to try.  It won't be a happy fit.

Of course, because it sometimes works, some real code does "foreach $k
qw(...)", and we mustn't break that without warning.  So this patch
also contains a backcompat hack.  Anywhere that the grammar calls
for a paren outside an expression, it will also accept a qw token.
When it does this, it generates the phantom parens as before, and emits
a deprecation warning.  So the phantom parens are left in exactly the
partially-functional state they were in before, except for now generating
warnings.  We get to remove the hack in 5.15, leaving completely clean
grammar for qw.

A couple of the new deprecation warnings are generated by code in the
core distro.  One in ExtUtils::MakeMaker has now been preemptively fixed
(thanks to Schwern and BinGOs).  Some are fixed by this patch.  There
remain warnings from B-Deparse and Pod-Simple, for which CPAN is upstream.


Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About