[perl #16249] possible invalid syntax error generated

I have since learnt the lexer a little better and am convinced this
patch is correct, so I have applied it as 5dc13276b28.

(In fact, you’ll see that Larry fixed UNI the same way in commit a0d0e21ea6:

@@ -144,6 +105,7 @@ static I32	nexttoke = 0;
 	expect = XTERM, \
 	bufptr = s, \
 	last_uni = oldbufptr, \
+	last_lop_op = f, \
 	(*s == '(' || (s = skipspace(s), *s == '(') ? (int)FUNC1 : (int)UNIOP) )
 
 #define UNIBRACK(f) return(yylval.ival = f, \
)


On Sun Dec 11 16:29:57 2011, alh wrote:
> On Sun, Dec 11, 2011 at 7:23 PM, Father Chrysostomos via RT <
> perlbug-followup@perl.org> wrote:
> 
> > On Sun Dec 11 14:26:00 2011, alh wrote:
> > [...]
> > > The reason I'm not sure if this is the right fix or not (or maybe
it is
> > > and it's unfortunate that it is) is that it seems we rely too much on
> > > the coder having to know when to set/clear PL_last_lop and
> > PL_last_lop_op.

That’s why the macros take care of setting it.  It gets clear, as far as
I can tell, whenever the buffer is overwritten with the next line.

> > > These don't appear to be documented well so I'm not absolutely certain
> > > of their purpose, when they are used, etc etc,

PL_last_uni’s main purpose in life seems to be the ambiguity warning. 
PL_last_lop is used in various places to see whether the previous token
was a listop.

> > > (and it gets even more
> > > confusing with oldbufptr and oldoldbufptr...)

The local variable s is the current cursor, which is carried from one
invocation of yylex to the next via PL_bufptr.

PL_oldbufptr is set to PL_bufptr at the beginning of yylex, and
PL_oldoldbufptr to PL_oldbufptr (but not in that order).

So, in general PL_oldbufptr points to the beginning of the current
token, and PL_oldoldbufptr the previous token.

> > >
> > > If someone could shed some light on these, that'd be awesome.
> >
> > That looks correct to me, but I don’t think I understand this code any
> > better than you do.
> >
> 
> What really bugs me about this piece is:
> 
>                 /* See if it's the indirect object for a list operator. */
> 
>                if (PL_oldoldbufptr &&
>                    PL_oldoldbufptr < PL_bufptr &&
>                    (PL_oldoldbufptr == PL_last_lop
>                     || PL_oldoldbufptr == PL_last_uni) &&
>                    /* NO SKIPSPACE BEFORE HERE! */
>                    (PL_expect == XREF ||
>                     ((PL_opargs[PL_last_lop_op] >> OASHIFT)& 7) ==
> OA_FILEREF))
> 
> We enter this block if PL_oldoldbufptr == PL_last_lop or PL_last_uni, but
> then we still match if PL_opargs[PL_last_lop_op].
> 
> That just doesn't seem right...

The PL_last_uni bit seems suspicious.  But it seems to be necessary to
let things like readdir through.

-- 

Father Chrysostomos


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=16249

• [perl #16249] possible invalid syntax error generated by Father Chrysostomos via RT