develooper Front page | perl.perl5.porters | Postings from December 2001

Re: [ID 20011229.002] Syntax error in line with leading ," (comma and quote)

Thread Previous | Thread Next
Gurusamy Sarathy
December 29, 2001 20:10
Re: [ID 20011229.002] Syntax error in line with leading ," (comma and quote)
Message ID:
On Sat, 29 Dec 2001 22:11:16 GMT, Nicholas Clark wrote:
>nick@Bagpuss [build]$ ./bleadperl-intops0/perl /tmp/bug 
>syntax error at /tmp/bug line 468, near ".."
>Unrecognized character \xB7 at /tmp/bug line 1040.
>nick@Bagpuss [build]$ cat bleadperl-intops0/.patch 
>So it's a bug introduced somewhere between patch 7158(ish) and 7645.
>I have it down to a terser test case. The number of bytes appears to be
>significant. You can remove a few ###s but not many before it messes up.

I think the problem is caused by the skipspace() in the KEY_open case
in toke.c.  skipspace() can invalidate PL_linestr and therefore s,
PL_bufptr and their ilk.  A kludgy fix would be to turn that
C<t = skipspace(d)> into C<t = d; while (SPACE_OR_TAB(*t)) ++t;>. 

This problem is actually endemic to toke.c.  Every location in toke.c
that ends up calling skipspace() and doesn't update s and other local
variables that are pointing into PL_linestr is suspect in this regard.
There are several more places like this. :-(

The real solution to this is to get rid of the stupid line-by-line
lexing and do it by accumulating the entire source file contents (or
as much of it as needed at once) in one big buffer, more or less
like how it is done for the eval"" case.  This will have the pleasant
side effect of eliminating the egregious difference between how
eval"" and physical source files are lexed.  To make it work right,
you'll need to use perlio to treat a buffer as a file for handling
the DATA file handle (which currently doesn't work inside eval"" at
all--another one of those pleasant side effects).


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About