develooper Front page | perl.perl5.porters | Postings from August 2012

[perl #114102] unterminateable heredocs caused by newline in delimiter

Thread Previous
From:
Father Chrysostomos via RT
Date:
August 2, 2012 08:45
Subject:
[perl #114102] unterminateable heredocs caused by newline in delimiter
Message ID:
rt-3.6.HEAD-11172-1343922298-1708.114102-15-0@perl.org
On Thu Aug 02 02:19:38 2012, nicholas wrote:
> On Thu, Aug 02, 2012 at 12:20:19AM -0700, Father Chrysostomos via RT
> wrote:
> > On Mon Jul 16 03:04:46 2012, nicholas wrote:
> > > Yes, if it can't ever work, why is it even accepted?
> > >
> > > Strikes me that it's buggy to accept a terminator which contains a
> newline
> > > (or anything else which the parser *cannot* later deal with), and
> that
> > > really the only sane thing to do is to report it as an error
> early.
> > >
> > > I also *think* that changing this can't actually change the
> behaviour of
> > > any existing program, because (if I understand it correctly), all
> > currently
> > > fail to parse, due to the "missing" heredoc terminator.
> > >
> > > All it does is change the error reported, to one that is
> meaningful.
> >
> > It works in string eval.  See also #78348 and #114040.
> 
> OK, so *right now* if we change the code to reject a terminator
> containing
> a newline for not-a-string-eval we could at least give a more helpful
> error message of "not yet supported" ?

We could.  I’m not volunteering. :-)  But I’ll consider any patch that
comes along.

> 
> And your comment in #114040
> 
>     It seems to me that toke.c was written under the assumption that
> the
>     current buffer would only contain one line of code. And then
> string eval
>     came along, so we ended up with if(in_eval) sprinkled here and
> there. If
>     we could unify the code, we could avoid many discrepancies that
> result.
> 
> eval was added later:
> 
> commit a559c25918b1466cdb50c9f978a86f01be0bac10
> Author: Larry Wall <lwall@jpl-devvax.jpl.nasa.gov>
> Date:   Wed Jan 27 22:18:25 1988 +0000
> 
>     perl 1.0 patch 8: perl needed an eval operator and a symbolic
> debugger
> 
>     I didn't add an eval operator to the original perl because
>     I hadn't thought of any good uses for it.  Recently I thought
>     of some.  Along with creating the eval operator, this patch
>     introduces a symbolic debugger for perl scripts, which makes
>     use of eval to interpret some debugging commands.  Having eval
>     also lets me emulate awk's FOO=bar command line behavior with
>     a line such as the one a2p now inserts at the beginning of
>     translated scripts.
> 
> Do we have a meta ticket for discrepancies due to eval being parsed
> with
> all the source code available, versus files being parsed line by line?
> 
> Also, I'm not sure if we can fix all of them safely. By definition, we
> haven't
> exhausted memory if we get to running the string eval :-)
> (Something too big has already hit problems)
> 
> Whereas parsing a large file is potentially unbounded - when do we
> stop
> reading ahead into ever growing buffers?

When we run out of memory.  Yes, that does mean a missing " can cause
memory errors;

$ perl -e 'print q|"|; print " "x70, "\n" for 1..100000000' |perl
perl(40446) malloc: *** mmap(size=2397048832) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Out of memory!

but the (worse) alternative is to impose arbitrary restrictions in a
place where people require otherwise.  (I know that some code has 40K
strings embedded in it.  It would not surprise me if others have used
much larger strings.)

> To be fair, this case, a heredoc terminator, the problem is bounded.

Indeed.

$ perl -le 'print "<<", f x 254' | perl
Can't find string terminator
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
anywhere before EOF at - line 1.
Pint:perl.git-copy sprout$ perl -le 'print "<<", f x 255' | perl
Delimiter for here document is too long at - line 1.

-- 

Father Chrysostomos


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=114102

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About