develooper Front page | perl.perl5.porters | Postings from April 2019

Re: [perl #134058] Bleadperl v5.29.9-123-gaabfeadc64 breaksETHER/signatures-0.13.tar.gz

Thread Previous | Thread Next
Karl Williamson
April 26, 2019 22:16
Re: [perl #134058] Bleadperl v5.29.9-123-gaabfeadc64 breaksETHER/signatures-0.13.tar.gz
Message ID:
On 4/26/19 3:33 PM, Dave Mitchell wrote:
> On Wed, Apr 24, 2019 at 10:26:44AM -0700, Karl Williamson via RT wrote:
>> If I comment out that assert, it thinks things are syntax errors, so it needs further investigation
> The proximate cause is that signatures.xs not clearing PL_lex_stuff at a
> certain point.
> This Proof of Concept fix passes all tests in blead:
> --- signatures.xs-	2019-04-26 21:53:04.430901853 +0100
> +++ signatures.xs	2019-04-26 22:07:32.304999812 +0100
> @@ -151,6 +151,9 @@
>   		return op;
>   	}
> +	assert(PL_lex_stuff == op_sv);
> +	PL_lex_stuff = NULL;
> +
>   	tmp = hook_toke_scan_str (aTHX_ s);
>   	tmp2 = hook_parser_get_lex_stuff (aTHX);
>   	hook_parser_clear_lex_stuff (aTHX);
> but I don't know whether it works with older perls, or why switching
> away from the 'stolen' scan_str() to perl's version broke things.

Thanks for doing this.  I was dreading having to work on this ticket, as 
I don't much understand the lexing process (I can deal with the low 
level, but not the grand view).

When run under older perls the stolen chunk is used, so it is lquite 
ikely to work, and if not, it's a simple matter to make that code 
dependent on the perl version.
> What is happening is that perl parses a sub prototype by using scan_str()
> to extract the balanced delimiter:  in thise case the '()'s in
>      sub foo ($x) { ... }
> scan_str() sets PL_lex_stuff to the extracted prototype, i.e. '$x'
> It then creates an OP_CONST whose op_sv is set to PL_lex_stuff, and then
> PL_lex_stuff is cleared, i.e. this code in yylex():
> 		if (have_proto) {
> 		    NEXTVAL_NEXTTOKE.opval =
>                          newSVOP(OP_CONST, 0, PL_lex_stuff);
> 		    PL_lex_stuff = NULL;
> 		    force_next(THING);
> 		}
> B-Hooks-Parser / signatures.xs hook into the ck_foo() mechanism, so that
> the hook is called for the OP_CONST in the newSVOP() above. The hook
> revisits the lexing stream that's just been processed to confirm whether
> it is of the form 'sub maybe_a_name (...)', using calls to
> hook_toke_skipspace() etc to extract and skip over the bits + pieces and
> to confirm that they form a sub declaration rather than anything else.
> At this point it calls scan_str() to skip over the prototype. Here it goes
> wrong, because PL_lex_stuff is still set, which causes scan_str() to think
> it's scanning the second part of a s/// or tr///, and so sets PL_lex_repl
> instead of PL_lex_stuff.
> So that diff above just ensures that PL_lex_stuff is NULL before calling
> scan_str() to skip over the prototype/signature again.

I believe this means this ticket is no longer a blocker.

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About