develooper Front page | perl.perl6.users | Postings from September 2018

Re: Grammar doesn't seem to match any token

Thread Previous
From:
Brad Gilbert
Date:
September 24, 2018 13:40
Subject:
Re: Grammar doesn't seem to match any token
Message ID:
CAD2L-T3Fy8HsnJnHkE2-NP2Wd1TuoRUm3=4dF-qyD6K+w+idSw@mail.gmail.com
I recommend to default to using `token` rather than `rule` or `regex`.

If you need backtracking, use `regex`

If you have a lot of parts that match whitespace use `rule` (generally
used for combining other tokens.)
On Sun, Sep 23, 2018 at 7:13 PM Patrick R. Michaud <pmichaud@pobox.com> wrote:
>
> I suspect the rule:
>
>     rule other { .  }
>
> means that in
>
>     $input = '~i <<<YZ';
>     my $w = Weaver.new();
>     Weave.parse($input, :actions($w));
>
> the "other" rule cannot match the "Y" or the "Z" because there would need to be a space between them.  The use of "rule" as a regex declarator implies that spaces in the regex source are :sigspace, which means that whitespace in the rule prevents two word characters from being adjacent at that point and any available whitespace is consumed.
>
> Change "rule" to "token" on these rules and I suspect you'll get <other> to match (although <other> will also end up matching the space after the "i" in the text string, since white spaces are no longer significant).  Or try just changing the <other> rule to be a token and leave the others as rules.
>
> Phrased another way, the <other> rule as written now is roughly equivalent to writing
>
>    token other { . <!ww> \s* }
>
> which will match a word character only when it's not immediately followed by another word character.
>
> Pm
>
>
> On Sun, Sep 23, 2018 at 08:01:31PM -0400, yary wrote:
> > Let's see.
> >
> > If you have my $input = '~i o<<<', then <other> matches.
> >
> >  'rule' turns on :sigspace. If you use 'token' instead of 'rule' then
> > <other> matches.
> >
> > I don't quite have the full picture of what's happening.
> >
> > -y
> >
> > On Sun, Sep 23, 2018 at 7:07 PM, Mark Carter <alt.mcarter@gmail.com> wrote:
> >
> > > My grammar doesn't seem to match the 'other' rule. What's wrong with it?
> > >
> > > grammar Weave {
> > >         token TOP {  <el> * }
> > >         rule el {  <lt> | <tilde> | <other>  }
> > >         rule lt { '<'  }
> > >         rule tilde { '~' \S+ }
> > >         rule other { .  }
> > > }
> > >
> > > class Weaver {
> > >         has Str $.outstr;
> > >
> > >         method TOP   ($/) { make $<el> ; put("top called") ; put($<el>) }
> > >         method el    ($/) { put($/) }
> > >         method tilde ($/) { say 'tilde called' }
> > >         method lt    ($/) { make '&lt;' ; put('&lt'); $!outstr ~= 'X' }
> > >         method other ($/) { $!outstr ~= '.'; say 'other called'; put('.');
> > > }
> > >
> > > }
> > >
> > > $input = '~i <<<YZ';
> > > my $w = Weaver.new();
> > > Weave.parse($input, :actions($w));
> > > say $w.outstr; # outputs XXX
> > >
> > > It never once says 'other called'. It seems to be matching the '<' signs
> > > OK, and I think the '~' is OK, too. It's just any other token that it's not
> > > matching.
> > >

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About