Front page | perl.perl6.internals.api.parser |
Postings from November 2000
Dan Sugalski wrote: > > At 04:16 PM 11/28/00 -0800, Steve Fink wrote: > >Perl5 is parseable with a single token of lookahead and lots of > >parser/lexer communication. Sort of. It would be nice to prevent it from > >getting any worse. > > I'm really thinking that the lexer, parser, and tokenizer can't be anywhere > near as separate as we'd like. I think we're going to end up with a rather > odd mutant beast. Hopefully one that's understandable by reasonably sane > people... With the PL_expect stuff and recursive calls in there, I'd hardly call them separable now. So I have no argument with what you said, but I do if you s/as we'd like/as it is now/. I think there are big advantages in keeping the parsing and tokenization (what's the difference between a lexer and a tokenizer, btw?) logically as separate as possible, but no more. It's a very useful separation for keeping things clean and understandable. It just has one minor drawback -- it doesn't actually work. Not for perl, at least, and not for a whole lot of other languages too. But weakening the separation enough to make things possible and hopefully relatively easy shouldn't have to mean rolling them together completely. On the other hand, I don't really mean I don't want it to get any closer together than it is now. We do need a higher bandwidth of communication between the two. Right now, it's pretty much a shiny clean byacc parser resting atop a writhing intestinal sea of tokenization code. The parser really ought to be shouldering more of the burden. Please don't think too much about the bizarre mixture of those metaphors.Thread Previous | Thread Next