develooper Front page | perl.perl6.internals.api.parser | Postings from December 2000

Re: Now, to try again...

Thread Previous | Thread Next
Dan Sugalski
December 17, 2000 12:21
Re: Now, to try again...
Message ID:
At 02:08 PM 12/16/00 +0000, David Grove wrote:
>Ok, _from_ the books on the reading list, I'm seeing no precedent for a
>parser/lexer/tokenizer that uses multiple input "languages". Yes I know
>that GCC does F77/ASM/C/C++ but I'm not sure those completely relate.
>Simon (?) brought up the problem that we might end up with a monolithic
>beastie, and my concern is simplicity for the (advanced) user to come up
>with syntax variants.

Well, that's not really the case. GCC (and Compaq's GEM compiler suite) do 
*not* have parsers that eat multiple languages. What they have is custom 
parsers for their particular language that emits a standard intermediate 
language. The bits of the compiler that handle the intermediate language 
representation is what's common between compilers. If a picture helps, try:

| C parser |\
+----------+ \   +-----------+  +----------+
              +---+ optimizer +--+ code gen +---> object files
+---------+  /   +-----------+  +----------+
| Fortran | /
|  parser |/

>Reading what you say, "parser/lexer/tokenizer" (multiple things) "part"
>(one thing). That's got to be a stumbling block of some kind.

Why? In some ways it seems like a win, since when things are that tightly 
coupled there's more communication between the bits. Of course, it makes 
things bigger, and that can make your head hurt trying to take it all in.

>If we
>separate out the "parser" (or look at each of them individually - which
>might further help clarity), we have something that might be "plug and
>pl...", er..., "plug and code". But then, if we're using flex/bison
>(lexx/yacc), that doesn't sufficiently satisfy what I'm looking for in
>end-user usability.

We're not going to, I think.

>If joe hacker-wannabe wants to program in Java, he
>shouldn't have to write flex spex, mess with gobs of perlguts, or,
>hopefully, go outside the Perl language.

If they want Java, it means that either someone very clever will need to 
write a Java parser, or they'll use Java. Either one's fine with me. J. 
Random Programmer will *not* be writing language parsers. Most folks are 
just not up for (or to) that.

>I see this as leaving two
>options: having a perl layer above the parser, or replacing the parser
>with perl.
>If I understand the latter and our goals, this is where we're going,
>although I'll still voice that I think a simpler Perl abstraction on top
>of it all that a user can define as easily as a good non-compiled module
>would be highly beneficial. It would give us a single parser that we plug
>No semantics, guys, please. I'm trying to figure out how we're going to
>allow multiple input "languages" with a single, small parser.

We're not. We're going to do it with source filters and a large parser. 
Possibly multiple ones, though we likely won't be writing most of 'em.

>Maybe not a
>source filter so to speak, but a layer above what is normal for this
>collection of operations. My concern is to give users the ability to
>create innovation within Perl without being perl masters.

People are always bounded by their own limits and the tools available, and 
there's a limit to what your tools will do for you. (And most of what 
you're thinking about is probably not going to be done by most people, and 
that's OK)

I don't want us to get too hung up about all this--if it turns out that 
providing a general-purpose extended language parser just doesn't come 
easily, then we can put it off for perl 6.2 or later. As long as it's 
spec'd out, we can add it in when we get time. Having the spec means we can 
take the future into account during the present.


--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai                         have teddy bears and even
                                      teddy bears get drunk

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About