develooper Front page | perl.perl6.internals.api.parser | Postings from November 2000

Basic embedding [was: Re: The external interface for the parser piece]

Benjamin Stuhl
November 28, 2000 10:47
Basic embedding [was: Re: The external interface for the parser piece]
Message ID:

--- Steve Fink <> wrote:
> Dan Sugalski wrote:
> > 
> >    int perl6_parse(PerlInterp *interp,
> >                    void *source,
> >                    int flags,
> >                    void *extra_pointer);
> Given that other things may want to be streamable in
> similar fashion (eg
> the regular expression engine), why not have a
> PerlDataSource union or
> somesuch that encapsulates all of the possibilities of
> the final three
> arguments? Or put all possibilities into a PerlIO*? That
> gives direct
> support for compressed source, source streamed over a
> network socket,
> etc., with a more common framework than

Hear, hear! This is almost an embedding issue, though
(cc-ing perl6-internals-api-embed): How much of the
standard perl RTL is _required_ (I.e., PerlIO, perl malloc,
etc.). Offhand, I think that there is a very strong case to
require at least the basic PerlIO, since without it, perl6
can't count on having a non-bugridden I/O library, and also
can't take advantage of PerlIO's non-std. features
(whatever they end up being).
> Things like PERL_CHAR_SOURCE meaning nul-terminated char*
> sound
> unnecessarily specific.
> Also, you gave two options: nul-terminated and
> length-first. What about
> a "chunked" encoding, where you get multiple length-first
> chunks of the
> input (as in HTTP/1.1's Transfer-Encoding: chunked, for
> one example of
> many)? Or are nuls explicitly forbidden in source code?
> And, in a related question, the above interface appears
> that you call
> perl6_parse once. Will this be good enough, or do you
> want to have a
> PerlParseState* in/out parameter that allows restarting a
> parse once you
> get more of the input available? (With this, you don't
> need an explicit
> chunked encoding, since the caller can deal with that
> without being
> required to buffer the whole thing in memory before
> calling
> perl6_parse.) Or would that go into the PerlInterp too?
> And finally, how do I get the output out of the
> PerlInterp? Is it stored
> under some variable name, or does the PerlInterp start
> out empty and
> gains the parsed syntax tree as its only syntax tree, or
> ? (The latter
> sounds messy if the PerlInterp is also running code, code
> that wants to
> call some standard utility functions implemented in
> Perl.) Maybe I'm not
> making sense.

This sort of leads into an idea I've been having about what
defines an interpreter. I've sort of been musing on the
following embedding interface:

/* inits subsystems: PerlIO,memory,etc. call once at start
of program */
int perl_boot(); 

/* subsystem shutdown - call at program shutdown */
int perl_shutdown();

/* a perl6 interpreter - defines complete interpreter*/
typedef struct _perl_interp perl_interpreter;
typedef struct _perl_thread perl_thread;
struct _perl_interp {
   perl_thread *thread_list;
   perl_thread *root_thread; /* "top-level" thread - used
to parse the primary script (or provide an arbitrary
perl_thread for embedders) */
   HV *shared_stash;  /* subroutines are global to an
interpreter */
   HV *subroutine_stash;
/* a thread of execution in a perl_interpreter - contain's
thread's stash and stacks */
struct _perl_thread {
   perl_interpreter *threads_interp;
   OP *pc;
   SV *sp;
   HV *thread_stash;
   void *save_stack;
   RE_context *RE_data;
   perl_parser_state *parser;

/* creates an interpreter */
perl_interpreter * perl_create_interp(int flags);

/* ... ways of calling in (parse command line, call code,
etc.)... */

/* destroy and free an interpreter */
void perl_delete_interp(perl_interpreter *);

/* the embedder is expected to provide the following */
/* get this OS thread's current perl_thread */
perl_thread* perl_fetch_thread();
/* set this OS thread's current perl_thread (called in
Thread->new &co.) */
perl_thread* perl_set_thread();
/* get the perl_thread who will handle signals */
perl_thread* perl_get_sig_thread();

The idea behind the perl_interpreter/perl_thread separation
is that perl6 internal calls will actually pass a
perl_thread * around, since that is the basic unit of
execution, and if bytecode/optree is to be shared between
threads (as I devoutly hope it will be), there needs to be
something to aggregate a group of perl_threads. 

To go back to parser API design, I think that
perl6_parse_perl should take a perl_thread* to provide
context for sub {} declarations, parse errors, &co.
Top-level code would be treated as either the top-level
script, or an eval'', depending on the flags. 

-- BKS

Do You Yahoo!?
Yahoo! Shopping - Thousands of Stores. Millions of Products. Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About