develooper Front page | perl.perl6.language | Postings from September 2005

Unified prelude, FFI, multiple runtimes

Thread Next
From:
Yuval Kogman
Date:
September 12, 2005 03:16
Subject:
Unified prelude, FFI, multiple runtimes
Message ID:
20050912101533.GD15828@woobling.org
Hi,

please point your browser to

	"http://nothingmuch.woobling.org/compilation_of_circular_prelude."
	~ any(<graffle vdx pdf png jpg>).pick;

My proposition:

1.	*the* perl 6 compiler should ship a reference implementation of
	the prelude, that is circular. For example

		multi &*infix:<*> (Int $x, Int $y) { [+] $x xx $y }
	
	is the reference implementation of multiplication on integers.

2.	each block of code has a cryptographic digest, which is the hash
	of it's body with the digests of all the functions it calls.
	This digest is stable regardless of optimization semantics, and
	is applied to the PIL structure.

3.	a foreign function interface must exist, and have function
	somewhat like the Inline:: modules function today (presumably
	with a lower level but zero-copy interface an option).

4.	FFI definitions are defined using a uniform interface, whose
	most basic interface is a constructor for a code object that
	takes a runtime and a string body, e.g.

		Code::Foreign.new(:language<C>, :body("int foo () { return 10 }"));

	(Some FFIs might only allow construction of foreign functions at
	compile time.)

5.	Functions have a notion of equivelence. This is managed based on
	the digest. For example

		my &c_int_mul = BEGIN { Code::Foreign.new(:language<C>, :body("
			int foo (int x, int y) { return x * y }
		") };
	
		multi &infix:<*> (int $x, int $y --> int) {
			[+] $x xx $y;
		}

		my $str = &infix:<*><int, int --> int>.digest; # must specify the variant

		&c_int_mul.set_equiv($str); # could be in another file

		# or, if they are maintained together

		&c_int_mul.set_equiv(&infix:<*><int, int --> int>);

	This equivelence is with respect to the semantics of the input
	and output. The test suite supposedly can assure that these are
	really equivelent by running the same tests against either
	version.

6.	the runtime is just an FFI provider, which happens to be the
	default one too

7.	When applying code in the runtime, the runtime is free to use
	any equivelent function

8.	In order to run perl code at all, some native equivelent
	functions must be provided by the runtime, for the basic
	operations, and things that can't have reference implementations
	like IO calls (which are implemented in the prelude as stubs).

9.	static analysis may be leveraged to compile direct calls to
	native functions when compile time resolution is possible. In
	the example graph, for example, no eval or symbol table
	assignments are made, so there is no way the code will ever
	change. Hence the entire program can be pre-resolved. This
	should be controlled via the 'optimize' pragma.

the reasons for this will now be discussed.

A unified prelude with reference implementations for even the
simplest operations gives us several things:

	* a circular structure that doesn't make sense when we try to
	  run it, since the prelude depends on itself (bad)
	  	* must break circularity by going to native operations
	* a reference spec that alternative runtimes can compare to
	* a way to kickstart implementations
		* just provide a root set of native operation, enough to
		  break circularity in one single point
	* demagicalization of the language

Since FFIs are going to be a core feature of perl 6, they can be
used to bootstrap the whole compilation process. In effect, the
"primitive" operations are now just FFI calls to the runtime we
happen to be executing on.

To promote code reuse and to simplify the model the notion of
equivelence is introduced, letting the runtime pick which version of
a function (FFI or native) it uses.

To make things safe, when the prelude is bug fixed and the runtime
is not yet updated, the cryptographic hash of the function changed,
so it is no longer equal to the native one based on the way they are
paired.

To make things modular, the paring of FFI and pure perl functions is
orthogonal to their location of definition based on the hashing
scheme.

This has some nice properties:

	Modules like Template::Stash::XS are depracated. Instead, an FFI
	based Template::Stash can be automatically loaded from the
	runtime library prefix, and it will be set as equivalenet to the
	pure perl Template::Stash.

	Modules like DBD::your_db which rely on a certain library can be
	stubbed in Perl 6 to tell the user that they must use a certain
	runtime. The stubs are set as equal to the FFI calls into the
	library.

WRT MMD, you can set the entire MM equivalent to
a certain foreign function, and you can also set any variant
individually. You can even set a single variant to be equivalent to
a multimethod to make the FFI implementation simpler. The compiler
simply presents the runtime with all the possible MMD choices, and
lets the runtime choose between conflicting ones.

For example, in this version the runtime is told that it has two
choices:

	multi factorial (int $n) { $n * factorial($n - 1) }
	multi factorial (0) { 1 }

	my &c_fact = BEGIN {
		Code::Foreign.new(:language<C>, :body("
			int factorial (int n) {
				switch (n) {
					case 0:
						return 1;
					default
						return n * factorial(n - 1);
				}
			}
		")
	}

	# the C version is compatible with both behaviors
	&factorial.set_equiv(&c_fact); # all variants
	&c_fact.set_equiv(&factorial<int $n> | &factorial<0>); # or some variants

The runtime needs to choose &factorial<int $n> or &c_fact for one
variant, and &factorial<0> or &c_fact for the other variant. If it
chooses the same implementation for both, then MMD is taken out of
the picture entirely.

Note that this choice might be delayed to runtime if the MMD
choices cannot be statically determined, and might be re-made if
unsafe statical analysis was made (all variants could be determined,
but new ones may be introduced in runtime). This is highly dependant
on the capabilities of the runtime.

-- 
 ()  Yuval Kogman <nothingmuch@woobling.org> 0xEBD27418  perl hacker &
 /\  kung foo master: /me dodges cabbages like macalypse log N: neeyah!


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About