Front page | perl.perl5.porters |
Postings from July 2012
Re: supporting untarring of extensions (was Re: Detecting duplicateextension directories)
Thread Previous
|
Thread Next
From:
Jesse Luehrs
Date:
July 11, 2012 12:11
Subject:
Re: supporting untarring of extensions (was Re: Detecting duplicateextension directories)
Message ID:
20120711191101.GW30375@tozt.net
On Fri, Jan 27, 2012 at 07:39:39PM +0000, Nicholas Clark wrote:
> I *think* I can see the genesis of a general solution
> to how to bootstrap arbitrary extensions:
I would really love to see this happen.
> Backstory:
>
> The build on *nix has never depended on having Perl installed.
> I assume because it originates from the days before Perl was commonly
> installed.
>
> But it has this interesting and I think useful side effect - bootstrapping
> to a new platform.
>
> Our cross compiling is about as functional as a chocolate teapot.
>
> *But* (I belive) all our build tools cross compile nicely. (sh, sed, awk,
> grep, make, cc). Hence one *can* bootstrap Perl 5 onto a new platform, albeit
> in a rather round about way, by first bootstrapping a native toolchain.
>
> So we aren't actually stuffed. I think it's important to keep this lack of
> dependency on perl 5 to build perl 5, at least as long as we are unable to
> cross compile.
>
>
> Building arbitrary extensions from tarballs:
>
> This is somewhat a de-lux feature. A value add.
>
> I see no reason why *it* can't rely on having a perl already installed
> in order to work.
>
> Moreover, the core ships a very nice new toolchain, so that installed perl
> doesn't actually need to have any modules installed.
>
> So, I think it might be possible take advantage of all the current build
> system as follows:
>
> 1) require that the tarballs are in a new directory, analogous to cpan/
> For want of a better name, and to keep BooK happy for now:
>
> 19:20 <+meta> Nicholas: crunch
>
> crunch/
>
> 2) in Configure, if it can detect an installed perl that is new enough to
> run the toolchain (5.8.1+ currently?), it then calls out to a bootstrap
> script written in Perl. (Much nicer than shell. And portable)
>
> 3) that is run with an -I options sufficient to use all the new, shipped
> toolchain modules (much like the list in write_buildcustomize.pl)
>
> 4) it uses whatever means CPAN can currently use to unpack tarballs.
> I think that they *could* be into crunch/ but it's probably clearer to the
> end user to use a new name:
>
> 19:22 <+meta> Nicholas: clank
>
> 5) everything is sufficiently set up now that the rest of the core build
> system will pick up the new modules. This should work for simple things
So by "arbitrary extensions" here, are you referring to things that
already exist as dual-life in the core, or entirely new modules to add
to a custom distribution? (or both?) My personal interest is in getting
as many dual-life modules as possible extracted out of the main build
process, because keeping things in sync with their CPAN versions is
quite a pain, due to the amount of changes that are necessary to get the
distribution to build in-core.
I think I can see how this would be possible - keep a very minimal
(doesn't use perl) build process available for building just perl and
the toolchain modules, to allow for bootstrapping. Then, just build and
install that "perl-minimal", and that will allow you to run the full
build process, which builds everything with perl via CPAN tarballs. Is
this what you are envisioning?
> Refinements:
>
> 6) I believe that which tests are run is determined by MANIFEST, not by
> scanning the filesystem. I think that this means that the extraction script
> should be writing out a new file containing pathnames of tests, and
> t/harness adapted to read from it too
>
> 7) dependency analysis can be done by the extraction script and whatever CPAN
> uses. requires and test_requires are fairly easy. build_requires is a bit
> more tricky
>
> a) I think that such modules need to be added to the list used by
> write_buildcustomize.pl. I suspect that the answer is to write out a
> file and teach it to read it, to avoid editing it in place
> b) I suspect that the modules need to be build ahead of their dependents.
> There is this section in Makefile.SH:
>
> : Prepare dependency lists for Makefile.
> dynamic_list=' '
> dynamic_ext_re="lib/auto/re/re.$dlext"
> extra_dep=''
> for f in $dynamic_ext; do
> : the dependency named here will never exist
> base=`echo "$f" | sed 's/.*\///'`
> this_target="lib/auto/$f/$base.$dlext"
> dynamic_list="$dynamic_list $this_target"
>
> : Parallel makes reveal that we have some interdependencies
> case $f in
> Math/BigInt/FastCalc|Devel/NYTProf) extra_dep="$extra_dep
> $this_target: $list_util_dep" ;;
> Unicode/Normalize) extra_dep="$extra_dep
> $this_target: uni.data" ;;
> esac
> done
>
> Again, I suspect that the right solution is to write out a file and
> teach Makefile.SH to suck it into the Makefile it writes, to avoid
> editing anything in place.
>
> 8) Everything is set up to assume nested Makefiles. I have this suspicion
> that to cater for modules that require Module::Build to build, the
> initial path of least resistance is to adapt make_ext.pl to write out a
> Makefile that has targets that call do ../../miniperl Build $target
>
> 9) The core by necessity builds extensions with miniperl not perl
> If extensions really need perl (ie dynamic loading) this might get fun.
> It's either going to be a lot of PERLLIB games (how make_ext.pl used to
> work) which risks busting the environment variable length limit
> *or* compile a third perl binary, a sort of hybrid between miniperl
> (so that it uses lib/buildcustomize.pl to set @INC) and perl
> (to give it DynaLoader.o and hence dynamic module loading)
>
> In which case, it makes sense to have the "extras" unpacked into clank/ -
> we
> i) don't do anything in clank/ until pretty much all of test_prep is made
> ii) we use hybridperl rather than miniperl (or perl) to build them
>
>
>
> The above, I think, should get us
>
> *) general purpose tarball extraction
> *) parallel make
> *) parallel testing
>
> for very little modification of the existing build bootstrapping.
>
> Which, if I'm right, is a win.
>
> Nicholas Clark
>
> PS No, I'm not proposing to do this. It's not my itch. But I'm hoping that
> it's a reasonable plan for scratching for those who have the itch.
-doy
Thread Previous
|
Thread Next