develooper Front page | perl.perl5.porters | Postings from August 2016

Re: make_ext.pl run every darn time

Thread Previous
From:
Nicholas Clark
Date:
August 8, 2016 17:27
Subject:
Re: make_ext.pl run every darn time
Message ID:
20160808172656.GA1987@ceres.etla.org
On Mon, Aug 08, 2016 at 11:54:34AM -0500, Craig A. Berry wrote:
> On Mon, Aug 8, 2016 at 7:39 AM, Jarkko Hietaniemi <jhi@iki.fi> wrote:

> > Not suggesting that.  Saying that what we have now could be better.
> >
> > For example, does a new miniperl need to be started 101 times?
> 
> 
> Ah, I see what you're talking about.  In Makefile.SH it runs
> make_ext.pl for each and every extension instead of running it once
> for each category of extension.  It looks like that change came along
> here:
> 
> http://perl5.git.perl.org/perl.git/commit/07f3cc2a611daebb9d83f869d36ef63cf4fa2565?f=Makefile.SH
> 
> On VMS we don't do it that way but instead pass the --dynamic,
> --static, and/or --nonxs flags to make_ext.pl and thus it's only run a
> total of 3 times.
> 
> So it looks like figuring out whether the current version of
> make_ext.pl handles LINKTYPE correctly would be the only impediment
> reducing the number of invocations.

That would be counter-productive (at least for "XS extensions") because it
would destroy build parallelism on *nix. (Which has to be handled at the
level of the Makefile, because GNU make, BSD make etc purposefully don't
expose *how* they track how many jobs are currently running. So your code
can't find out that $m of $n_max jobs are currently running, so it's fine to
do $n_max - $m more in parallel.)

There are also some dependency rules between extensions that assume the
current targets, but I think these are only for a couple of XS extensions.

It might be a win for non-XS extensions, particularly as it's many of those
that have make_ext.pl doing Makefile.PL/ExtUtils::MakeMaker emulation to
avoid (even more) shelling out/onwards. So this approach might eliminate
(arm wave) 50 or so invocations of miniperl.

There are a lot more non-XS extensions, and most of them are trivial to
"build". I suspect that it's worth trying for them.

(I also don't think that there's a way of saying "use these make targets if
we're a parallel build, or those targets if we're not". Otherwise that would
also seem like a viable approach (modulo dependencies between XS modules) -
parallel builds keep the current hammer, non-parallel builds go for the
approach that is as frugal as possible)

Nicholas Clark

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About