develooper Front page | perl.perl5.porters | Postings from August 2008

Re: make tests parallel

Thread Previous | Thread Next
From:
Nicholas Clark
Date:
August 6, 2008 02:31
Subject:
Re: make tests parallel
Message ID:
20080806093120.GC40282@plum.flirble.org
On Thu, Jul 31, 2008 at 09:23:22PM -0700, Eric Wilhelm wrote:
> # from Nicholas Clark
> # on Thursday 31 July 2008 14:39:
> 
> >I believe that Andy Armstrong has a pretty functional prototype
> >that will run discrete directories of tests in parallel, which is a
> > good first step, so you wouldn't be starting from scratch.
> 
> I believe you mean running the contents of a directory in serial.  I had 
> a fair bit of luck doing that with `runtests` "back in the day", but 
> IIRC, I had to put a silly hack in t/harness to get the list of tests 
> out of there.

Yes, I did. Well spotted.

>   http://scratchcomputing.com/svn/TAP-Harness-Parallel/trunk
> 
> >* figure out the best way to invoke parallel tests from the Makefile
> 
>   make test TEST_JOBS=9
> 
> At least, that's what I used in parrot's harness the other day.  One 
> could (and some may) argue that the environment variable should be more 
> specifically named, but I don't.

I had hoped that it would be possible to make a stab at parsing the command
line options to make to try to automatically run the same number of tests
as make jobs, but, for GNU make:

$ make -j 6 foo
perl -e 'print "$_\n" foreach @ARGV' --  --jobserver-fds=3,4 -j
--jobserver-fds=3,4
-j
$ make -j 3 foo
perl -e 'print "$_\n" foreach @ARGV' --  --jobserver-fds=3,4 -j
--jobserver-fds=3,4
-j


which I guess makes sense - rather than having subordinate makes also run
6 (or 3) jobs, it ensures that the total number of jobs is no more than the
value given to the top level command line -j flag.

Nicholas Clark

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About