develooper Front page | perl.perl5.porters | Postings from February 2013

Re: Parallel testing?

Thread Previous | Thread Next
From:
H.Merijn Brand
Date:
February 6, 2013 16:38
Subject:
Re: Parallel testing?
Message ID:
20130206173817.78531478@pc09.procura.nl
On Wed, 6 Feb 2013 05:19:33 +0100, Leon Timmermans <fawaka@gmail.com>
wrote:

> On Wed, Feb 6, 2013 at 5:03 AM, Peter Martini <petercmartini@gmail.com> wrote:
> > So, I was running 'make -j 32 test' on my new 32-thread Solaris box,
> > and found that the test suite can't take advantage of the parallelism.
> >
> > Are there any known issues that prevent that, or is it simply a matter
> > of round tuits?
> >
> > I'm slowly setting that machine up as a smoker, and being able to burn
> > through the tests on all 32 threads would make it oh-so-much sweeter.
> 
> The parallelization can't be done on a make level, but Test::Harness
> does support it. Ā«TEST_JOBS=32 make test_harnessĀ» should do the trick
> ;-)

And this is documented in INSTALL (since Jun 2009):
--8<---
The core distribution can now run its regression tests in parallel on
Unix-like platforms. Instead of running C<make test>, set C<TEST_JOBS> in
your environment to the number of tests to run in parallel, and run
C<make test_harness>. On a Bourne-like shell, this can be done as

    TEST_JOBS=3 make test_harness  # Run 3 tests in parallel

An environment variable is used, rather than parallel make itself, because
L<TAP::Harness> needs to be able to schedule individual non-conflicting test
scripts itself, and there is no standard interface to C<make> utilities to
interact with their job schedulers.
-->8---

and in the perlhack docs (since Sep 2009):
--8<---
   Parallel tests
       The core distribution can now run its regression tests in parallel on
       Unix-like platforms. Instead of running "make test", set "TEST_JOBS" in
       your environment to the number of tests to run in parallel, and run
       "make test_harness". On a Bourne-like shell, this can be done as

           TEST_JOBS=3 make test_harness  # Run 3 tests in parallel

       An environment variable is used, rather than parallel make itself,
       because TAP::Harness needs to be able to schedule individual non-
       conflicting test scripts itself, and there is no standard interface to
       "make" utilities to interact with their job schedulers.

       Note that currently some test scripts may fail when run in parallel
       (most notably ext/IO/t/io_dir.t). If necessary, run just the failing
       scripts again sequentially and see if the failures go away.
-->8---

Which makes me wonder, can (GNU) make "see" how much -j is passed and
(auto)convert that to TEST_JOBS on commands like

 $ make -j32 test_harness

> Leon


-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.17   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About