develooper Front page | perl.perl5.porters | Postings from February 2013

Re: Parallel testing?

Thread Previous
Nicholas Clark
February 6, 2013 21:12
Re: Parallel testing?
Message ID:
On Wed, Feb 06, 2013 at 05:38:17PM +0100, H.Merijn Brand wrote:

> Which makes me wonder, can (GNU) make "see" how much -j is passed and
> (auto)convert that to TEST_JOBS on commands like
>  $ make -j32 test_harness

I've no idea, but I doubt it. As I understand it (forget where the docs are
online), the architecture has a job server, spawned from the top level make,
which knows how many jobs there are, and every child make merely knows that
it has to ask for a "token" to be able to run another job. This implies that
most of the system doesn't even have any idea how many jobs there are.

I *did* think recently, and this is an arm-wavy "not tried" idea, that it
might be possible to fake up a way to get parallelism from make without
needing an environment variable. It relies on/assumes

1) there's a makefile syntax for "I'm really a submake command", which causes
   the file descriptors, environment and whatever to persist, such that a
   make invoked from a child process will see them
2) that the TAP parser is sufficiently lax that it will ignore the banner
   that make prints out
3) that you can tell the TAP::Harness runner to run `make some_test_target`
   instead of `./perl some/test.t`

In that, it might be possible

1) to write out a massive makefile, with a rule for each test, so that you
   invoke `$ENV{MAKE} ... some_test_target` instead of `perl some/test.t`
   to run the individual scripts
2) to have the parallel test runner *try* to do this massively in parallel
3) but the make command run actually blocks on the job server until a
   token is available for it to run the test

Of course, each test's output now has any extra clutter added by the make
program, so it does rely on the TAP parser being lax.

I'm not *sure* if this will work. In particular, I have this niggling doubt
that it quite works with the way the job server allocates tokens. In that I
*think* that a running make implicitly has one token (else it was not
allowed to start). And it can use that token to run one job (serially).
Whereas this plan requires that make can run, but has to get a token to run
the test itself.

Anyway, that's a brain dump. I have no plans to work on this, as it I'm not
confident that it will even work, and it seems silly to spend a lot of time
on something that is an incremental improvement on something that already
works well enough, when a bunch more things are more pressingly broken.

(Getting parallel testing on Win32 would be far more useful. I believe that
this involves arranging for a proxy process, which sets up a pipe between
it and the test to be run, but pumps data onwards to the harness on a
socketpair, and does the Win32 dance to properly close a socket. As by
default neither pipes nor sockets are useful. You can't select on pipes,
and on Win32 sockets dump data on the floor if you close them. Most useful.)

Nicholas Clark

Thread Previous Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About