develooper Front page | perl.perl5.porters | Postings from February 2017

Re: Moving XS-APItest in harness testing

Thread Previous
James E Keenan
February 19, 2017 14:12
Re: Moving XS-APItest in harness testing
Message ID:
On 02/19/2017 12:20 AM, Karl Williamson wrote:
> Is there any reason not to move these tests away from the cpan, dist,
> ext tests? and into the tests for the core?
> I have a smoke-me that does this, if you'd like to see how it impacts
> your testing performance.

Smoke test results for this branch appear here:

The git-id for this branch is:

The git-id for blead at the point where Karl cut the branch is:

There were source code changes between these commits other than Karl's 
changes to the testing code.  So if you then go to the individual smoke 
reports for this pair of commits when run on the same machine, you can 
get a rough estimate of any shortening of smoking time.

 > This two line change improved my elapsed 'make test' time by 9% on an 8
 > effective core system.

In my FreeBSD-11.0 VM, I got this:

blead -
2 hours 12 minutes (average 33 minutes 6 seconds)

branch -
1 hour 56 minutes (average 29 minutes 4 seconds)

Note that this includes configuration and build time, not just 
test/test_harness time.

I saw similar improvements on the other two smoke test runs on the branch.

> Presumably the reason we do base first is because if it's failing,
> nothing is going to work.  Then we do the other core tests, in parallel.
>  Then we wait for those to finish before starting up the remainder of
> the tests.  Those are done in alphabetical order in a given test
> directory, presumably because cpan tests like to have a specified order,
> and have named their tests accordingly.  But we do the longest-running
> directories first.
> But the API tests for XS code are not even dependent on base working.
> But since base is fast, there's no real gain in doing them parallel with
> base.  But there is no reason I can see, except tradition, not to do
> them in parallel with the rest of the core tests.

I agree with Karl's reasoning here.  Could we get some more eyeballs on 
the patch he's already written so that we have the best possible 

> I also think there should be a way to tell harness to run cpan etc in
> parallel with core.  A lot of time seems to be wasted waiting for all of
> core to finish before these start.  On many-core systems like dromedary,
> this is a significant fraction of the elapsed time.  One wouldn't want
> this to be the default, but I think an expert-level option is called for.

I'm not opposed to this idea, but I think it may take some time to 
develop a good patch.

Thank you very much.
Jim Keenan

Thread Previous Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About