develooper Front page | perl.perl5.porters | Postings from June 2015

[perl #123867] [PATCH] WIP add parallel support for Win32 dmake building

From:
bulk88 via RT
Date:
June 24, 2015 00:01
Subject:
[perl #123867] [PATCH] WIP add parallel support for Win32 dmake building
Message ID:
rt-4.0.18-17428-1435104103-1185.123867-15-0@perl.org
After sorting the .c'es by time (I am only moving the >1 second ones), here are the results on my 8 core with "dmake -P8 ../perl523.dll".

VC with no -GL
32 seconds time sort
32 seconds time sort
32 seconds time sort
34 seconds no time sort
35 seconds no time sort
35 seconds no time sort

GCC
49 seconds time sort
48 seconds time sort
48 seconds time sort
56 seconds not time sort
56 seconds not time sort
56 seconds not time sort

So time sorting the .c files has very visible results. The order I decided is

MICROCORE_SRC	=		\
		..\toke.c	\
		..\regcomp.c	\
		..\regexec.c	\
		..\op.c		\
		..\sv.c		\
		..\pp.c		\
		..\pp_ctl.c	\
		..\pp_sys.c	\
		..\pp_pack.c	\
		..\pp_hot.c	\
		..\gv.c		\
		..\perl.c	\
		..\utf8.c	\
		..\dump.c	\
		..\hv.c		\
		..\av.c		\
		..\caretx.c	\
		..\deb.c	\
		..\doio.c	\
		..\doop.c	\
		..\globals.c	\
		..\mro_core.c	\
		..\locale.c	\
		..\keywords.c	\
		..\mathoms.c    \
		..\mg.c		\
		..\numeric.c	\
		..\pad.c	\
		..\perlapi.c	\
		..\perly.c	\
		..\pp_sort.c	\
		..\reentr.c	\
		..\run.c	\
		..\scope.c	\
		..\taint.c	\
		..\universal.c	\
		..\util.c

utf8.o took 1.7 seconds to create as from the earlier list. At 1.6 seconds is win32.o, but I'd have break up the organization and meaning of MICROCORE_SRC macro to put win32.c in the middle of it (win32.c is in a different macro called WIN32_SRC). These numbers also mean, that the MAXIMUM penalty, and therefore best case scenario improvement, for not time sorting every last .c file, is 1.6 seconds. DynaLoader and Extensions_static deps take 4.0 and 2.2 seconds to build, and they are currently scheduled at the end due to a poor dmake parallel algo, after all .c files, so the 1.6 sec penalty is dwarfed by poor scheduling of DynaLoader and Extensions_static targets. I will not reorganize putting DynaLoader and Extensions_static first before core .c files are started since I have to make the core .c files depend on fake dependencies for them to have the same deps as DynaLoader and Extensions_static (core .c files dont need /lib/CORE headers, XS/EUMM makefiles do) and I'd rather w
 ork on the gmake makefile and see if gmake has the same poor scheduling as dmake before I try such things. Also putting DynaLoader and Extensions_static first only speeds up reaching ../perl523.dll target, reaching test/test-prep target means running Extensions which takes many minutes and makes the wall time to build ../perl523.dll on a multi core machine irrelevant since Extensions is always the last target to finish executing.

I've attached the psuedo-branch which is the final version of the parallel patch.  The psuedobranch can be benchmarked for the effect of alpha sorting between commit "WIP add parallel support for Win32 dmake building
" and "coredir parallelism". Once Tonyc/whoever is fine with it, I'll git squash it all together.

-- 
bulk88 ~ bulk88 at hotmail.com

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=123867



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About