develooper Front page | perl.perl5.porters | Postings from February 2014

[perl #121119] [PATCH] speed up miniperl @INC searching, buildcustomize

Thread Previous | Thread Next
Tony Cook via RT
February 3, 2014 05:30
[perl #121119] [PATCH] speed up miniperl @INC searching, buildcustomize
Message ID:
On Fri Jan 31 03:31:05 2014, davem wrote:
> On Thu, Jan 30, 2014 at 11:49:41AM -0800, bulk88 via RT wrote:
> > On Thu Jan 30 05:30:31 2014, davem wrote:
> > > Perhaps this implies that's there's something sub-optimal in the
> > > way
> > > perl
> > > does its @INC scanning under Win32?
> >
> > Go to the RT ticket through web interface, and look at the
> > bad_inc_procmon.txt attachment, it contains strace like data with
> > wall
> > time on the very left. The calls are sniffed by a filter driver that
> > gets first dibs on all I/O packets (since NT Kernel uses a packetized
> > (or transactional) async I/O model) coming from user mode.
> [snip lots of trace output]
> > and the loop continues. Note that by calling all the
> > "SomethingFileMoreSomethingA" function calls, each A call does a
> > to UTF16 conversion. There goes a lil bit of CPU. Maybe win32_stat
> > should do a utf16 conversion and use only kernel32W() and wclib()
> > calls
> > afterwards which dont convert.
> Well, what's not clear to me is whether it's the windows system calls
> that
> have the big overhead, or perl doing stuff like UTF16 conversions. I
> would
> suspect the former.

win32_stat() does a fair bit to try and emulate a POSIX stat() - including checking for the number of links and making sure file attributes are properly updated for hard links.  Unfortunately, setting ${^WIN32_SLOPPY_STAT} to 1 in didn't make much a difference in a rough build benchmark (from 712 to 697 seconds, which is probably just noise, considering I used the machine as my desktop.)

> Other than that, I'm really in a position to contribute further to to
> this
> thread. I know almost zero about Windows APIs, so I have no way of
> knowing
> whether perl is using those APIs inefficiently, or whether Windows is
> just
> slow in this area. Or even whether you just have very slow hardware.

I'm not sure why perl builds on Win32 are so slow - I get very fast builds in a VM on the same hardware (eg. a nmake on Win32 took 712 seconds, a Configure + non-parallel make on an Ubuntu VM took 335 seconds).

> On UNIX, it is normal to run make with -j N, which will do a parallel
> build, with make building up to N targets in parallel. So all the
> parallelism is handled by make, not make_ext.

dmake (the tool, not our dmake makefile) supports parallel builds, but from discussing this on IRC, it's not practical since MSVC will attempt to lock the common PDB file and abort if it fails to do so.

Maybe a flag to skip checking for .pmc files would help, but I wouldn't expect require's .pmc checks to be a noticable timesink for typical usage.


via perlbug:  queue: perl5 status: open

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About