develooper Front page | perl.perl5.porters | Postings from January 2014

Re: [perl #121119] [PATCH] speed up miniperl @INC searching,buildcustomize

Thread Previous | Thread Next
Dave Mitchell
January 31, 2014 11:30
Re: [perl #121119] [PATCH] speed up miniperl @INC searching,buildcustomize
Message ID:
On Thu, Jan 30, 2014 at 11:49:41AM -0800, bulk88 via RT wrote:
> On Thu Jan 30 05:30:31 2014, davem wrote:
> > Perhaps this implies that's there's something sub-optimal in the way
> > perl
> > does its @INC scanning under Win32?
> Go to the RT ticket through web interface, and look at the
> bad_inc_procmon.txt attachment, it contains strace like data with wall
> time on the very left. The calls are sniffed by a filter driver that
> gets first dibs on all I/O packets (since NT Kernel uses a packetized
> (or transactional) async I/O model) coming from user mode.
[snip lots of trace output]
> and the loop continues. Note that by calling all the
> "SomethingFileMoreSomethingA" function calls, each A call does a ASCII
> to UTF16 conversion. There goes a lil bit of CPU. Maybe win32_stat
> should do a utf16 conversion and use only kernel32W() and wclib() calls
> afterwards which dont convert.

Well, what's not clear to me is whether it's the windows system calls that
have the big overhead, or perl doing stuff like UTF16 conversions. I would
suspect the former.

Other than that, I'm really in a position to contribute further to to this
thread. I know almost zero about Windows APIs, so I have no way of knowing
whether perl is using those APIs inefficiently, or whether Windows is just
slow in this area. Or even whether you just have very slow hardware.

> On Thu Jan 30 03:50:16 2014, nicholas wrote:
> > 
> > Thanks for ensuring that the patch gets to the list.
> > 
> > No, specifically it's completely unsafe for a parallel make.
> > 
> > You end up with race conditions where the build fails because one
> > process
> > process loads from lib/ a partially written module and aborts, because
> > it
> > happens just as another process is copying that file there.
> > (And the rest of the time you don't have a problem because the file is
> > either
> > not in lib, so loaded from the original dist/... etc, or it's fully
> > copied to
> > lib, so loaded from there)
> > 
> > Which is exactly what the comment in the file tries to explain.
> make_ext has no provisions for parallel building. No select(), no open3. Can you explain?

On UNIX, it is normal to run make with -j N, which will do a parallel
build, with make building up to N targets in parallel. So all the
parallelism is handled by make, not make_ext.

If life gives you lemons, you'll probably develop a citric acid allergy.

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About