develooper Front page | perl.perl5.porters | Postings from June 2016

Re: Inefficient stat-ing of files?

Thread Previous
From:
David Farrell
Date:
June 24, 2016 14:44
Subject:
Re: Inefficient stat-ing of files?
Message ID:
CAAzBBqsoiyQt-cmk7TxG3fdef_ckwf-r_8fUKhcb01pcwwBOxA@mail.gmail.com
Actually, maybe the user can already do this by manipulating @INC in a
BEGIN{} block?

On 23 June 2016 at 22:16, David Farrell <davidnmfarrell@gmail.com> wrote:

> Hi Porters,
>
> I used strace on a Perl one liner and found that 12% of the time was spent
> stat-ing files like this:
>
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/strict.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/
> strict.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/strict.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/
> strict.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/strict.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/strict.pm",
> {st_mode=S_IFREG|0444, st_size=4433, ...}) = 0
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/warnings.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
>
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/
> warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/warnings.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/
> warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/warnings.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/
> warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/warnings.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/warnings.pm",
> {st_mode=S_IFREG|0444, st_size=44835, ...}) = 0
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/XSLoader.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/XSLoader.pm",
> 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/XSLoader.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/XSLoader.pm",
> 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/XSLoader.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/XSLoader.pm",
> 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/XSLoader.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/XSLoader.pm",
> {st_mode=S_IFREG|0444, st_size=10305, ...}) = 0
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/Exporter.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/Exporter.pm",
> 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/Exporter.pmc",
> 0x7ffecd546980) = -1 ENOENT (No such file or directory)
> stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/Exporter.pm",
> 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
>
> In these cases Perl was stat-ing 7 files for every 1 file that existed.
> I'm no systems programmer but have you considered using the directory of
> the most recent successful stat rather than cycling through directories in
> the same order? I wonder if that could save startup time, particularly for
> programs with hundreds of dependencies.
>
> This is with Perl 5.22 on Fedora, Linux kernel 4.3.5-300
>
> Interested to hear your thoughts, thanks!
>
> David
>

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About