Front page | perl.perl5.porters |
Postings from June 2016
Inefficient stat-ing of files?
Thread Next
From:
David Farrell
Date:
June 24, 2016 14:44
Subject:
Inefficient stat-ing of files?
Message ID:
CAAzBBqsP6nFHDXQwgwafczhtDPv14rscs14s7SjQ_+qsFOiHQw@mail.gmail.com
Hi Porters,
I used strace on a Perl one liner and found that 12% of the time was spent
stat-ing files like this:
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/strict.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/
strict.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/strict.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/
strict.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/strict.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/strict.pm",
{st_mode=S_IFREG|0444, st_size=4433, ...}) = 0
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/warnings.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/
warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/warnings.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/
warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/warnings.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/
warnings.pm", 0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/warnings.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/warnings.pm",
{st_mode=S_IFREG|0444, st_size=44835, ...}) = 0
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/XSLoader.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/XSLoader.pm",
0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/XSLoader.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/XSLoader.pm",
0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/XSLoader.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/x86_64-linux/XSLoader.pm",
0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/XSLoader.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/5.22.0/XSLoader.pm",
{st_mode=S_IFREG|0444, st_size=10305, ...}) = 0
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/Exporter.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/x86_64-linux/Exporter.pm",
0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/Exporter.pmc",
0x7ffecd546980) = -1 ENOENT (No such file or directory)
stat("/home/dfarrell/.plenv/versions/5.22.0/lib/perl5/site_perl/5.22.0/Exporter.pm",
0x7ffecd5468b0) = -1 ENOENT (No such file or directory)
In these cases Perl was stat-ing 7 files for every 1 file that existed. I'm
no systems programmer but have you considered using the directory of the
most recent successful stat rather than cycling through directories in the
same order? I wonder if that could save startup time, particularly for
programs with hundreds of dependencies.
This is with Perl 5.22 on Fedora, Linux kernel 4.3.5-300
Interested to hear your thoughts, thanks!
David
Thread Next
-
Inefficient stat-ing of files?
by David Farrell