develooper Front page | perl.perl5.porters | Postings from November 2009

Re: Speeding up mktables; NYTprof

Thread Previous | Thread Next
karl williamson
November 25, 2009 21:43
Re: Speeding up mktables; NYTprof
Message ID:
Oops, forgot to put the function in.
karl williamson wrote:
> Nicholas Clark wrote:
>>> [...]
>> Also, you mentioned Encode and the files it generated. I found a lot 
>> of scope
>> for optimisation within enc2xs, which dramatically decreased its 
>> memory use
>> and run time, without needing large fundamental design changes to how it
>> worked. And I managed that with only Devel::DProf. I'm curious what 
>> can be
>> achieved with Devel::NYTProf on mktables. (Which is a task that is within
>> the skill set of any of the 600 subscribers to this list. I'm hoping 
>> that it
>> might appeal to at least one.)
> So, I tried it with NYTProf.  As I expected (I had used DProf earlier), 
> the highest usage subroutine was my pure Perl version of 
> Scalar::Util::refaddr, reproduced below.  A third of the total time was 
> spent in this routine.  (This is required because miniperl doesn't do 
> dynamic loading, so refaddr is not available.)
> When I was writing mktables, I was under the impression that refaddr 
> would be brought into the core for 5.12.  There was an agreement to that 
> effect, but I guess no one ever got around to actually doing it.
> When I run this under perl instead of miniperl, and change objaddr to 
> just return refaddr, the combination still takes quite a lot of time. If 
> refaddr were in the core would it be in-lined?
> I found a few surprises; I haven't pored over the results, though.  One 
> is that I left in a trace statement that got to the trace subroutine 
> before discovering that it had nothing to do.  This added not very much 
> time.
> Based on looking at existing code in, I had presumed that 
> the Perl optimizer would remove code that depended on a constant 
> subroutine that returns false.  That is, 'foo if DEBUG' would be 
> optimized away if there was a line: 'sub Debug { 0 }'  But that appears 
> to not be the case.
> There were more string evals than I expected, though the total time did 
> not add up to all that much.  I couldn't find a way in NYTProf to 
> highlight those.
> There are two columns in the nytprofhtml output for subroutines that I 
> can't figure out what they mean, and saw no documentation for, 'P' and 'F'.
> I also did not see anything there for memory usage.  I don't know how 
> Perl handles using up too much memory.  The old mktables kept all its 
> tables in memory, and so I felt free to do so as well.  But the new 
> mktables handles quite a few more tables than the old one.  I would 
> think you would get thrashing if the memory usage got too big.  I wonder 
> if Steve's machine doesn't have much memory.  When I run mktables using 
> perl instead of miniperl, I get execution times between 30 and 40 seconds.
sub objaddr($) {
     # Returns the address of the blessed input object.  Uses the XS 
version if
     # available.  It doesn't check for blessedness because that would do a
     # string eval every call, and the program is structured so that this is
     # never called for a non-blessed object.

     return Scalar::Util::refaddr($_[0]) if $has_fast_scalar_util;

     # Get the package
     my $pkg = ref($_[0]) or return undef;

     # Change to a fake package to defeat any overloading
     bless $_[0], 'main::Fake';

     # Numifying a ref gives its address.
     my $addr = 0 + $_[0];

     # Return to original class
     bless $_[0], $pkg;
     return $addr;

I found that any overload in a class caused the numifying to fail if I 
did it in that class; hence the blesses are necessary.

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About