develooper Front page | perl.perl5.porters | Postings from January 2022

Re: Benchmarking a 'no-snails' world (was: Re: PSC #049 2022-01-07)

Thread Previous | Thread Next
From:
Paul "LeoNerd" Evans
Date:
January 17, 2022 22:15
Subject:
Re: Benchmarking a 'no-snails' world (was: Re: PSC #049 2022-01-07)
Message ID:
20220117221533.401f4a54@shy.leonerd.org.uk
On Mon, 17 Jan 2022 18:28:19 +0000
"Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk> wrote:

> Next up, a perl built from my discourage-defav-in-sigsub branch (this
> is significantly slower than the release perl above in absolute terms,
> because it's an unoptimised debug build; but ignore that):

ilmari points out that using an unoptimised debug build is unlikely to
give useful results.

I've rebuilt these tests using non-debug -O3, and also running the
actual code for 4 times longer to reduce timing jitter/noise:

>   $ ./perl -Ilib benchmark-entersub.pl 
>   full: 9.0413s
>   bare: 7.0131s (speedup x1.29)
>   sigs: 6.4964s (speedup x1.39)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 4.2597s
  bare: 3.5885s (speedup x1.19)
  sigs: 3.5397s (speedup x1.20)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 4.3395s
  bare: 3.6818s (speedup x1.18)
  sigs: 3.5970s (speedup x1.21)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 4.0384s
  bare: 3.3404s (speedup x1.21)
  sigs: 3.4798s (speedup x1.16)

So both bare and sigs are "about 20% faster", though not much
difference between them. I seem to be getting more noisy results this
time around though, so maybe harder to interpret. This is why I'm
posting three separate runs.

> Finally, by noticing that the example code we're benchmarking doesn't
> really depend on the values it returns, I decided to break perl by
> doing *even less work* than would actually be required to make the
> args give the right answers, just to get an upper bound on the highest
> possible speedup that could be achieved. In this broken version, I
> don't set up GvAV(PL_defav), nor do I set up the AV in PAD_SVl(0). I
> don't copy the arguments anywhere at all. OP_ARGELEM now can't find
> them and will just return undef. I even had to stub out the contents
> of pp_argcheck so it doesn't even perform an arity check. To be
> clear: this version of perl is totally useless, but should be even
> faster than it is possible to achieve for real, because any real perl
> would have to do more work than this version:
> 
>   $ ./perl -Ilib benchmark-entersub.pl 
>   full: 8.7818s
>   bare: 6.9137s (speedup x1.27)
>   sigs: 6.7048s (speedup x1.31)

Now (on three separate runs):

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 4.0394s
  bare: 3.3806s (speedup x1.19)
  sigs: 3.3622s (speedup x1.20)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 3.9630s
  bare: 3.2788s (speedup x1.21)
  sigs: 3.2075s (speedup x1.24)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 4.0202s
  bare: 3.6163s (speedup x1.11)
  sigs: 3.3566s (speedup x1.20)

Again fairly noisy (especially this last one), but still quite
indistinguishable from the first runs; or indeed from those of a plain
unpatched bleadperl:

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 3.3048s
  bare: 2.6910s (speedup x1.23)
  sigs: 2.5559s (speedup x1.29)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 3.4209s
  bare: 2.7970s (speedup x1.22)
  sigs: 2.6608s (speedup x1.29)

  $ ./perl -Ilib benchmark-entersub.pl 
  full: 3.2664s
  bare: 2.6584s (speedup x1.23)
  sigs: 2.5078s (speedup x1.30)

In fact if anything, I'd say these plain bleadperls are running even
more efficiently than the patched versions.

-- 
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk      |  https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/  |  https://www.tindie.com/stores/leonerd/

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About