develooper Front page | perl.perl5.porters | Postings from March 2013

Re: [perl #117239] Re: [perl #117259] Re:Bleadperlv5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz

Thread Previous | Thread Next
Marc Lehmann
March 26, 2013 14:18
Re: [perl #117239] Re: [perl #117259] Re:Bleadperlv5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz
Message ID:
On Tue, Mar 26, 2013 at 01:35:52PM +0100, demerphq <> wrote:
> If you are referring to the timing results that Merijn posted then I
> dont know what you mean by "fooled" me. I looked at the numbers and
> concluded that the variance was in the noise.

The obvious conclusion is that result is suspicious and needs better
verification, not that the variance is in the noise.

> As for /your/ "benchmark", well, it doesn't seem to be very useful
> either. "real    0m0.808s" is the output of the bash time function,
> and does not constitute a benchmark in any way.

Of course it does, don't get silly.

It might not be a very exact benchmark, but unless you can point out an
error that cancels out 16% despite being a stable value, then there is
nothing wrong with that.

> which could almost be tailor written for this exact case.

None of that applies to this case.

> Try running under dumbbench for a while and see what it says. THAT
> would be a real benchmark.

Do it yourself, prove that my benchmark is wrong, and you have a
point. Until then, you are just spreading FUD to make my results look
somehow unreliable.

> What you posted is a discrete timing, and on its own tells us almost
> nothing.

It tells you that at least under some conditions, the code is 16% slower,
with the effect being smeared out over two million runs. Any variances in the
iteration are as likely part of a valid real-world timing as they are true

Cache colouring can have an effect this large, but the naive blog posting
you mentioned doesn't seem to be aware of that - the proposed robust
estimation doesn't take this into account.

Massaging your data set to throw out values you don't like without
evidence for them being actually false is unsound.

Recompiling and repeating the test, as I did, does take all this into

> "That probably doesn't mean much to p5pers who don't mind slowing down perl
> by a factor of up to two because its fun to run your own mmu emulation in
> the process emulation code, but it means a lot to JSON::XS."
> are unhelpful and just plain rude. We care a lot about slowing down
> perl, and if we do slow down perl we do only because we have good
> reasons, like correctness or security.

The example in my "rude" statement is neither about correctness or about
security, so your statement is triivally shown to be wrong. You are just
trying to be difficult...

> However when you say snotty things like me being fooled, or that we
> don't care about performance, you come across as an extremely
> unpleasant person,

Well, the evidence supports both.

If you find reality unpleasant, then I am the wrong person to complain to.

> to help you. The fact is I could mention a few things that would grind
> an extra few percent out of JSON::XS, which you seem to care about,

I am quite sure there are bug gains to be gotten. I have a few ideas
myself, and have consistently found significant gains in the past.

But frankly, either show your goods or shut up, because from what I see
of your experience here, it is as likely that you are mistaken as you are
right (did you benchmark your "few things", or do did you just make this

> but I probably won't because I am unwilling to help someone who is
> repeatedly rude and insulting to me and my colleagues. You need to

I think I am just pointing out the obvious.

> think about that for a while. You are missing out on the support of
> other clever people only because you cant keep a civil tongue in your
> head. That is sad.

I never had reason to believe that I miss out on support of other clever
people. Maybe I did, but that seems to be an empty threat.

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_    
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /
      -=====/_/_//_/\_,_/ /_/\_\

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About