[perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz

# New Ticket Created by  schmorp@schmorp.de 
# Please include the string:  [perl #117271]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=117271 >


> > Now, if you didn't quote anything, what would your statement refer to?
> 
> Ah, right, I didn't consider that "quoting" even tho of course it is.

And, was I right in my reply or not?

> > In case you wonder why it is a no-goal, I can discover these hash seeds
> > with a debugger, or an XS module, and I doubt your change will stop me
> > from doing it (to the extent this still exists).
> 
> Peter Rabbitson already replied to this.

Yes, you should read the resulting thread, it's quite informative.

> > On Fri, Mar 22, 2013 at 01:20:40AM +1100, Peter Rabbitson <rabbit-p5p@rabbit.us> wrote:
> > Because I originally said it's about avoiding DOS attacks (and later
> > refining that to mean cpu/memory starvation), and yves said, no, it's
> > purpose is to make it hard to discover the seed, after which I pointed out
> > that this is unlikely the real purpose, because you can already discover
> > this seed in other ways (for example, by setting an env variable before
> > starting the program).
> 
> You know that this is not about XS nor having access to a debugger,
> nor the ability to set the environment var, yet bring it up anyway.

I brought it up originally because I used it as an argument against you
claiming the purpose was not a DOS attack, but avoiding the seed to be
discovered.

You conveniently ignore my actual argument, and that what you said it's
real purpose is, is misleading or disingenious.

> So you are trolling.

I am obviously not, but it seems you ran out of arguments quickly,
ignoring all mine.

Sad.

> Your attitude makes me want to split hairs with you.

"I am not responsible, officer, he made me kill him!".

Given that you misunderstood what I wrote multiple times and then ignored
when I set you right, I am fairly sure you don't know what my attitude is.

But using that as excuse is lame. I am not playing any games, I talk about
technical issues - possibly in form of a rant.

Attacking me personally won't bring us forward, it just makes you into a
jerk, and since I don't think you are, you will only regret this later (as
has happened before when you jumped to conclusions...).

Let's go back to the technical issue. Feel free to use clear language, but
ignoring all my technical agruments and attacking my person is a waste of
time.

> highly disagreeable style of communicating with people and should not
> be surprised when they do not want to be helpful towards you.

You can disagree with my style, and I don't *expect* you to be
helpful. Just reasonable.

What you now seem to tell me is that you would insist on a bad patch, just
because the person who brought the problems to light used a disagreeable
style of communication.

> The patch to make hash iterator traversal random is specifically to
> make it harder for an attacker to determine the hash seed.

That's what I assumed. I made a specific attack example and reminded you
of existing mitigations that work on a broeder class of such problems, and
you disagreed.

It would bring su forward if you gave an example of where your patch
actually did something useful in a system that expects such resource
attacks (not specifically that one).

Or an attack where typical mitigations (resource limits and monitoring)
wouldn't work.

> > The purpose of your change is to make it harder to apply such DOS attacks.
> > Saying that's not the purpose, but the purpose is just to hide this
> > information is dishonest, because hiding this information isn't a useful
> > purpose in itself.
> 
> I haven't been in any way dishonest. There is nothing false about my
> statement at all.

Well, you are good at denial, but when I ask you direct questions about
this and you keep refusing to answer, that is dishonest, too.

Really, all you do is say "it's not true", and when asked for details, you
are silent.

> > I think it strongly implies it. Beat me if you wish, but your
> > interpretation is as good as mine, regardless of the intent behind it.
> 
> No, my interpretation is correct and yours is a figment of your
> imagination. There is no equivalence there.

I just looked up the meaning of "same" at thefreedictionary.com.

First meaning it lists is "identical".
Second meaning it lists is "similar in kind, quality, quantity, or degree".
(there are two more).

Sorry, but calling that a "figment of my imagination" is just your attempt
at bullying me.

The wording is clearly ambiguous in this case, and just saying "I am big
bozo, I am right" doesn't make it so.

> > Sorry, but "guaranteed to be the same as either the keys ..." clearly
> > implies some consistency, at least between those. Also saying the order
> > is the same for the "same hash" is ambiguous, as "same" has tow meanings,
> > namely, "identical" (of same identity) and "similar in some kind" (e.g.
> > same contents).
> 
> No, no, no. There is no consistency, there never has been, there never
> will be, and arguing about it is unproductive.

Ok, so ordering I get from keys is not consistent with the ordering I get
from values? If that are your plans it is indeed high time to fork perl,
because that isn't just clearly documented, it is clearly required for a
lot of existing and useful code.

Or you disagree with what "same" means? It seems arguing against
dictionaries is unproductive, though, it leads nowhere (except you being
wrong in this case, but it's still pointless...).

> >> the only guarantee of any form provided is that the keys() function
> >> will return items in the same order that each() and values would return
> >> them assuming the hash has not been modified (a secondary clause in the
> >> documentation for "each" specifies it is safe to delete the key that was
> >> just returned by each()).
> >
> > It also says the ordering is the same for the same hash.
> 
> Yes, as in identity.

And it does that where...?

> keys(%hash)
> returns items in a different order than
> values(%hash)
> does then please file a bug.

So suddenly it is consistent. This is all very confusing to me.

> > I know how it's meant, but I also know what is said, and assuming that
> > "the same hash will result in the same ordering" is not unreasonable.
> 
> The docs say "the same hash unmodified", so that implies very strongly
> that it means identity.

Well, not to me, and the other person I asked (I am too lazy to ask a
bigger control group, sorry).

> > Note that the earlier randomisation patch already broke the semantics, as
> > the ordering changed even within the same version (but different program
> > runs).
> 
> No, it didnt break any semantics. Stop making stuff up.

If you didn't break semantics, then explain why JSON::XS fails it's
testsuite?

> > Saying it changes in future versions but is the same for the same hash
> > STRONGLY implies that the ordering is the same for the same version of
> > perl, which is already not true.
> 
> No it doesn't.

Why mention it at all then? If the ordering is randomly changing, then why
mention it might change in future versions? It frankly makes no sense.

> >> perlfunc.pod didnt exist prior to that commit.
> > And it clearly changed, if you took notice.
> Not in any way relevant to this discussion, if you took notice.

Your point being that it is a documentation bug, that the existing
documentation is ambiguous or what?

I am not claiming your patch breaks documented behaviour (for every
interprettation). I am claiming the code in question doesn't rely on
undocumented behaviour.

> > You'd have to make the stronger claim that no other interprettaion is
> > reasonably possible, and I don't think you can, simply because "same", in
> > english, is ambiguous as to whether mean identity or equality.
> 
> There is nothing here to argue about, you are wrong, and there your
> interpretation is wrong, and that is all there is to it.

You seem to be unable to explain why it would be wrong, while I gave ample
evidence.

The only reason it seems to be wrong is that the person who wrote it might
have intended otherwise, but there is no evidence for that (was it tomc?),
and even if, people wrote programs according to the docs, and if the
documentation allows the behaviour because it is worded differently, and
this behaviour is useful and present for more than a decade (probably two
decades, who knows), then you can't just say "you are wrong".

Because all evidence for that is your word, and your refusal to explain
it.

> >> I don't think that, and haven't made that claim. I think you can make
> >                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Are you really so stupid as to claim you didnt say something only a
> few lines after you say it?

Please reconsider. It took me a while to find the context, but as you
force me to, here it is:

>> > - Perl hash already are very slow (just write yourself some XS functions to
>> >   interface to gcc's unordered_hash and watch the speed - despite the
>> >   overhead of XS).
>>
>> Yes, and this is a very tiny drop in a large bucket. If you think that
>> the changes I made are going to affect hash performance in any kind of
>> significant way then please provide evidence.
>
> I don't think that, and haven't made that claim.

And this is true. Calling me stupid and claiming I did, when I didn't, is
crossing the line.

I didn't say it would affect it in any significant way (heck, I don't know
what you mena with significant - there are a lot of slowdowns in perl that
I find significant but others don't).

I said they are already slow, and extra overhead (for sorting, or the
randomisation). And what I meant (as I think I explained later) is that
adding these kinds of overhead is likely unnecessary.

> Did you not say:
> 
> "I think you can make faster hashes with better properties without the
> drawbacks."

Yeah.

> And was I not replying to you saying it?

No.

Your interpretation is probably a valid one and I wouldn't gfault you if
you made assumptions based on it, but after I told you it isn't, whats the
point of arguing further?

I told you twice I didn't say that, while explaining what I mean with what
I actually said.

Sorry, but your interpretation is wrong, and I know, because I am
authoritative here. Trying to repeatedly claim I said somethig else after
I clarified is dishonest.

> > mouth, and I explain it again what I meant, and this time, please either
> > believe what I write, or argue against it, but please don't attack strawmen:
> >
> > I think one can make faster and better hashes which fix this problem without
> > breaking code.
> >
> > That means the breakage is not necessary.
> 
> So prove it.

I don't have to, I gave good evidence for it, while you gave no evidence
for the contrary.

> > I already did some benchmarks with unordered hashes. I wonder what they
> > do less than perl hashes, specially now that almost all conssitency in
> > ordering is gone, which is usually the only problem.
> 
> Which just shows you really have no idea at all how Perls hashes work.
> 
> Which pretty much ends this discussion.

Again, it seems when you run out of arguments you just play big boss.

I have to assume that means you just made your claim up.

> > If you want me to respect your opinions on what is fine code, please
> > respect mine.
> 
> I don't really give a shit about your opinion actually.

That's fine, but what you are doing is far worse: while I give evidence
for my opinions, you give none for yours, and discredit mine by calling it
silly etc.

> Especially as you frame it rudely,

Where?

> make personal aspersions, and act like a jerk when you express it.

Where?

I would guess you again either ginroe ym questions, or just claim it's
true without evidence.

(sure, I just say that to atcually get you to bring some evidence, but I
don't think you can...)

Really, all you do is run into personal attacks after you ran out of
arguments (which was very early, too).

> > I don't know if they are unwarranted, but at least they were documented,
> > even if that turns out to be ambiguous wording.
> 
> As I said before, there is nothing ambiguous about saying nothing
> about something.

How philosophical. Not the point, though-

> Ambiguity comes when there are more than one option,

Hey, I checked a few dictionaries, and listing "similar" as equal meaning
to "identical" is the norm.

Again, you are just claiming english is wrong and you are right. That's...
well, not very useful.

The claim that the same hash results in the same ordering is ambiguous
w.r.t. the meaning of what "same hash" means.

Identity is one meaning, but the others are there.

> here there are zero options.

Only because you ignroe the extra options I stated. As well as you
basically ignored all my other arguments.

With none on your own.

Sure that's the kind of maintainer that does good to perl: making
decisions because they are right, even though all the evidence points
against it.

> There is no reasonable interpretation of
> the documentation that supports your position.

There certainly is, I gave it, and ignoring it doesn't make it go away.

> > Well, the code in question (which defines my assumptions) works with older
> > versions of perl, so I call your bluff.
> 
> So sure, I can make it break with a one line change.

I know you can break code. If you wish I will call you master of breaking
code.

That's the problem. I don't want you to break code, not without good
reason. And all reasons you gave is "because you are right, and I am
wrong".

Right, I understand all that.

> > You seem to be very combative, and I guess this is why you make so many
> > simple mistakes: I usually really mean what I write (and even then, I do make
> > mistakes).
> 
> You are rude and insulting in your manner, you should not be surprised
> when people you communicate with are combative.

So where was I rude? Where was I insulting?

The problem here is that you misinterpreted what I said, by changing
words, AFTER I explained to you what I meant..

And now you claim I forced you to. That's just lame, and you know that.

> >> A) I have reasons,
> >
> > Never said anything to the contrary.
> 
> You certainly implied it with "breaking existing code for questionable
> or no reason is making it worse".

No, I implied what I wrote: the reasons are questionable or
nonexistant. And the context you again so conveniently clipped was:

> To me, breaking existing code for questionable or no reason is making it
> worse.

Quoting things out of context and then building strawmen about what wasn't
actually said is dishonest.

> >> B) any code broken was already broken but the author did not know it
> >
> > Empty claim.
> 
> Ive already demonstrated how your assumptions are broken by a simple hash copy.

Well, you didn't, as I have explained. The problem here is not "a simple
hash copy", but the assumptions behind the breakage in JSON::XS and in
other modules.

AFAIK, they don't rely on your example, or the assumptions you invent.

> > It says the ordering is the same for the same hash.
> 
> And it is.  (So long as you dont modify it by inserting new items.)

Experiments show you are wrong.

> > Changing the hash function doesn't break the code. Changing the hash
> > function within the same version of perl, and within the same process, is
> > what changes it.
> 
> We don't change the hash function during the process.

Well, the observable effects are different, which is the point. You are
splitting hais a lot again, when it's totalyl unnecessar< because I have
made explicit what the problem is, namely the breakage of JSON::XS.

That the function gets different inputs but the form stays the same is
immaterial, and I am just so sure you know that.

> > So if the ordering is really the same for the same hash, within the same
> > process (using the same perl), then I am indeed totally wrong.
> >
> > Is that the case?
> 
> @keys1= keys %hash;
> @keys2= keys %hash;
> 
> will always return identical @keys1 and @keys2 assuming %hash is a
> "normal" perl hash.

I call that a consistent ordering, you don't.

I think you have a totally unworkable definition of consistent. At least
that is clear now, and explains a few things.

What I mean with consistent is that the ordering is the same, within
limits.

> But since you bring it up again I will note I checked the log and you
> are mentioned a whole 6 times.

The my dick is longer than yours argument is not an argument. I still
regularly contributed, not just by posting on that mailinglist with this
e-mail address, over more than a decade.

Point being, it means nothing.

> > I interpreted your comment as irony, as am pretty confident it was meant
> > like it. If you really honetsly say you meant it as-is, I will, however,
> > believe you and admit my reaction to your last comment was wrong.
> 
> I meant every word in that sentence literally.

Ok, my turn to say sorry for having misinterpreted you. Oh wait, you
insulted me, you didn't apologize.

Well, I really am sorry for misinterpreting you then, I honestly thought
it was irony.

At least we can settle that part then?

> >> express, which is that you think that these changes contradict the
> >> documentation. The documentation does not support you in this regard.
> >
> > Well, it clearly does. It's your interpretation that doesn't support mine,
> > and now the code.
> 
> There is no "clearly does" involved here.

Well, I have explained why. All your say is "it's wrong".

I go with the evidence and not the big mouth.

> The documentation simply does not say what you seem to think it says.

That's beside the point, you said my interpretation is unreasonable (and
other things), and I say it was, and gave evidence on why.

I am also not the only one who interpreted it that way, as evidences by
the breakage that ensued after both stages of patches in that area.

That doesn't prove the reasonableness, of course.

> If it did it would say it explicitly,

It says it explicitly, but unfortunately, in an ambiguous way.

> but given how perls hash works (which I do know, and which you clearly
> do not),

I actually know very well how hashes worked before your patch. I studied
them long ago, and then regulalry.

You have no evidence that my knowledge is bad, after all, my code actually
worked, doesn't it, and I didnt make a wrong statement, did I?

What's your reason for making baseless claims at the lack of my
knowledge? You just want to be an ass, or you just want to feel bigger
than me? Really, I am curious.

> I am very very certain that no person that did understand how perls
> hashes work would ever say what you think the documentation says.

Maybe, but I wonder what relevance that has.

Do you really mean to claim that everybody should disregard the
documentation and read thre actual code before being able to write
reasonable or correct code?

I think it should be possible to write correct code by documentation and
experimenting alone, but we have to have to disagree on that point.

> > But the bug is in the perl core. Nothing I apply can fix that, becasue I
> > can't apply fixes to the perl core.
> 
> Excuse me? No the bug is in your code,

No, it's in the core. It was introduced recently, and apparently by your
patch (I blindly trust andk here).

> expecting to snapshot a serialized hash and have that be the same as
> some other serialized hash with the same keys.

I expect the same hash to have same ordering within one program run, yes.

> There has never been any reasonable basis to expect this to always
> happen.

Except the documentation, which to me, will always be a reasonable
basis. Askign people to understand all the perl core code before
reasonably expecting to write perl is not, uhum, reasonable for me.

But sure, you cna disagree here, but it's a horrible outlook for the
future of perl.

> Post some patches or shut up.

Not as long as you spread falseness or baseless accusations, no. You have
no right to that. At all, no matter how powerful you are.

> I dont care what your opinion is on this subject anymore.

And I told you that is valid. But you can't hide between not having been
told anymore.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp@schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\

• [perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz by schmorp @ schmorp . de
• Re: [perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz by Deven T. Corzine
• Re: [perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaksMLEHMANN/JSON-XS-2.33.tar.gz by Marc Lehmann
• Re: [perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaks MLEHMANN/JSON-XS-2.33.tar.gz by Leon Timmermans
• Re: [perl #117271] Re: Bleadperl v5.17.9-200-g0e0ab62 breaksMLEHMANN/JSON-XS-2.33.tar.gz by Marc Lehmann