develooper Front page | perl.perl5.porters | Postings from July 2020

Re: PERL_PERTURB_KEYS=2

Thread Previous | Thread Next
From:
hv
Date:
July 28, 2020 20:09
Subject:
Re: PERL_PERTURB_KEYS=2
Message ID:
202007281927.06SJRaP17979@crypt.org
demerphq <demerphq@gmail.com> wrote:
:On Mon, 27 Jul 2020 at 15:57, <hv@crypt.org> wrote:
:>
:> I have an application that I believe is otherwise deterministic, but had
:> a heisenbug caused by an unintentional dependency on hash ordering.
:>
:> Before managing to find the bug by other means, I tried unsuccessully
:> to create a reproducible testcase by experimentally setting PERL_HASH_SEED
:> to $h > 0, and PERL_PERTURB_KEYS to $k in (0, 1 (default), 2).
:>
:> I found that with $k=1 the application failed randomly for each $h;
:> with $k=0 it consistently succeeded for each $h; and with $k=2 it
:> again failed randomly for each $h. That last ("DETERMINISTIC") feels
:> like a bug.
:>
:> I tried unsuccessfully to reproduce the issue with a simple test:
:>   $ENV{PERL_PERTURB_KEYS} = 2;
:
:>   for my $h (1 .. 10) {
:>     for (1 .. 3) {
:>       local $ENV{PERL_HASH_SEED} = $h;
:>       print "$h: ";
:>       system(q{perl}, q{-e}, q{
:>         %x = map +($_ => 1), ("a".."z");
:>         print join("", keys %x), "\n";
:>       })
:>     }
:>   }
:> .. but saw no non-deterministic behaviour there (and in fact $h does
:> not seem to cause any variation in results either).
:
:I do not really understand the question. What I see, below, is what I
:expect, 10 triplets of the same thing.
:
:$ perl t.pl
:1: ynxpwfzakiceqbjlhutvsodgmr
:1: ynxpwfzakiceqbjlhutvsodgmr
:1: ynxpwfzakiceqbjlhutvsodgmr
:2: xhipulwjymensrbvkotfdzcagq
[...]

Apologies if I've been unclear.

With 5.32 I see all 10 triplets being the same as each other:
1: fmpdvjxiezlghkrotnyasuqwbc
1: fmpdvjxiezlghkrotnyasuqwbc
1: fmpdvjxiezlghkrotnyasuqwbc
2: fmpdvjxiezlghkrotnyasuqwbc
2: fmpdvjxiezlghkrotnyasuqwbc
2: fmpdvjxiezlghkrotnyasuqwbc
3: fmpdvjxiezlghkrotnyasuqwbc
3: fmpdvjxiezlghkrotnyasuqwbc
3: fmpdvjxiezlghkrotnyasuqwbc
4: fmpdvjxiezlghkrotnyasuqwbc
4: fmpdvjxiezlghkrotnyasuqwbc
4: fmpdvjxiezlghkrotnyasuqwbc
5: fmpdvjxiezlghkrotnyasuqwbc
5: fmpdvjxiezlghkrotnyasuqwbc
5: fmpdvjxiezlghkrotnyasuqwbc
6: fmpdvjxiezlghkrotnyasuqwbc
6: fmpdvjxiezlghkrotnyasuqwbc
6: fmpdvjxiezlghkrotnyasuqwbc
7: fmpdvjxiezlghkrotnyasuqwbc
7: fmpdvjxiezlghkrotnyasuqwbc
7: fmpdvjxiezlghkrotnyasuqwbc
8: fmpdvjxiezlghkrotnyasuqwbc
8: fmpdvjxiezlghkrotnyasuqwbc
8: fmpdvjxiezlghkrotnyasuqwbc
9: fmpdvjxiezlghkrotnyasuqwbc
9: fmpdvjxiezlghkrotnyasuqwbc
9: fmpdvjxiezlghkrotnyasuqwbc
10: fmpdvjxiezlghkrotnyasuqwbc
10: fmpdvjxiezlghkrotnyasuqwbc
10: fmpdvjxiezlghkrotnyasuqwbc

:You should see the same. What do you see instead and what do you expect to see?

I expected either to see the same as you (PERL_PERTURB_KEYS=2 working as
documented) or variation within triplets (reproducing the non-determinism
I appear to get with my larger application.

:BTW, PERL_PERTURB_KEYS = 2 does not stop the peturbing, it makes the
:perturbing deterministic, when PERL_PERTURB_KEYS = 1 it mixes in data
:which can vary from run to run. PERL_PERTURB_KEYS stops the perturbing
:entirely.
:
:Also PERL_PERTURB_KEYS is orthogonal to PERL_HASH_SEED. And yes,
:setting the seed changes the order, which is why there are 10 sets of
:the same order.
:
:I think what you want is to do PERL_HASH_SEED=0 which is magic, and
:sets PERL_PERTURB_KEYS=0 and sets the seed to a standard default, so
:from the point of view of the hash engine it is totally deterministic.

What I _was_ trying to do was deterministically to reproduce my heisenbug.
But I found the bug by other means (examining all uses of C<keys> and
C<values> in my application), so that's no longer the issue.

PERL_PERTURB_KEYS=0 did not help me, because for all hash keys that failed
to reproduce the bug - the test case passed every time.

I expect PERL_PERTURB_KEYS=2 to give me deterministic behaviour for a
given hash seed, but in my larger application I fail to see that - the
bug remains a heisenbug.

I'm in the process of trying to cut it down to the point I can either
see what I've done wrong or post a real test case here, but it'll likely
take a few days.

Hugo

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About