develooper Front page | perl.perl5.porters | Postings from July 2020

Re: PERL_PERTURB_KEYS=2

Thread Previous | Thread Next
From:
demerphq
Date:
July 28, 2020 21:39
Subject:
Re: PERL_PERTURB_KEYS=2
Message ID:
CANgJU+VVsHS0gfo4XBSSRrF+xgjCm2vFYF6HkD=cRCN+nnpaKQ@mail.gmail.com
Can you tell me what PERL_HASH_SEED_DEBUG=1 reports for you?

I have a feeling your perl is built with hadh seed randomization disabled.

Yves

On Tue, 28 Jul 2020, 22:09 , <hv@crypt.org> wrote:

> demerphq <demerphq@gmail.com> wrote:
> :On Mon, 27 Jul 2020 at 15:57, <hv@crypt.org> wrote:
> :>
> :> I have an application that I believe is otherwise deterministic, but had
> :> a heisenbug caused by an unintentional dependency on hash ordering.
> :>
> :> Before managing to find the bug by other means, I tried unsuccessully
> :> to create a reproducible testcase by experimentally setting
> PERL_HASH_SEED
> :> to $h > 0, and PERL_PERTURB_KEYS to $k in (0, 1 (default), 2).
> :>
> :> I found that with $k=1 the application failed randomly for each $h;
> :> with $k=0 it consistently succeeded for each $h; and with $k=2 it
> :> again failed randomly for each $h. That last ("DETERMINISTIC") feels
> :> like a bug.
> :>
> :> I tried unsuccessfully to reproduce the issue with a simple test:
> :>   $ENV{PERL_PERTURB_KEYS} = 2;
> :
> :>   for my $h (1 .. 10) {
> :>     for (1 .. 3) {
> :>       local $ENV{PERL_HASH_SEED} = $h;
> :>       print "$h: ";
> :>       system(q{perl}, q{-e}, q{
> :>         %x = map +($_ => 1), ("a".."z");
> :>         print join("", keys %x), "\n";
> :>       })
> :>     }
> :>   }
> :> .. but saw no non-deterministic behaviour there (and in fact $h does
> :> not seem to cause any variation in results either).
> :
> :I do not really understand the question. What I see, below, is what I
> :expect, 10 triplets of the same thing.
> :
> :$ perl t.pl
> :1: ynxpwfzakiceqbjlhutvsodgmr
> :1: ynxpwfzakiceqbjlhutvsodgmr
> :1: ynxpwfzakiceqbjlhutvsodgmr
> :2: xhipulwjymensrbvkotfdzcagq
> [...]
>
> Apologies if I've been unclear.
>
> With 5.32 I see all 10 triplets being the same as each other:
> 1: fmpdvjxiezlghkrotnyasuqwbc
> 1: fmpdvjxiezlghkrotnyasuqwbc
> 1: fmpdvjxiezlghkrotnyasuqwbc
> 2: fmpdvjxiezlghkrotnyasuqwbc
> 2: fmpdvjxiezlghkrotnyasuqwbc
> 2: fmpdvjxiezlghkrotnyasuqwbc
> 3: fmpdvjxiezlghkrotnyasuqwbc
> 3: fmpdvjxiezlghkrotnyasuqwbc
> 3: fmpdvjxiezlghkrotnyasuqwbc
> 4: fmpdvjxiezlghkrotnyasuqwbc
> 4: fmpdvjxiezlghkrotnyasuqwbc
> 4: fmpdvjxiezlghkrotnyasuqwbc
> 5: fmpdvjxiezlghkrotnyasuqwbc
> 5: fmpdvjxiezlghkrotnyasuqwbc
> 5: fmpdvjxiezlghkrotnyasuqwbc
> 6: fmpdvjxiezlghkrotnyasuqwbc
> 6: fmpdvjxiezlghkrotnyasuqwbc
> 6: fmpdvjxiezlghkrotnyasuqwbc
> 7: fmpdvjxiezlghkrotnyasuqwbc
> 7: fmpdvjxiezlghkrotnyasuqwbc
> 7: fmpdvjxiezlghkrotnyasuqwbc
> 8: fmpdvjxiezlghkrotnyasuqwbc
> 8: fmpdvjxiezlghkrotnyasuqwbc
> 8: fmpdvjxiezlghkrotnyasuqwbc
> 9: fmpdvjxiezlghkrotnyasuqwbc
> 9: fmpdvjxiezlghkrotnyasuqwbc
> 9: fmpdvjxiezlghkrotnyasuqwbc
> 10: fmpdvjxiezlghkrotnyasuqwbc
> 10: fmpdvjxiezlghkrotnyasuqwbc
> 10: fmpdvjxiezlghkrotnyasuqwbc
>
> :You should see the same. What do you see instead and what do you expect
> to see?
>
> I expected either to see the same as you (PERL_PERTURB_KEYS=2 working as
> documented) or variation within triplets (reproducing the non-determinism
> I appear to get with my larger application.
>
> :BTW, PERL_PERTURB_KEYS = 2 does not stop the peturbing, it makes the
> :perturbing deterministic, when PERL_PERTURB_KEYS = 1 it mixes in data
> :which can vary from run to run. PERL_PERTURB_KEYS stops the perturbing
> :entirely.
> :
> :Also PERL_PERTURB_KEYS is orthogonal to PERL_HASH_SEED. And yes,
> :setting the seed changes the order, which is why there are 10 sets of
> :the same order.
> :
> :I think what you want is to do PERL_HASH_SEED=0 which is magic, and
> :sets PERL_PERTURB_KEYS=0 and sets the seed to a standard default, so
> :from the point of view of the hash engine it is totally deterministic.
>
> What I _was_ trying to do was deterministically to reproduce my heisenbug.
> But I found the bug by other means (examining all uses of C<keys> and
> C<values> in my application), so that's no longer the issue.
>
> PERL_PERTURB_KEYS=0 did not help me, because for all hash keys that failed
> to reproduce the bug - the test case passed every time.
>
> I expect PERL_PERTURB_KEYS=2 to give me deterministic behaviour for a
> given hash seed, but in my larger application I fail to see that - the
> bug remains a heisenbug.
>
> I'm in the process of trying to cut it down to the point I can either
> see what I've done wrong or post a real test case here, but it'll likely
> take a few days.
>
> Hugo
>

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About