develooper Front page | perl.perl5.porters | Postings from October 2003

Re: 5.8.2-RC1 and mp2

Thread Previous | Thread Next
From:
Stas Bekman
Date:
October 29, 2003 23:42
Subject:
Re: 5.8.2-RC1 and mp2
Message ID:
3FA0C0EB.1060402@stason.org
Nicholas Clark wrote:
> On Tue, Oct 28, 2003 at 04:10:34PM -0800, Stas Bekman wrote:
> 
>>Nicholas Clark wrote:
>>
>>>On Tue, Oct 28, 2003 at 03:44:56PM -0800, Stas Bekman wrote:
>>>
>>>
>>>>To remind what the story is: as mp2 uses PERL_HASH to cache the key 
>>>>hashes, the hash seed has to stay indentical across different perl 
>>>>interpreters. The workaround for 5.8.1 was to take over the setting 
>>>>PL_hash_seed and telling Perl not to reset it by setting PL_hash_seed_set 
>>>>to TRUE>
>>>>
>>>>I remember Nick saying, that 8.2 sets that hash seed to 0 and keeps it 
>>>>that way, taking special measures only if under attack. I haven't 
>>>>debugged the issue yet, but it seems that in 5.8.2 PERL_HASH is not a 
>>>>deterministic function and will return different values at different 
>>>>times. This is not how it was in 5.8.0. Before I fire up gdb, can you 
>>>>please summarize how different 5.8.2 is from 5.8.0 in the hash seed 
>>>>aspect? Thanks.
>>>
>>>
>>>Yes, sorry, I think there is still at least one of your p5p e-mails that
>>>I have yet to reply to.
>>>
>>>If "under attack" (from any sort of user data)  5.8.2 switches internally
>>>to randomised hashing. It's using PL_new_hash_seed and
>>>PL_new_hash_seed_set internally in a similar fashion to 5.8.1's use of
>>>PL_hash_seed and PL_hash_seed_set (in external code)
>>
>>So 5.8.2 is using PL_new_hash_seed only if under attack, right? Otherwise 
>>using PL_hash_seed, which is supposed to be 0, if 5.8.2 == 5.8.0.
> 
> 
> Correct
> 
> 
>>That's exactly the problem. If I revert the changes I did to workaround the 
>>change in 5.8.1, and build with 5.8.2 it breaks right away. I doubt that 
>>100 normal GVs will trigger the "under attack" condition.
>>
>>
>>>I hope that the fix is a simple as replicating whatever mod_perl did
>>>with PL_hash_seed and PL_hash_seed_set onto
>>>PL_new_hash_seed and PL_new_hash_seed_set.
>>
>>I've lost you. Earlier you said that under attack passed PERL_HASH values 
>>are going to be ignored for all affected GVs, and hash values recalculated 
>>for *each* repeated fetch_gv call. If I understood you correctly no 
>>workaround will be needed. I guess we aren't on the same line. Which is why 
>>a test will help to verify the assumptions. Though before that we have a 
>>problem long before the attack, that I'd like to figure out first.
> 
> 
> I think that I've failed to be clear. mod_perl will still need an
> analogous workaround for 5.8.2. All 5.8.1 hashes are randomised.
> Some 5.8.2 hashes are randomised. (But only the hashes being fed
> pathological data)

I don't think so. If we do that we abolish the protection. Besides it doesn't 
work. it has no effect at all if I preset:

         PL_new_hash_seed_set = TRUE;
         PL_new_hash_seed = MP_init_hash_seed;

like I did for 5.8.1 (which works):

         PL_hash_seed_set = TRUE;
         PL_hash_seed = MP_init_hash_seed;

> During the perl build Encode's enc2xs and a script in Unicode::Normalise
> both hash sufficiently pathological data that the randomisation kicks in.
> Unicode::Normalize seems to be doing something as innocent as hashing a large
> list of integers.
> 
> It would not be impossible for mod_perl to have some real data that was
> similar. But it seems unlikely.

As you can see below, it's not only likely, it happens in our ever-growing 
test suite ;)

> My understanding of GVs may be failing me here. You keep mentioning
> fetch_gv, but that's not part of the perl API. As far as I can see all
> the functions in gv.c in perl call hv.c functions to do the lookups, and
> hv.c functions now know when to rehash.

sorry, mp2 implements a light-weighted version of gv_fetchpv (it doesn't use 
it). It uses PERL_HASH to cache the hash values and then it directly retrieves 
those GVs via hv_fetch_he(stash, mgv->name, mgv->len, mgv->hash);

> I'm going to get the mod_perl source after I send this message.

Please ask me any questions directly if you have any problems to get it running.

> Three independent experiments you may wish to try before resorting to gdb are
> 
> 0: Run with PERL_HASH_SEED=0 (to nullify randomisation)
>    If this works, then it has to be the conditional randomisation.

works!

> 1: Place a return; immediately after
>    /* Pick your policy for "hashing isn't working" here:  */
>    This will disable the randomisation

works!

> 2: Change
>    #define HV_MAX_LENGTH_BEFORE_SPLIT 4
>    in hv.c from 4 to (say) 16 to attempt to put off randomisation as long
>    as possible (in case something in the GVs or elsewhere is triggering
>    randomisation.

works!

So the conclusion is that the randomization kicks in too early? If so that 
doesn't sound efficient at all. I don't think we get more than 40 keys in each 
HV. Next I'm going to try your scripts that reproduce the attack conditions.


__________________________________________________________________
Stas Bekman            JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/     mod_perl Guide ---> http://perl.apache.org
mailto:stas@stason.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org   http://ticketmaster.com


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About