Front page | perl.perl5.porters |
Postings from October 2003
Re: 5.8.2-RC1 and mp2
From: Stas Bekman
October 29, 2003 23:42
Re: 5.8.2-RC1 and mp2
Message ID: 3FA0C0EB.email@example.com
Nicholas Clark wrote:
> On Tue, Oct 28, 2003 at 04:10:34PM -0800, Stas Bekman wrote:
>>Nicholas Clark wrote:
>>>On Tue, Oct 28, 2003 at 03:44:56PM -0800, Stas Bekman wrote:
>>>>To remind what the story is: as mp2 uses PERL_HASH to cache the key
>>>>hashes, the hash seed has to stay indentical across different perl
>>>>interpreters. The workaround for 5.8.1 was to take over the setting
>>>>PL_hash_seed and telling Perl not to reset it by setting PL_hash_seed_set
>>>>I remember Nick saying, that 8.2 sets that hash seed to 0 and keeps it
>>>>that way, taking special measures only if under attack. I haven't
>>>>debugged the issue yet, but it seems that in 5.8.2 PERL_HASH is not a
>>>>deterministic function and will return different values at different
>>>>times. This is not how it was in 5.8.0. Before I fire up gdb, can you
>>>>please summarize how different 5.8.2 is from 5.8.0 in the hash seed
>>>Yes, sorry, I think there is still at least one of your p5p e-mails that
>>>I have yet to reply to.
>>>If "under attack" (from any sort of user data) 5.8.2 switches internally
>>>to randomised hashing. It's using PL_new_hash_seed and
>>>PL_new_hash_seed_set internally in a similar fashion to 5.8.1's use of
>>>PL_hash_seed and PL_hash_seed_set (in external code)
>>So 5.8.2 is using PL_new_hash_seed only if under attack, right? Otherwise
>>using PL_hash_seed, which is supposed to be 0, if 5.8.2 == 5.8.0.
>>That's exactly the problem. If I revert the changes I did to workaround the
>>change in 5.8.1, and build with 5.8.2 it breaks right away. I doubt that
>>100 normal GVs will trigger the "under attack" condition.
>>>I hope that the fix is a simple as replicating whatever mod_perl did
>>>with PL_hash_seed and PL_hash_seed_set onto
>>>PL_new_hash_seed and PL_new_hash_seed_set.
>>I've lost you. Earlier you said that under attack passed PERL_HASH values
>>are going to be ignored for all affected GVs, and hash values recalculated
>>for *each* repeated fetch_gv call. If I understood you correctly no
>>workaround will be needed. I guess we aren't on the same line. Which is why
>>a test will help to verify the assumptions. Though before that we have a
>>problem long before the attack, that I'd like to figure out first.
> I think that I've failed to be clear. mod_perl will still need an
> analogous workaround for 5.8.2. All 5.8.1 hashes are randomised.
> Some 5.8.2 hashes are randomised. (But only the hashes being fed
> pathological data)
I don't think so. If we do that we abolish the protection. Besides it doesn't
work. it has no effect at all if I preset:
PL_new_hash_seed_set = TRUE;
PL_new_hash_seed = MP_init_hash_seed;
like I did for 5.8.1 (which works):
PL_hash_seed_set = TRUE;
PL_hash_seed = MP_init_hash_seed;
> During the perl build Encode's enc2xs and a script in Unicode::Normalise
> both hash sufficiently pathological data that the randomisation kicks in.
> Unicode::Normalize seems to be doing something as innocent as hashing a large
> list of integers.
> It would not be impossible for mod_perl to have some real data that was
> similar. But it seems unlikely.
As you can see below, it's not only likely, it happens in our ever-growing
test suite ;)
> My understanding of GVs may be failing me here. You keep mentioning
> fetch_gv, but that's not part of the perl API. As far as I can see all
> the functions in gv.c in perl call hv.c functions to do the lookups, and
> hv.c functions now know when to rehash.
sorry, mp2 implements a light-weighted version of gv_fetchpv (it doesn't use
it). It uses PERL_HASH to cache the hash values and then it directly retrieves
those GVs via hv_fetch_he(stash, mgv->name, mgv->len, mgv->hash);
> I'm going to get the mod_perl source after I send this message.
Please ask me any questions directly if you have any problems to get it running.
> Three independent experiments you may wish to try before resorting to gdb are
> 0: Run with PERL_HASH_SEED=0 (to nullify randomisation)
> If this works, then it has to be the conditional randomisation.
> 1: Place a return; immediately after
> /* Pick your policy for "hashing isn't working" here: */
> This will disable the randomisation
> 2: Change
> #define HV_MAX_LENGTH_BEFORE_SPLIT 4
> in hv.c from 4 to (say) 16 to attempt to put off randomisation as long
> as possible (in case something in the GVs or elsewhere is triggering
So the conclusion is that the randomization kicks in too early? If so that
doesn't sound efficient at all. I don't think we get more than 40 keys in each
HV. Next I'm going to try your scripts that reproduce the attack conditions.
Stas Bekman JAm_pH ------> Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide ---> http://perl.apache.org
mailto:firstname.lastname@example.org http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com