Front page | perl.perl5.porters |
Postings from October 2003
Re: 5.8.2-RC1 and mp2
From: Nicholas Clark
October 29, 2003 13:11
Re: 5.8.2-RC1 and mp2
Message ID: 20031029211049.GJ6287@plum.flirble.org
On Tue, Oct 28, 2003 at 04:10:34PM -0800, Stas Bekman wrote:
> Nicholas Clark wrote:
> >On Tue, Oct 28, 2003 at 03:44:56PM -0800, Stas Bekman wrote:
> >>To remind what the story is: as mp2 uses PERL_HASH to cache the key
> >>hashes, the hash seed has to stay indentical across different perl
> >>interpreters. The workaround for 5.8.1 was to take over the setting
> >>PL_hash_seed and telling Perl not to reset it by setting PL_hash_seed_set
> >>to TRUE>
> >>I remember Nick saying, that 8.2 sets that hash seed to 0 and keeps it
> >>that way, taking special measures only if under attack. I haven't
> >>debugged the issue yet, but it seems that in 5.8.2 PERL_HASH is not a
> >>deterministic function and will return different values at different
> >>times. This is not how it was in 5.8.0. Before I fire up gdb, can you
> >>please summarize how different 5.8.2 is from 5.8.0 in the hash seed
> >>aspect? Thanks.
> >Yes, sorry, I think there is still at least one of your p5p e-mails that
> >I have yet to reply to.
> >If "under attack" (from any sort of user data) 5.8.2 switches internally
> >to randomised hashing. It's using PL_new_hash_seed and
> >PL_new_hash_seed_set internally in a similar fashion to 5.8.1's use of
> >PL_hash_seed and PL_hash_seed_set (in external code)
> So 5.8.2 is using PL_new_hash_seed only if under attack, right? Otherwise
> using PL_hash_seed, which is supposed to be 0, if 5.8.2 == 5.8.0.
> That's exactly the problem. If I revert the changes I did to workaround the
> change in 5.8.1, and build with 5.8.2 it breaks right away. I doubt that
> 100 normal GVs will trigger the "under attack" condition.
> > I hope that the fix is a simple as replicating whatever mod_perl did
> > with PL_hash_seed and PL_hash_seed_set onto
> > PL_new_hash_seed and PL_new_hash_seed_set.
> I've lost you. Earlier you said that under attack passed PERL_HASH values
> are going to be ignored for all affected GVs, and hash values recalculated
> for *each* repeated fetch_gv call. If I understood you correctly no
> workaround will be needed. I guess we aren't on the same line. Which is why
> a test will help to verify the assumptions. Though before that we have a
> problem long before the attack, that I'd like to figure out first.
I think that I've failed to be clear. mod_perl will still need an
analogous workaround for 5.8.2. All 5.8.1 hashes are randomised.
Some 5.8.2 hashes are randomised. (But only the hashes being fed
During the perl build Encode's enc2xs and a script in Unicode::Normalise
both hash sufficiently pathological data that the randomisation kicks in.
Unicode::Normalize seems to be doing something as innocent as hashing a large
list of integers.
It would not be impossible for mod_perl to have some real data that was
similar. But it seems unlikely.
My understanding of GVs may be failing me here. You keep mentioning
fetch_gv, but that's not part of the perl API. As far as I can see all
the functions in gv.c in perl call hv.c functions to do the lookups, and
hv.c functions now know when to rehash.
I'm going to get the mod_perl source after I send this message.
Three independent experiments you may wish to try before resorting to gdb are
0: Run with PERL_HASH_SEED=0 (to nullify randomisation)
If this works, then it has to be the conditional randomisation.
1: Place a return; immediately after
/* Pick your policy for "hashing isn't working" here: */
This will disable the randomisation
#define HV_MAX_LENGTH_BEFORE_SPLIT 4
in hv.c from 4 to (say) 16 to attempt to put off randomisation as long
as possible (in case something in the GVs or elsewhere is triggering