The evil Perl reads freed things

I'm part way through trying to rearrange the innards of ponie so that
the sv head also hides behind a PMC.

At the moment ponie in CVS has stopped interjecting PMCs between the head
and the body, and replaced the SV arena macros with functions.

What my checked out copies are trying to do is replace the SV head
arena with SV heads malloc() as needed (and free() when finished with)

My working copy uses the ptr_table hash tables sv.c uses for ithreads
cloning to track the allocated SV head pointers, so that all the visit
functions can still work. This seems to work fine, once I realised that
during global destruction I can't free the heads immediately that their
reference count hits zero, because other things may still refer to them.

However, I've hit a snag. What I think might be a big snag.
Perl seems to assume in other places that the head of a freed scalar is
still around. For example, test 13 of t/cmp/for.t

@a = (3,4);
eval { @a = () for (1,2,@a) };
print $@ =~ /Use of freed value in iteration/ ? "ok" : "not ok", " 13\n";


That croak error comes from this part of pp_hot.c:

    if (sv && SvREFCNT(sv) == 0) { 
        *itersvp = Nullsv; 
        Perl_croak(aTHX_ "Use of freed value in iteration"); 
    } 


and that SvREFCNT() is looking at the reference count of a freed head,
which is now in the arena, waiting to be reused.
Only now I'm really deallocating the memory associated with the head,
instead of leaving it in the arena, with valgrind I see:

==18619== Invalid read of size 4
==18619==    at 0x81438A3: Perl_pp_iter (pp_hot.c:1900)
==18619==    by 0x8124973: Perl_runops_debug (dump.c:1563)
==18619==    by 0x80CD1F5: S_run_body (perl.c:1741)
==18619==    by 0x80CCCFE: perl_run (perl.c:1659)
==18619==  Address 0x42857F0C is 4 bytes inside a block of size 16 free'd
==18619==    at 0x4002839A: free (vg_replace_malloc.c:186)
==18619==    by 0x8148C07: S_del_SV (sv.c:206)
==18619==    by 0x8154EF7: Perl_sv_free2 (sv.c:6071)
==18619==    by 0x813A224: Perl_av_clear (av.c:494)

(and without valgrind the test fails, presumably because that memory has
already been overwritten and the "reference count" isn't zero)

Other tests fail in strange ways, and valgrind reports errors which are
reads in the heads of now-free scalars.

So, I'm not sure what to do, as this assumption that you can read any
head even after it's undead seems to be quite deeply ingrained. And it
needs to be solved before we can usefully replace heads with PMCs.

Nicholas Clark

• The evil Perl reads freed things by Nicholas Clark
• Re: The evil Perl reads freed things by Nicholas Clark
• Re: The evil Perl reads freed things by Tim Bunce
• Re: The evil Perl reads freed things by Nicholas Clark