develooper Front page | perl.perl5.porters | Postings from May 2015

Premature freeing with "Non-eval closures don't need CvOUTSIDE"

Thread Next
From:
Christian Jaeger
Date:
May 6, 2015 05:31
Subject:
Premature freeing with "Non-eval closures don't need CvOUTSIDE"
Message ID:
5549a735.c2bbb40a.7674.1b57@mx.google.com
Hi

TLDR: coderefs captured by closures can't be weaken'ed anymore without being collected too early.

I've started using a coding style that uses local self-recursive functions and at the same time returns closures to delay evaluation. Since making a (local) function visible to itself creates a cycle, Scalar::Util's "weaken" needs to be used. This by itself works fine. But when capturing the weakened variable in yet another closure, while this still works in v5.14.2 it does not in v5.20.2 or bleadperl; the latter collect the reference, which leads to an error for attempting to call undef as a function.

    sub foo {
        my $f; $f= sub {
            my ($n)= @_;
            sub {
                if ($n > 0) {
                    $n + &{&$f($n - 1)} # $f is undef here
                } else {
                    0
                }
            }
        };
        my $f_= $f; weaken $f;
        &$f_;
    }

    my $res= &{foo 2};

In case the above came out badly formatted, see:

 https://github.com/pflanze/functional-perl/blob/master/bugs/perl/weaken-coderef-simplified

or a longer version which also checks whether leaking occurs:

 https://github.com/pflanze/functional-perl/blob/master/bugs/perl/weaken-coderef


I've bisected this down to the following commit:

 commit a0d2bbd5c47035a4f7369e4fddd46b502764d86e
 Author: Father Chrysostomos <sprout@cpan.org>
 Date:   Wed Jun 20 14:23:02 2012 -0700

     [perl #89544] Non-eval closures don-F¢t need CvOUTSIDE-A

     A closure doesn-F¢t need an outside pointer at run time, unless it has a-A
     string eval in it.  CvOUTSIDE is only used at compilation time to look
     up variables by name.

     Since CvOUTSIDE is reference-counted, a closure can unnecessarily hang
     on to variables it is not using (see the test in the diff).  So stop
     setting it when cloning a closure, unless it is needed for eval.

 :100644 100644 96308a2... 072ff1e... M  cv.h
 :100644 100644 468ba6c... 0ab4f5e... M  pad.c
 :100644 100644 a241d91... 7fdb829... M  t/op/closure.t

(I haven't managed to revert this commit in current bleadperl (it seems there is also at least one patch with fixes on top of this one).)

Currently the functional-perl code base contains a hack [*] that simply doesn't 'weaken' code references in newer Perls, but this does leak (not severely enough to trigger any of functional-perl's leak tests, which are designed to detect other leaks, but it will definitely be a problem in some cases when using a functional coding style).

[*] https://github.com/pflanze/functional-perl/commit/8afaf3ba7fa782cc6c1015fb467a9d05f74b53e8#diff-4f461f7cbffab7251d6b8a01bcf2d5b3R92


Here's the test that "Non-eval closures don't need CvOUTSIDE" introduces:

    # [perl #89544]
    {
       sub trace::DESTROY {
           push @trace::trace, "destroyed";
       }

       my $outer2 = sub {
           my $a = bless \my $dummy, trace::;

           my $outer = sub {
               my $b;
               my $inner = sub {
                   undef $b;
               };

               $a;

               $inner
           };

           $outer->()
       };

       my $inner = $outer2->();
       is "@trace::trace", "destroyed",
          'closures only close over named variables, not entire subs';
    }

It seems that the purpose of the patch is to avoid retaining references to variables that are not used within the closure (during closure creation), supposedly by collecting the names of variables used within the closure's body at compile time and then only embed references to those into the closure. This is certainly a welcome change. But why does that lead to $f becoming undef in my test case? $f is visible, after all. Does perhaps the search for variables skip embedded closure bodies?

Thanks,
Christian.

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About