develooper Front page | perl.perl5.porters | Postings from January 2018

Captured lexicals stolen from outer scope's pad

Thread Next
Paul "LeoNerd" Evans
January 22, 2018 03:29
Captured lexicals stolen from outer scope's pad
Message ID:
Hrm, that subject line sounds like a scary tale from a local
newspaper. Anyhow, my tale involves a long explanation of observing
current behaviour of 5.26.1 (so no ancient history today), and some
questions on why and what workarounds I can do about it.

I have some code which does, at the toplevel of a .pm file, the

  my @POWERDOWN_TO_NAME = qw( normal 1k 100k 500k );


  sub read_config
     sub { ... $POWERDOWN_TO_NAME[$idx] };

(The code can be seen around

Of the various named functions in this .pm file, this is in fact the
only one which captures this particular lexical.

I observe a curiously odd fact when I memory dump a perl 5.26.1 process
running this program. This fact is that while both the file-level scope
and the `read_config` CV have a pad slot named `@POWERDOWN_TO_NAME`,
they contain different SVs as values. Only the named function's pad slot
contains a real AV with those 4 elements in it. The file scope's AV
contains an empty AV:

((this output slightly truncated to delete irrelevant fields))

pmat> show 0x559073ab47d0
CODE(PP) at 0x559073ab47d0 with refcount 1
  named as &Device::Chip::MCP4725::read_config
  scope=CODE() at 0x559073ad4348
  pad[0]=PAD(11) at 0x559073ac50f0

pmat> show 0x559073ac50f0   ((i.e. the pad of &read_config))
PAD(11) at 0x559073ac50f0 with refcount 1
  [  4/@POWERDOWN_TO_NAME]=ARRAY(4) at 0x559073ab4560

pmat> elems 0x559073ab4560
  [0] SCALAR(PV) at 0x559073ac4f70 = "normal"
((i.e. this is the real array's expected value))

pmat> show 0x559073ad4348  ((i.e. the scope of &read_config))
CODE() at 0x559073ad4348 with refcount 4
  pad[0]=PAD(19) at 0x559073399db0

pmat> show 0x559073399db0  ((i.e. the pad of the scope of &read_config))
PAD(19) at 0x559073399db0 with refcount 1
  [  3/@POWERDOWN_TO_NAME]=ARRAY(0) at 0x559073adad00

A noticeably different AV, with this time no elements in it. Devel::MAT
hasn't been able to capture any more detail about it than that but some
other debugging reveals that this AV will look like

  SV = PVAV(0x560c1b7a1400) at 0x560c1bcbf330
    REFCNT = 1
    ARRAY = 0x0
    FILL = -1
    MAX = -1
    FLAGS = (REAL)
((the addresses differ here from the pmat output because this was a
different instance of the running program))

This particular arrangement confuses me, but doesn't seem to upset perl
itself. Perl is happy to run this function and all works out just fine.

However, when I try to rewrite this code into Future::AsyncAwait, this
causes some difficulty.

For various reasons inside F::AA I call cv_clone() on the named CV
invoking an `await` expression, so that I can have a clean copy to
modify and operate on. When that inside perl calls S_cv_clone_pad that
gets to the following line of code:

  if (!outpad || !(sv = outpad[PARENT_PAD_INDEX(namesv)])

which (because the lexical is still marked with PadnameOUTER()) wants
to walk a step up the CvOUTSIDE chain and grab the SV from its parent
pad instead. In my case with my program above, that causes the
newly-cloned copy of the running sub to grab a copy of this empty
stale AV, not the live one containing 4 elements that the running
function has. Understandably, when the running code resumes from its
`await` and tries to access elements of this captured lexical, it
doesn't find any, and gets upset.

So, on to my questions:

 1) Why does perl decide to steal the AV from the outside's pad and
    replace it there with an empty stale AV?

    I looked around the source code but I couldn't find a place that
    happened. It occurs to me it couldn't possibly do that if two named
    functions closed over the same lexical, as they wouldn't be shared
    any more, so the theft must be happening at some point after
    compilation, once it's found to be safe to do that.

 2) What can I do to either
    a) Stop it doing this, or
    b) Cause it not to be a problem for cv_clone()?

    Having not found the code (see question 1) I haven't been able to
    work out how to make it not do that. I figured that if I could
    convince perl that there are (or will be) other CVs that will close
    over the same lexicals, it won't do that (see above point).

    Alternatively, just before I call cv_clone() I could just walk the
    padlist looking for PadnameOUTER slots whose SV pointers in the pad
    and the corresponding outside slot don't agree, and put them back
    into place again. But not knowing the reasons why perl does this in
    the first place, I wasn't sure if that would be safe.

Paul "LeoNerd" Evans      |  |

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About