develooper Front page | perl.perl5.porters | Postings from March 2012

NWCLARK TPF grant report #25

Nicholas Clark
March 29, 2012 04:21
NWCLARK TPF grant report #25
Message ID:
[Hours]		[Activity]
2012/02/20	Monday
 0.75		arrays
 1.50		continue
 0.50		process, scalability, mentoring
 1.50		reading/responding to list mail

2012/02/21	Tuesday
 0.50		clang warnings
 1.50		process, scalability, mentoring
 1.00		reading/responding to list mail

2012/02/22	Wednesday
 0.75		RT #37033, RT #111070
 2.00		clang warnings
 2.75		perlfunc

2012/02/23	Thursday
 8.00		RT #37033

2012/02/24	Friday
 2.00		RT #37033
 0.25		RT #37033 related tidy up

2012/02/25	Saturday
 1.75		perl.c init functions

2012/02/26	Sunday
 2.00		-e
 2.00		t/run/*.t

Which I calculate is 28.75 hours

Sorry for the delay in these reports - I've been ill, been away, been ill
while away, and generally disrupted. Also, once everything started to get
back to normal there were unfortunately several things that needed doing
more urgently than catching up on the report backlog.

The biggest single activity of the week was diagnosing and resolving
ticket #37033. The low bug number reveals that this is quite a long standing
bug, related to the parser leaving file handles open in some cases.
The background is that the perl interpreter is either passed a filename,
or if none is passed defaults to reading from STDIN. If reading from STDIN,
the interpreter should not close it, as it did not open it (and implicitly
closing STDIN out from underneath a program is bad). But any file handle the
interpreter opened (internally, as a side effect of doing its job) should be

The problem was the the check for close-or-not used to be simply
"is this filehandle STDIN?"

The problem is the interaction of the definition of STDIN with the POSIX
semantics of open. "is this STDIN?" is pretty much "is this file handle open
on file descriptor 0?". open gives you the lowest unused file descriptor.
So if the user program closes STDIN, then the next file handle that is opened
meets the "is this STDIN?" test. And the parser was mistaking this for the
situation where it had defaulted to reading the program from STDIN because
no filename was passed in, so it was leaving the file handle open. Of course,
this is a leak. However, it was also starting to show up as visibly buggy
behaviour in code that manipulated STDIN - close STDIN, do some "innocent"
calculation, (re)open STDIN, only it's not on file descriptor 0 now - what's

What changed? Karl's work in improving Unicode semantics means that several
ops now need look up certain Unicode properties in some cases. If these
aren't cached yet, there's a call back into the parser as part of the
loading routines. And if that call into the parser happened while STDIN was
closed, oh dear, come close time the parser used to thing that had been
defaulting to reading from STDIN, and left the handle open.

The parser now explicitly tracks what it opened.

A side effect of tracking all this down was that I started to look at the
code in perl.c that handles the parsing of the command line options and
the initialisation they trigger, along with the the handling of the
"script file name", the "-e" option and the default to STDIN. Some of the
cleanup has been committed to blead and will be in 5.16.0. Other parts are
in the branches nicholas/RT37033-followup and smoke-me/kick-FAKE_BIT_BUCKET
However it remains a work in progress - right now perl needs to open
/dev/null as part of -e handling, because of assumptions in the parser.
However, I think I can see roughly how to make a small change to the source
filter code that would permit -e to avoid needing to open any file handle.

Nicholas Clark Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About