develooper Front page | perl.perl5.porters | Postings from July 2013

NWCLARK TPF grant report #95

Nicholas Clark
July 26, 2013 14:07
NWCLARK TPF grant report #95
Message ID:
[Hours]		[Activity]
2013/06/26	Wednesday
 3.25		failures under -DPERL_GLOBAL_STRUCT_PRIVATE

2013/06/27	Thursday
 0.25		Regexp::Grammars
 4.00		toke.c heredoc at EOF
 0.50		what does "deprecated" mean?

2013/06/28	Friday
 1.00		APIs
 0.25		Porting/
 0.25		RT #118653
 0.25		reading/responding to list mail
 0.25		utils/

2013/06/30	Sunday
 0.75		VMS-Filespec and known_extensions

Which I calculate is 10.75 hours

A quiet week, because we were visiting my parents. It was only planned to be
partly a holiday, but no plan survives contact with the enemy (or good

As the network at my parents is whatever we bring with us, I concentrated on
things that could be done locally. George's clang smoker* had been showing
failures for configurations with -Accflags=-DPERL_GLOBAL_STRUCT_PRIVATE when
built with with clang's address sanitizer. PERL_GLOBAL_STRUCT_PRIVATE is a
build option intended for extremist embedding - not just no global
variables, but even the variable used to hold the address of the structure
wraps the globals is itself hidden behind a function.

I had thought that the problems were fundamentally insoluble, due to
conflicting requirements between freeing that structure within global
destruction, versus code needing to look into it (to get the thread local
context) in the routine that called perl_destruct(). However, it turned out
that there is no fundamental conflict. The "use after free" error was
actually more obscure than that - it was actually code run by atexit() after
main() returns which was the problem, and the code wasn't defensive enough
to cope. As that code was just checking for a flag, the fix was as simple as
setting a variable to NULL after calling free(), and adding a NULL check in
the routine called by atexit().

Other problems that ASAN reported were mostly caused as a side effect of how
PERL_GLOBAL_STRUCT_PRIVATE is the only configuration that allocates storage
for the globals using malloc(). Every other configuration has the globals as
actual globals (either individual variables, or a structure which is
global), which results in them being zero-initialised. Hence the setup code
for PERL_GLOBAL_STRUCT_PRIVATE needs to zero a couple more globals.

The final problem it revealed *wasn't* specific to
PERL_GLOBAL_STRUCT_PRIVATE, but we hadn't noticed it before on any other
configuration with the existing test cases. There had been a long standing
bug that perl didn't cope correctly with a here-doc at the end of the script
without a final newline (RT #65838). The fix for this in some cases could
end up reading from free()d memory, if a particular buffer needed to be
resized. However, the code itself is only run if the Perl program ends with
a heredoc (which is an unusual structure), and if the last line of the file
on disk has no terminating newline character (which is also unusual as many
editors default to adding a final newline). Hence it's pretty rare to hit

Nicholas Clark

*;O=D Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About