develooper Front page | perl.perl5.porters | Postings from July 2013

NWCLARK TPF grant report #90

Nicholas Clark
July 26, 2013 13:50
NWCLARK TPF grant report #90
Message ID:
[Hours]		[Activity]
2013/05/20	Monday
 0.25		RT #116098
 0.25		RT #117209
 0.25		RT #118055
 0.25		TAP::Parser
 1.00		fakethr.h
 1.25		pp_pack.c

2013/05/21	Tuesday
 7.50		RT #118055 sparc32 use64bitint
 0.25		makedepend.SH, x2p/Makefile.SH
 0.50		reading/responding to list mail
 0.50		smoke-me branches

2013/05/22	Wednesday
 1.25		RT #118055 sparc32 use64bitint
 0.50		diag.t/regen/embed
 2.50		reading/responding to list mail
 0.50		smoke-me branches

2013/05/23	Thursday
 1.50		HvFILL
 3.25		RT #118055 sparc32 use64bitint
 0.25		SelfLoader (CPAN #85572)

2013/05/24	Friday
 1.75		DG/UX
 0.50		RT #118055
 0.25		RT #118121
 2.50		RT #118157
 0.25		process, scalability, mentoring
 4.75		reading/responding to list mail

2013/05/25	Saturday
 0.00		RT #118055
 0.50		RT #118157
 1.50		perl5db

2013/05/26	Sunday
 0.50		RT #118055

Which I calculate is 34.25 hours

This week I finally removed fakethr.h, and eliminated all references to it
and FAKE_THREADS. fakethr.h and FAKE_THREADS were for a "green" threads
implementation of 5005threads. 5005threads itself is long gone, and it's not
clear that -DFAKE_THREADS *ever* built correctly. Certainly it did not work
for the 5.005 release, and it did not work at the time of the commits for
the initial checkin. The closest that it seems to have been to working is
around commit c6ee37c52f2ca9e5 (Dec 1997), where the headers no longer
contained errors, but perl.c failed to compile.

I had spotted this dead code a couple of years ago, but stalled on the
problem that ExtUtils::MakeMaker contained a hard-coded list of headers
which it uses to generate dependency rules for XS code, and fakethr.h was
one of them. Hence simply innocently removing the file will have the less
than desirable effect of causing all XS code to fail. (It's very easy to
break most of CPAN. It's a lot harder to *not* break CPAN)

Manually removing another line felt like a bodge, as it doesn't actually fix
the real problem, so with plenty of other things to make progress on, I took
no action. Fortunately as part of his hash work, Yves fixed the header list
in ExtUtils::Makemaker properly (in a way that had eluded me), such that
there is no longer list to maintain (and get stale). Hence, with that
pre-requisite unblocked, I carefully purged the code once v5.18.0 shipped.
So that's the last part of 5.005 threads removed. That is, until the next
part is found.

This week I also worked on RT #118055. For v5.18.0 Father Chrysostomos
significantly re-wrote the OP generation code to use a slab allocation
scheme, which solves many previous problems with memory leaks when syntax
errors cause compilation to abort. (The "blame" here seems to be one of
those subtle problems of using code in ways it wasn't originally
intended. yacc doesn't complicate its design by making efforts to clean up
on compilation failure, because it assumes that compilation failure will
mean process exit, and everything gets freed anyway. This holds true for the
main Perl program too - syntax errors abort. But Perl 5 gained the ability
to be embedded (where total cleanup matters), and is now often used as a
persistent process (where even the occasional failed eval will start to add
up), so the cost of the problem has slowly been growing.)

However, there was an unfortunate bug slab allocator none of us spotted.
We were actually being naughty with alignment - the slab allocator assumes
that everything in the OP structure only needs alignment with the platform's
pointer size, but because PMOP was using an IV when it should have be using a
pointer-sized integer this didn't always hold true.

The problem was only relevant on 32 bit platforms when optionally compiling
with 64 bit IVs and ithreads (neither of which is the default), and the only
platform we're aware of that enforces the alignment is sparc32.
Frustratingly it was only spotted (by the Debian Perl team) just after
v5.18.0 was released, else it would have been a release blocker.

Ideally we'd fix the header to change the structure to use the correct type
but that's obviously not viable for a maintenance fix, as it's not binary
compatible. Having chewed over various ways to work around the problem, all
of which felt complex, fragile or both and had the potential to be slower, I
spotted a different approach which was simple and could be constrained to
just the class of affected builds.

OP allocation cannot only be from slabs - for various reasons OPs need to be
allocated without a slab being available. Hence each OP already had a flag
saying whether it was from a slab, or had been allocated directly using
malloc(). malloc() will, by design, always return suitably aligned memory.
Hence the fix is for the affected configurations to conditionally compile in
code that intercepts allocations for PMOP, and always allocates them with
malloc(). It's 2 lines of C code, 2 lines of C pre-processor, and will be in

This week I removed the code to support DG/UX. DG/UX was a Unix sold by Data
General. The last release was in April 2001. It only runs on Data General's
own hardware.

It should have been removed before v5.18.0 but it got missed, and then
seemed an inappropriate thing to do during the code freeze. There is still a
lot of code for platforms we don't have the infrastructure to support. In
some cases we doubt that Perl even compiles natively. Specifically, if
you're using Perl on any of z/OS, Windows CE, NeXT, AmigaOS, DJGPP, NetWare
(natively), OS/2 or Plan 9, we'd really like help in getting Perl 5
buildable and regularly smoked. Otherwise it's not clear how long we can
keep considering those platforms as viable.

Nicholas Clark Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About