NWCLARK TPF grant report #87

[Hours]		[Activity]
2013/04/29	Monday
 0.75		RT #114502
 0.25		RT #117327
 4.75		TAP::Parser
 1.25		genpacksizetables.pl
=====
 7.00

2013/04/30	Tuesday
 2.25		RT #116989
 1.50		RT #117501 (Open64 compiler)
 1.00		RT #117687
 0.50		TAP::Parser (CPAN #84939)
 1.50		reading/responding to list mail
=====
 6.75

2013/05/02	Thursday
 2.25		RT #117031
 1.00		RT #117835
 1.50		TAP::Parser (CPAN #84939)
=====
 4.75

2013/05/03	Friday
 0.50		RT #113920
 1.00		RT #116943
 1.00		RT #116989
 0.25		RT #117031
 0.50		TAP::Parser (Win32 failure)
 3.25		pp_pack.c
 2.00		pp_pack.c (BYTEORDER)
 0.75		reading/responding to list mail
=====
 9.25

2013/05/04	Saturday
 3.00		pp_pack.c (hton* and htov*)
=====
 3.00

2013/05/05	Sunday
 0.25		RT #117835
 1.25		pp_pack.c (hton* and htov*)
=====
 1.50

Which I calculate is 32.25 hours

I worked on lots of different bugs this week. Some were more fun than
others. RT #116989, last week's "favourite", had a final sting in its
tail. We thought we had a solution. I make a smoke-me branch to see what the
various test machines would make of it. Everything was happy except for
*Darwin*'s flavour of g++ 4.2.1, which objected to using "-ofilename",
petulantly demanding a space in between.  It's just special. It's
particularly special as the the FreeBSD g++ 4.2.1 is happy with or without a
space. Same compiler, same reported version, but clearly one vendor patched
theirs. Well *at least* one vendor :-(

So another round of testing with a space in, and finally everyone was happy
and it was merged to blead, in time for v5.18.0.

I also looked at TAP::Parser. No-one pounced on my suggestion in a previous
report on where to look for memory savings, so I followed it up myself. My
guess was roughly correct - it uses an array for each test to store the
numbers of the passing subtests. So for the usual case of all the subtests
passing (in the right order, and matching the plan), this is an array [1
.. $plan]. Fortunately TAP::Parser is very nicely factored, with all the
code using accessor functions instead of poking into the internals, so it
was very easy to alter it to use an alternative storage representation, and
hide this change from *everything* else just by changing the accessors
functions to "unwrap" the optimisation.

While working with the code I tried out a second approach to reducing memory
usage, albeit somewhat "unwarranted chumminess with the [...]
implementation" - forcing the test numbers to be stored as numbers. The test
lines are parsed by a regular expression, hence the captured results are
internally strings. The grammar uses qr/\d+/, so these strings can only be
numbers, hence by adding 0, the value stored is an IV, not a PV, which is
smaller.

Combined, these dramatically cut memory use (about 90%). The bad news is that
this change didn't make it in time for v5.18.0 - it's really not the thing you
change during the pre-release code freeze. The good new is that it's already
on CPAN, you you can upgrade all your installations, if you so wish. Including
your older installed Perl versions.

Nicholas Clark

• NWCLARK TPF grant report #87 by Nicholas Clark