develooper Front page | perl.perl5.porters | Postings from September 2021

DAVEM TPF Grant#2 FINAL (Aug,Sep) 2021 report

Thread Next
Dave Mitchell
September 10, 2021 10:56
DAVEM TPF Grant#2 FINAL (Aug,Sep) 2021 report
Message ID:
This is my final report for my second TPF perl5 maintenance grant, which
has now come to an end. (The good news is that a new grant has already
been approved.)

Many thanks to my (now former) grant monitors for their endless +1's :-)

First: the 'what I did last month' part of this report covers all of
August and up to 10th September. I made a start on getting the build back
into shape by starting to look at things like failing smokes and 'TEST
-deparse' failures.

      0:30 fix Text::Tabs failing smoke
      4:13 fix up deparsing
      7:53 process p5p mailbox
     12:36 TOTAL (HH::MM)

Now a brief review of the whole grant.

The grant has been running since 2013, for a total of 3624.5 hours. The
biggest single items in terms of hours (apart from general responding to
p5p emails etc) have been:

     45:05 make optree functions in op.c non-recursive and/or non-leaky.
     53:33 make op_private flag definitions be auto-generated
     60:16 [perl #127999] Slowdown in split + list assign
     76:11 fix smoke issues
     76:38 [perl #127875] Blead breaks Scope::Upper
     82:00 re-implement OPpASSIGN_COMMON
    112:03 RT #131260 sprintf implementation
    148:11 RT#120692 Slow global pattern match with input from utf8
    239:24 #124156: death during unwinding causes crash

which has involved:

* Completely re-implementing the internals of how perl pushes, stores and
  pops scope state on the context stack, resulting better performance for
  entering and exiting scope (like for loops, subs, blocks etc), and also
  resulting in the fixing (and enabling further fixes) of scope-related
  bugs, especially with things like combinations of eval, DESTROY and the
  correct setting of @_.

* Fixing some performance problems when matching patterns against long
  utf8 strings, where old optimisations like "this pattern can't match
  further than N bytes from the end" actually made things worse, because a
  bunch of simple bits of code like C<max = (p - N)> had been converted
  en-masse to C<max = skip_back_N_utf8_chars(p,N)>, so to speak.

* Making most of the optree-walking functions that are called during
  compilation non-recursive. This means that deep optrees (e.g. those
  created by machine-generated code like $a1 ? $b1 : $a2 ?  $b2 : ...)
  no longer crash the compiler. The peephole optimiser still recurses
  sometimes though.

* Making the definitions for all the private flags in opcodes all be
  defined in one file which can be processed to auto-generate all the
  required header files, tables, etc used by the perl core and things like
  B::Concise. Previously these were just a bunch of code comments and
  separate manually-maintained entries, with a *lot* of inconsistencies.
  It also used to be really hard to work out what spare private bits, if
  any, were available for each op.

* Heavily rework list assignment to improve performance and eliminate
  bugs, especially related to 'common to both side' situations like
  @a = ($b, $a[0]) etc.

* Heavily rework perl's underlying sprintf implementation for better
  performance, while making the code more readable (its a 1500 line
  function) and reducing/eliminating likely buffer-overrun candidates.

"I do not resent criticism, even when, for the sake of emphasis,
it parts for the time with reality".
    -- Winston Churchill, House of Commons, 22nd Jan 1941.

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About