future of maint

Front page | perl.perl5.porters | Postings from October 2009

Thread Next

From:

Dave Mitchell

Date:

October 8, 2009 15:48

Subject:

future of maint

Message ID:

20091008224832.GC18637@iabyn.com

I believe that maint-5.10 in its current form is unsustainable
(or to put it another way, I'm not prepared to sustain it in its current
form, and I doubt that anyone else would be stupid enough to), so I'd like
to start a debate about its future.

Some random facts:

Over about 18 months, 5.10.1 included about 2800 commits that were
cherry-picked from blead, (and there were a further 700 or so blead
commits that weren't picked)

Maint currently serves the following goals:

1) include regression fixes
2) include general fixes
3) include build fixes (so it will compile on new/updated platforms)
4) include newer CPAN modules related to (3)
5) generally update CPAN modules where newer releases are available
6) include new features that aren't too scary and don't break
   backwards compatibility
7) maintain binary compatibility
8) don't break anything.

Doing all the above is *very* hard.

So we need to decide which of above goals we wish to keep.

I think this will be highly influenced by how often blead gets released
(as a real release, not a devel release). When it was 3+ years there was
pressure to get things into maint as a way of getting it to the world. I'm
going to assume that from now onwards blead gets released once per year or
so, so that pressure is removed.

One serious possibility is to scrap maint altogether. This is essentially
what the Linux kernel did a few years back.

Another is just the critical fixes (ie (1) above). So, we release 5.10.0,
then a month or two later out comes 5.10.1 with most of the things we
broke in 5.10.0 fixed. Then a few subsequent releases as a few remaining
regressions are spotted, and/or critical security patches are released,
with frequency of releases tapering off.

However, (3) is also important; CPAN developers often want to test
their code against older perls, but often they won't build on their
current h/w, which is why it would be nice to produce new versions with
all the latest Configure, hints/ etc changes. However, the build
toolchain also includes a whole bunch of dual-life CPAN modules like
MakeMaker, so they would probably need to be pulled in too. And my
experience with 5.10.1 of getting all those modules in consistent state
that play with each other was exceedingly unpleasant. You also run into
things like the great post-5.10.1 lib/ext->ext/cpan/dist migration, which
puts a great kibosh on cherry-picking the bits you need without pulling in
the entire migration, plus all the other updated CPAN modules etc.

If we don't include all the newer CPAN releases, then over time we'll be
releasing maint with old crusty CPAN modules, and users and module authors
will rightly castigate us for shipping old broken crud. But pulling in all
the latest CPAN releases, while making sure we're synced with CPAN and that
everything works is really, really hard.  That was basically what the last
three months before 5.10.1 was released were taken up with.

Maintaining binary compatibility is really, really hard. You have to be
totally on your guard, since any core commit could potentially break
it. Remember above I said 2800 cherry-picks? That's a lot of land mines to
be swept. I really think BinCompat is a luxury we can no longer afford to
offer, unless maint releases really do just consist of a few critical fixes
here and there. Removing binCompat would hit OSes that bundle and
auto-update perl, like Redhat / Fedora for example. If Fedora X bundled
5.10.0, then it could never be automatically updated to 5.10.1 via yum,
since it could break a whole bunch of XS code that the end user had compiled.
They'd have to wait for fedora X+1 to be released, update, then recompile
everything.

One thing I want to make clear here is that I don't think git is a magic
bullet. Doing clever reorganisations of branches (more feature branches,
branches per CPAN modules etc) might make the paperwork associated with
being maint pumpking slightly easier, but won't make a significant dent to
the overall skill or time required. (For example you still need to read
and understand everything on p5p to decide whether to merge things or
not).

One item of trivia that I think would make maint slightly easier would
be to have just a single perldelta file that covers all releases in the
same branch, and that just prepends new info to the file, e.g.
perl510delta.pod might contain:

    =head1 Changes in 5.10.3
    ...
    =head1 Changes in 5.10.2
    ...
    =head1 Changes in 5.10.1
    ...
    =head1 Changes in 5.10.0
    ...

This would remove a whole bunch of extra work related to adding new delta
files and updating all the configuration. I also think the perldelta file
should become a lot more austere, just summarising a few significant
changes rather than listing every fixed bug, every updated module, every
new warning message etc. It's just too much work to maintain such a
magnum opus. With the intention of development releases once per month, I
think this would be important there too.

Finally my stance on continuing as maint pumpking is that I'm happy to
relinquish the role if someone else wants it, and that I'm not prepared to
continue with it unless its agreed that maint will be something very
minimal (e.g. one or two cherry-picks per week).  [ Or unless some
commercial entity regards maint in its current form as important enough
for them to fund me at my usual consultancy rate for a day or two per week
on a long-term basis.]

Dave.

-- 
The Enterprise successfully ferries an alien VIP from one place to another
without serious incident.
    -- Things That Never Happen in "Star Trek" #7

Thread Next