develooper Front page | perl.perl5.porters | Postings from February 2007

Re: Future Perl development

Nicholas Clark
February 3, 2007 07:38
Re: Future Perl development
Message ID:
On Fri, Feb 02, 2007 at 11:26:34PM +0100, Gerard Goossen wrote:

> and feedback. The low response reinforced my ideas about the current
> development of perl5.

I saw no reply to my question about it.

> I want a perl which evolves, which develops new features, which ideas
> are changing, on which can be experimented, new ideas tried, and old
> ideas can be replaced by new ideas. I don't want to be hold back by
> failed experiments, I want to use the insights of previous experiment
> to develop new things. Think of different garbage collection,
> continuations, monads, experiment with Object-Orientation, concurrency.
> Think of it as a natural language.

I doubt that much of this can be done with the current internals.

The C codebase is continuous evolution for the past 18 years. Perl 5 was not
a clean re-write of Perl 4, so there's cruft carried over.
(For the most extreme example, Perl_do_pipe, which is dead code from 4.036
that never got removed)

No-one fully knows the current internals. I don't know the regexp engine,
or that much about the parser. Others who do don't know about the things
I've worked on, such as string/numeric conversion, the SV data structures.
There's a lot of things that predate the current maintainers, and many lines
of code that are the (in)famous change 1

    Change 1 by mbeattie@localhost on 1997/03/28 13:17:33

            Perl 5.003 check-in

I know that there is complexity in the code, and cruft. The code has a
heritage of being extremely portable, and builds and tests fairly cleanly
just about anywhere. (And I suspect is more portable than most other
projects of this complexity, in particular most other scripting languages)

A large amount of the non-feedback we don't get is because it already works;
it already compiles and builds on most any platform you throw it at.

It would be nice to get the cruft out. But it's not clear what's cruft,
what's important portability code, and what's actually essential complexity.
There are parts of the Perl-space visible language implementation that are
caused by particular bits of the C implementation. People come to rely on
them. For example

$ perl -MDevel::Peek -e 'Dump $a--'
SV = NULL(0x0) at 0xf4338
  REFCNT = 1

$a is uninitialised when it is read for post-increment --, but for post-
increment ++:

$ perl -MDevel::Peek -e 'Dump $a++'
SV = IV(0xff76c) at 0xf4338
  REFCNT = 1
  IV = 0

it's initialised to 0. Which is batty and should go. But there are people
out there relying on that.

For example, Artur has investigated, and there is a fair amount of effort in
the internals that goes into supporting methods on bareword file handles. It
should be nice to get rid of that effort, which in turn would mean removing
that language side feature. But even if we did do that (and with a proper
depreciation cycle) I'm not confident that we could find all the code that
relates to it, and hence remove it all cleanly (without breaking other things
that we did not mean to break), so we would not gain the true efficiency
savings that should come with the pain of losing the feature.

> We can not do that without breaking the current perl.

This I feel is dangerous. Perl 5's killer app is CPAN. If the interpreter
starts to "mostly" support CPAN, but it's diverging, then there is no
incentive for firms using Perl to migrate from Perl 5 (as is). Witness
the cascade failures we've seen on CPAN when key dependencies (test modules)
change. If the core language becomes a moving target, and CPAN authors have
to run to stand still, it's going to detract from the advantages Perl has.

> My proposal to get this:
> We start with the current bleadperl. We remove some ugly
> things and add new ugly things. After a few months we call it perl7.
> perl7 is going to be maintained as usual, and code written for perl7

Maintained by whom?

Maintenance is not really a fun task, and we're already short of people
willing to volunteer to do it. It's not like there's anything I can see
that I can change to *make* it a fun task, and hence get more people to
volunteer to do it.

The only reason 5.8.8 came out was because I spent about a month of my
own time doing it (because I'd said that I would do it, and felt bad about
reneging on that commitment)
Again, 5.8.9 is being done in my own time - despite my best efforts, I can't
find a sane job in London that facilitates me working on the core.
(And I don't think that it's just me finding this hard)

The number of splits you're proposing exceeds the resources we have.

> keeps working on newer perl7 releases. Meanwhile we continue the

If Perl 7 is designed to become rapidly obsolete, why would anyone target
it or install it?

> This proposal, is only the outline, there are a lot of issues I didn't
> discuss, but they can come later. I would like some feedback on the
> big picture, how people feel about the option to continue developing
> in this direction.

I don't think that this approach is going to work. I think that attempting
to make major functional changes such as your suggesting is not going to
be possible with the current internals, and I don't think that it's viable
to attempt to start by attempting to simplify the current internals.

If I were starting again, there are bits of the internals I'd love to have
done differently. Specifically

1: Unicode would be designed in from the start (or more importantly, character
   sets would be abstracted)
2: Threading would be designed in from the start, rather than retrofitted.
3: Use vtables for the data access, rather than open structures which can
   be modified directly.

Right now, a lot of what we try to do is limited by the fact that the "API"
includes open access to the data elements of Perl's SV structures. Having
things abstracted behind vtables would make it easier to implement thread-
shared data (eg lock during read), move the polymorphism complexity from
the OP code and the SV/AV/HV code into the data types, and probably more
things I've not even thought of.

Problem is, it's faster to start again, than to try to get there from here.
And in turn, a lot of the special case code that is visible to Perl scripts
comes from particular parts of the existing C implementation. So whilst you
could write new backend and then a language "not entirely unlike" Perl 5,
it wouldn't be quite the real thing, inevitably "nutrimatic" Perl 5 wouldn't
quite run some important legacy code, so it wouldn't be good to migrate to.

Nicholas Clark Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About