develooper Front page | perl.perl5.porters | Postings from October 2012

Re: What happened to the whole "small core" idea?

Thread Previous | Thread Next
Peter Rabbitson
October 28, 2012 08:59
Re: What happened to the whole "small core" idea?
Message ID:
On Sat, Oct 27, 2012 at 06:53:14PM +0200, Aristotle Pagaltzis wrote:
> * Peter Rabbitson <> [2012-10-27 10:35]:
> > It has been 4 years, and still precious time is spent discussing how
> > to cram yet another *very* controversial and complex piece of syntax
> > directly into the core.
> Ironically, the controversy over this feature is generated by people who
> think the syntax proposed is *not comprehensive enough*. There has been
> no dispute whatsoever over how the syntax should look for the amount of
> functionality that is in the proposal, and only one point of dispute
> over how it should work (read-only alias vs copying), which resolved to
> consensus in due course.
> Funny, isn’t it?

It isn't funny, because it isn't true. I am not talking about the little 
scuffle you, doy and the rest had over this minor-ish detail. When I say 
"controversial" I mean "something that is not yet ready to be supported 
by a plain `perl -E ...`"

> Of all the complaints about insufficient functionality, only one has
> turned out to uncover a serious oversight which requires addressing:
> what to do in case of a mismatch in the number of expected vs passed
> arguments. (That actually throws the feature in jeopardy in my mind.
> I may have proposed a solution, and I would like to see it attempted,
> but due to this issue I am no longer certain of the feature. It may
> regrettably need even more time in the cooker.)
> > What happened to the “ship it on CPAN and see” mantra?
> It worked. There are about half a dozen signature modules on CPAN, all
> implementing variations of the same syntax. At which the point is the
> “and see” part satisfied? Seventeen more variations?

A variation which can be lifted *verbatim* from CPAN and moved into the 
core. For an example see Class::C3 => mro 'c3'. Or to put it another way 
- I will be satisfied at a point where you don't end up saying "ugh... 
what do we do on @_ size mismatch...?!" while you are creating a 
smoke-me branch. If you do not see the problem with such timing - I do 
not know what to say.

> > What happened to making perl smaller and less complex?
> It’s happening. The trend has been toward kicking out things, and FC has
> done a lot of work to expose hooks into the language at various levels.
> New features have been much more conservatively scoped and everyone is
> a lot more wary of adding huge new things. We have RJBS at the helm now,
> whose instincts I trust.
> But this is not always going to be the direction you seem to think of as
> an absolute. As Larry once remarked about Lisp, it is the most beautiful
> language in the world and every program written in it looks like oatmeal
> with nail clippings. Someone has to pay for simplicity and the language
> is not implicitly and by defintion the wrong place for that.

I am not saying "wrong" I am saying "last resort". See below.

> > Why after smartmatch and $_ localization and auto-deref (which has not
> > yet backfired, but it will), is this still happening?
> If my memory is still of service, there were roughly half a dozen fewer
> implementations of these things that shipped on CPAN than of modules
> that implement signatures. That is, zero of either.
> Personally I hated smartmatch the second it was proposed; I don’t know
> if I would use it even in Perl 6, and I never thought it had any place
> whatsoever in Perl 5.
> I also hated auto-deref, almost certainly for the same reasons as you
> would list here if you did. But I also think it partially addresses
> a real problem in the wrong way. I.e. I do believe there is a problem
> that ought to be solved, so even though I consider auto-deref a mistake,
> there is conceivably other, more fundamental syntax that I would not
> oppose – unlike you, presumably. So far I have not seen a proposal that
> seemed to fit the language, though.

I have a lot to say about this, but I do not want to derail the thread. 
Ask me at a conference some day, London maybe? Or heck - maybe finally
come to your monthly local .pm meets ;)

> Maybe all precedents are not the same.
> Do you hate the addition of the /r flag for s///, for example? (I love
> it, personally. I use it all the time – even more than I expected.) If
> not, why not?

I do not hate it, as it is in essence an op. Hating s///r would be like 
hating say(). Do I use it? - no, because I plan to support 5.8 as long 
as practical. Would I vote against its inclusion if we had a voting 
mechanism for these things? - no, I would simply abstain. Yet I want to 
point out again - we are not talking about a proposal to shorten the 
optree. We are talking about a feature with immense impact on @_, one of 
the very fundamentals of function calls in Perl.
> Use your discrimination.
> > Let's focus on sub signatures for a bit. I've read every thread about
> > them. Peter Martin's work kicks ass. The speedups are tangible which
> > is even more awesome. Yet the whole proposal seems to be set up “It
> > either happens in core or it doesn't happen at all”. Why?
> Why indeed? Why does it seem that way? After all it has happened on CPAN
> a half-dozen times. So where is your impression coming from that that is
> not the case?

I answered this further above, and in my email to Peter. However I will 
reiterate - there has never been an implementation of this on CPAN that 
mucked with @_ in such a radical way.

> > What prevents the core from exposing the correct-level hooks, have
> > *these* hooks unit tested, and relegate the syntax extension mucking
> > to CPAN? I do not think performance is an issue – after all linked
> > C is linked C.
> You make it sound so easy. How is XS supposed to add a new op type that
> skips populating @_? How do you implement compile-time checking for the
> number of arguments with syntactic override?

Erm... as if you are doing that here? I mean compile-time checks. How 
exactly are you checking foo(@stuff) at compile time again...? In any 
case - this is all a strawman - I specifically said - it is *perfectly 
fine* for the VM to expose *hooks* to do this properly from within XS. I 
never said it will be easier, nor faster nor even prettier. All I said 
is that this is what in my opinion needs to happen.

> How would you have implemented the /r flag of s/// on CPAN?
> Of course it’s all possible *somehow*, even if that means swapping out
> half of core for code copy-pasted from core into a module and tweaked
> there, like Devel::Declare used to be.

I never advocated this. See below.

> Or I suppose if you put enough smarts into it you could instead design
> a decomposition of the current architecture that provides enough hooks
> into what happens during e.g. a sub call to allow implementing in XS
> even things down to the level of not populating @_. That would get
> closer to your vision.
> But what is the evidence that doing such API design work in an abstract
> vacuum rather than as part of the implementation of a concrete feature
> will yield a design that actually covers the needs of real feature
> implementations?

?!?!?!?!? Where did I say the work has to happen in a vacuum? All I am 
saying is that it needs to happen *before* or *simulataneously with* the 
creation of the first such feature that needs them. I have *zero 
problem* with having perl 5.18 ship with enough stuff exposed on the 
perl API level to make a CPANized Sub::Signature::Shiny possible. The 
problem is that by not doing this work separately, but lumping it all 
into core, you are in effect blessing a particular *signature behavior 
and syntax* to be "the new perl way". And please do not try to claim, it 
is easy and inexpensive to change ones mind once a feature like this 

> Needless to say, it is far more complex to design an interface on top
> of which features can be implemented, than to design features. (I don’t
> have my Mythical Man-Month here, else I’d include here a quote about the
> tripling? of complexity? effort? required that Brooks pointed out.) Of
> course you have to find someone who is motivated to do this demanding
> work as an obstacle on the way to the feature they actually care about.
> And then those interfaces have to be supported forever.

As opposed to a possibly-deficient syntax which has to be supported 
forever-ish? Are you *seriously* saying it is harder to backout a 
perl-API change than it is to rip out a part of the *published* 

> There’s a parallel to the microkernel vs Linux argument in there too.
> > Maintainability can't be a problem either – if anything it will be
> > awesome to have a well factored boundary that will make a performant
> > CPAN module for such extensions possible.
> Of course it will be awesome. And ponies are awesome too. :-)

Well - so (awesome) are also function signatures. The burden of 
procuring extra insurance should lie on the person proposing the change, 
not on the programmers that got sucked into maintaining the overall 
language structure. I totally agree with you, that we are dealing with a 
scarce resource of contributions. What I disagree with is what is 
acceptable in such a situation. See below.

> > Hence why I particularly lament the lack of movement towards Jesse's
> > vision.
> I think you conveniently forgot the part where the very motivation for
> Jesse’s vision was his belief that the language has to evolve to stay
> relevant, and his desire to be able to break things while *adding* or
> changing syntax, without that entailing massive backcompat fallout. He
> was *not* advocating for language standstill.
> Here you are, holding up his proposal as the end of language change.


> Furthermore, he derived a concrete proposal from that, which personally
> I have criticised on numerous occasions as ill thought out on technical
> grounds. I see no way to follow through on it without a *geometric*
> increase in *language* complexity over time.

There is nothing wrong with *internal* complexity. In fact it is 
*inevitable*. I was specifically talking about *surface language* 
complexity. What perl looks like to *outsiders*, not to porters. The 
extra complexity e.g. of having to explain how the behavior of @_ 
changes depending on what pragmas are in place, potentially in an 
*unrelated* scope. Even worse - we are talking about shipping this as 
5.18 while there *isn't yet a single porter* who can explicitly list all 
the possibilities of interactions of the new sigs with legacy code in 
non-trivial calling hierarchies. I fail to understand how this can be 
acceptable to you, or to anyone else for that matter.

> Here you are, holding up his proposal as the road to simplicity.
> Topsy turvy world. :-)
> Practice has borne out my concern too, at least in the case of moving
> smartmatch to the CPAN, which was a tentative suggestion of his as an
> as an example for what he was imagining. The idea was ditched when no
> one could figure out a way to allow competing implementations to coexist
> in the same interpreter process, let alone with predictably composing
> semantics. It would have been a recipe for even more psychosis, not
> sanity – which to me appeared obvious from the start.

No one "could" or no one "cared to" once it became more and more obvious 
that ~~ is nearing being a bad idea as a whole? I am not sure this is an 
unsolvable problem on technicla grounds. It may very well be unsolvable 
on tuit-grounds, but for that - see below.

> > IMNSHO Perl5 does not need *any* extra *syntax* from here on out.
> IMequallyNHO it does. But it needs syntax that addresses shortcomings at
> the right level with solutions that fit the language, rather than myopic
> to the point of blindness local tweaks to single bits of the language,
> which fail to consider its entirety (e.g. auto-deref).

The severity of myopia is variable across any population. Without a case 
study of this feature all we have are dueling levels of technical and 
design experience attempting to predict what lies behind a particular 
singularity. This is no way to judge the impact and fitness of new 
features. For that one needs a study. Such a study has never taken 
place, as the feature has never appeared on CPAN in anything even 
resembling its current form.

> > In fact it stopped needing it around 5.6-ish. No new syntax unless
> > there is a massive benefit in adding it, and even then only if it can
> > not be done via a CPAN-distributed extension. Besides Reini's work on
> > strict types, nothing else currently comes to my mind as a candidate
> > for “this *has* to be in core”.
> There is a design principle called “sane defaults”.
> > “Even though EVERYONE agrees on the core syntax and EVERYONE wants it”[3]
> If I go count the number of old threads and patches on p5p plus the
> number of supportive participants in them, will that plus the half dozen
> signature implementations on CPAN make any impression on you? I am
> willing to do the work if that’s what it takes to convince you that this
> is a feature with, if not literally singularly universal (as you seem to
> interpret my words), then at least extremely widespread support.

Please do. I am sure I am *by far* not the only one on this list who 
will *tremendously* benefit from such a "resume".

> And if core devs disagree, how many other voices do theirs weigh up
> against?

My real complaint in the OP was about the invisibility of such voices. I 
am pretty sure that if there was anything resembling a "round table" you 
would not be asking "how many" - it would have been obvious. Of course 
neither of us can be proven right or wrong until the situation 
improves... if ever.

> Ultimately, of course, the pumpking decides. (Or Larry as the highest
> instance, obviously, when Larry does make a call on something in Perl 5
> any more.)
> > And the worst part? - it appears that nobody wants the fucking thing
> > except for a very very small but very very vocal minority.
> My experience has been that nobody wants 5.10.
> People who upgraded to 5.10 have kept on upgrading further for the most
> part. Everyone who didn’t has stuck with 5.8.
> IMO this is yet another indictment of the disaster that was Perl 5.10
> and the catastrophe that was the p5p process of that era. Fortunately
> those are now in the past.

I do not agree with this observation, as you probably gathered by now. 
In my eye the problem comes to this: given limited developer resources 
does one sacrifice development of new features, or does one sacrifice 
the syntactical coherence of the language as a whole? I do totally 
recognize that having this cake and eating it is not an option, because 
it violates the "limited developer resources" part. So, unless (in this 
case) Peter or a likeminded programmer is interested in starting a 
rudimentary perl API extension to make this feature CPAN-able, there are 
only two choices left: delay the feature *until* the former is possible, 
or close ones eyes and let it slip in the name of language evolution.

I know where do I (firmly) stand, and I know where do you stand (with 
equal resolve). The problem is that I do not know where the rest of p5p 
stands, nor does Peter, nor anyone else. The porters as a collective 
"take it as it comes". I find this status quo problematic. Enough so to 
speak up about it.

Outside the scope of this particular proposal, looking at the larger 
picture - what are your general thoughts?


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About