develooper Front page | perl.perl5.porters | Postings from March 2015


Thread Previous | Thread Next
March 7, 2015 00:15
Message ID:
khw on IRC suggested that I haven't been clear enough about my position on
how the experimental status of signatures influences the acceptability of
the signature op.  This message is an attempt to rectify that omission.
The issues here are the scope of experimental status, the visibility of
unstable features, and the choice of how we proceed within experiments.

Regarding scope, my understanding is that the experimental feature
is signature syntax.  That is, the syntactic tokens, their arrangment,
their placement, and how these represent semantic behaviour.  The present
debate is concerned with how signature behaviour is represented at the
op level.  Because the behaviour is perfectly mundane -- copying values
into lexical variables, arity checking, defaulting logic -- we already
have an established way of representing it in ops, and so I do not regard
the optrees generated by the signature parser as experimental.  I do
regard them as large, slow, and in need of optimisation, but optimising
them is a matter of op-level transformations, not of Perl language syntax.

Of course, optimisations can themselves be experimental.  The signature
op could be implemented with some explicit acceptance that it's
an experiment.  But in my view that would be a separate experiment,
not merely a new development in the existing experiment.  DaveM has
been explicit in taking a contrary view, that the signature op is a
mere implementation detail within the experiment of signature syntax.
This difference shouldn't be terribly significant in itself, but it's
tied up with our difference over the coupling between signature syntax
and the signature op.  The question of how to factor our experiments
shadows the question of how to factor the code.

Whether we have one experiment here or two is also tied up with views
about visibility.  The signature syntax per se is a very localised
concern, only affecting code that actually enables the new syntax by means
of the lexically-scoped feature flag.  In a perfectly modular fashion,
such code can readily interoperate with code that is not participating in
the experiment.  Code in other modules is never made aware of the use of
signature syntax, and has no need to be mindful of it.  (Other code can
detect the feature flag through %^H and its runtime shadow, but that's
generally not an issue.)  We should be well satisfied that our mechanisms
for module composability have matured to the point that we can comfortably
compose modules written in an experimental variant of the language with
non-experimental modules using stable versions of the language.

The signature op is not so cleanly localised.  A joy of having a heavily
reflective language is that the semantic optrees are not purely internal
to the interpreter, nor are they purely a form of code.  They are also
data, that gets passed across module boundaries.  I think it's a good
thing that CPAN modules can get involved at the op level; DaveM thinks
it's a mistake that should be rolled back; we have a rather fundamental
disagreement there.

If one accepts DaveM's position that ops are a purely internal
implementation detail, then in changing them there's no need for the
license offered by an experimental flag.  He's been explicit about
perceiving a total freedom to change the details of ops without any
concern for API stability.  That position implies that experimental
status is irrelevant.  In countering that position I necessarily also
have no regard for the present experimental status of signatures.
(Someone on IRC, I forget who, pointed out that we're mixing discussion
of the specific with the general, and that's true.  Our debate is not
well factored, and right now I don't see how to separate those two sides.)

If, on the other hand, we accept ops as data in a protocol, then the
existence of signature ops immediately becomes part of the protocol.
Any module that looks at ops created elsewhere may see them.  Such a
module is not necessarily aware of the experiment (whichever experiment
it is), and can have signature ops thrust upon it without having opted
in to receive them.  So on this point, the isolation between modules
that is a prerequisite to their composability breaks down.

Some people have pointed out that the signature ops only appear if
one is using the experimental feature of signature syntax.  For this
treatment of the experimental feature flag to have its intended effect,
we have to view the flag as much more global than it ostensibly is.
Apparently the view is that for any part of a program to enable the
experimental feature means that it has opted the *whole program* into
the experiment.  It may in some cases be necessary to use a feature flag
that way, but it's rather Perl4ish.  It's not a natural interpretation
of an ostensibly lexically-scoped flag, and I don't think it's what we
envisioned for a syntactic experiment.  So I am not reassured by this
use of the feature flag.

To be fair, in this system the impact of signature ops would initially
be ameliorated by the sheer rarity of signature-using code.  Although the
categorical module boundaries are broken down, there would in practice be
few signature ops flying around, due to the experimental status.  Also,
the class of modules that inspect ops generated by the outermost part
of subroutine syntax is quite limited.  One could expect interactions
between the two to be rare.

The above two paragraphs refer specifically to what DaveM has initially
produced, with the "fake signature" cpp flag off, so that signature ops
are only generated by signature syntax.  DaveM has expressed some intent
for signature ops to be generated in other situations too, a move which
I have also pressed for on the architectural principle of separating the
syntactic and semantic layers.  Any version that generates signature ops
without signature syntax incurs to a much greater extent the problems of
novel ops being thrust upon unwilling code.  Ops generated by compilation
of non-signature code are relatively likely to be inspected, need to be
understood, and need to be correctly modified.  The problem could still
be limited by the use of some feature flag, but as with the signature
syntax feature flag it would be controlling the generation of the ops
rather than where they end up being seen.  It's still effectively a
whole program opting in together.

The visibility of the new op type is further raised by the possibility of
non-core code generating it, which is an inherent part of our op-based
APIs.  Obviously any code that does this is tacitly taking part in the
signature op experiment, if the op type is so labelled, and is accepting
the stability issues that arise if, for example, the format of the op
structure is changed in a future version.  But as with other sources
of these ops, it means more opportunities for the ops to be passed to
unaware modules.

The final argument that keeps coming up with respect to the "experiment"
label is that it means we're free to change things arbitrarily,
and it's therefore fine to make this change.  It's true that we've
explicitly disclaimed stability of the experimental feature, so, if we
accept that the disclaimers extend to ops generated with it enabled,
from that point of view we are indeed allowed to make the core generate
arbitrarily strange ops.  We also have a general rule that we're allowed
to change the API between major versions, which can be applied regardless
of experimental status.  But I'm not arguing that we're not *permitted*
to make this change; I'm arguing that it's not *wise* to make this change.

We do change APIs with every major version, but we don't do it
arbitrarily, we do it with some sense that changes must be justified.
We break things when we're convinced that a change is necessary, and
for things that aren't pressing but we're convinced are beneficial to
change we have deprecation procedures to provide a measure of stability.
Experimental status shortcuts the deprecation process, by changing the
expected degree of stability.  Marking a new feature as experimental means
we can take a bit of a punt on things we're not totally sure are right.
But experimental status does not mean a suspension of our critical
faculties; it does not override our engineering judgement.  A bad change
is still bad if it happens in the context of an experimental feature.

There is, of course, a great deal of disagreement about the intrinsic
merits of the signature op, both the op itself and other aspects of
DaveM's patch.  Largely the disagreement is about which metrics should be
used to determine its merits.  DaveM judges it supremely by its runtime
performance; I weigh performance against architectural cleanliness,
valuing the latter quite highly.  I am of the firm opinion that the patch
as it exists so far is a bad idea, clearly the wrong direction for the
signature feature and for Perl in general.  I therefore oppose applying
this patch even if it is thoroughly covered in "experimental" markings.

There's a related suggestion that the signature op is OK because it is
only temporary.  This should not be confused with it being experimental.
As I said last year with respect to one of the proposals for signature
syntax, if we already know we're going to revert it and why then it's
not an experiment.  I can certainly countenance the use of a temporary
implementation that we intend to replace, but in this case the issue of
API visibility rears its head again.  Given the view that op types are
somewhat visible as part of a protocol around compilation, DaveM's patch
doesn't just provide a temporary implementation, it would be a temporary
API change.  That concept really doesn't work.  It means an API change,
which a bunch of relevant code needs to cope with as soon as it's in
any stable release.  Reverting the patch later doesn't undo the damage.

mst has also described (on IRC) a concern that temporary application might
get politically difficult to revert, if it would involve any degredation
in performance.  Along those lines I'm more concerned that we already
have a situation where anything we come up with will be judged by some
solely by comparison of performance against the signature op patch.
Actually applying the patch isn't needed for this to arise.


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About