develooper Front page | perl.perl5.porters | Postings from April 2021

(rfc) new feature pathways, documentation

Thread Next
B. Estrade
April 13, 2021 22:01
(rfc) new feature pathways, documentation
Message ID:
Let me know if something like the following exists. I am interested in 
helping to marshal whatever iterative *documentation* process might be 
involved in production some sort of thing that can be pointed to, in the 
vein of "so you want to add a feature..."

I must preface this by saying we're all adults here. At least I think 
so. I appreciate everyone's hard work and true kindness in tolerating my 
recent spate of mails and also pointing me in the right direction. None 
of the following is targeting individual efforts. And its all said out 
of appreciation for all that comes with perl/Perl and the community.

The motivation has come to me over the last 12 hours after, reading:

1. the trim PR on github
(thank you kind person who send me this link)
3. the infix 'isa' implementation
4. (Far More Than Everything 
You've Ever Wanted to Know about Prototypes in Perl - by T. Christiansen)
5. spending time "putting pen to paper" with the context switching thing 
since I can work out the semantics-context stuff out in perl "userland"

If there is anything that I am envisioning, I have never seen it; so I 
will describe it. If something of this nature does exists, I'd like to 
evaluate it and see if it's what I would like to see as a total n00b. I 
have "fresh" but not "totally ignorant" eyes - and though I have much 
ignorance, I will use this as a strength.

The sketch of the document I am envisioning essentially does the following:

I. Find Your Path

This section would attempt to help the reader to determine the end state 
of the "feature". E.g., options things like: CPAN, "dual life" (included 
in the perl distro), high level "core" feature (ala, 'trim', 'isa' noted 
above), perl "run time", perl API, data guts, other low level stuffs 
(I'd need help identifying this stack of turtles). I might use the term 
"perl userland" to mean "something a perl consumer might want direct 
access to in their scripting activities". "perl runtime" would mean what 
it does traditionally.

Some paths are related above, some are not. For example if the "goal" is 
a "dual life" module, you clearly need to get it on CPAN it needs to 
prove its worth. This document wouldn't cover how to "decide when to 
promote", only that a promotion is necessary; to be "dual life" one 
needs to be promoted from CPAN. And we know well, anyone can put 
anything on CPAN.

It would effective present a "decision" tree or a set of Socratic type 

* what functionality is desired?
* where does this functionality fit best (userland, runtime, "kernel", etc)?
* how does it look to use said functionality?
* does said functionality sit in a higher level, but require fundamental 
lower level support?
* what's the highest "level" on which this functionality can be implmented?
* how important is performance to your first iteration?
* really, how important is performance to your first iteration?
* ..
* ...

The goal: find the best place to implement the feature with the desired 
interface/semantics so that it can be proven a) unique, b) useful, c) 
compelling to use, etc. (more on the "goal" later).

II. Choose Your Weapon

This section assumes the interested party is sufficiently past step 1 
and would suggest things like combining "put it on CPAN" + the wisdom in 
#4 (prototypes and presenting natural feeling 'keywords'). In any case, 
I'd treat this section as completely as my current world view allows. 
And I can iterate with anyone interested. But the goal would be to 
create a high level perldoc - i.e., if nothing like this exists. If 
something like this exists, please show me and I will likely start there 
to achieve, in part, what I am envisioning here.

The point of the documentation is to establish a pathway. The point of 
the path way is to simultaneously a) find the right "place" for the 
feature implementation and b) more easily detect when lower level 
support of a feature is needed. The charge to add infix additions is a 
*perfect* example that points to extended support for prototypes (and 
possibly the 'override' mechanism).

Please let me know the current state of our documentation regarding 
this, I am committing to doing this if it would be beneficial - and 
guess what, it *doesn't* have to be a perldoc for me to be happy with 
it. I just want the document that I think would have really pulled me 
in. It just so happens I am very persistent. Or maddeningly hard headed; 
different people have vastly different opinions on this topic.

III. Efficiency && Efficacy

It is my estimation that the majority of things that have been 
laboriously baked into core or implemented in the way suggested by #2 
could have been done as: i) a module, ii) using prototypes as intended 
in #4, and iii) being patient, iv) humbly accepting the fact that no one 
thinks your function is as cool as you do.

Try::Tiny is clearly to me the best example of this of a success story. 
It is the platinum standard, IMO of how it should work. It started as a 
module. It unashamedly uses protoypes. And it proved to be one of the 
most popular modules in CPAN for reasons related to following the right 
path and being an actually useful thing. Look at how it's implemented. 
Moreover, consider the interface options. Until last night I zero idea 
that I could coerce a block into a subroutine ref, let alone an array of 

I must take a hard look at two other "things" that took IMO, the less 
righteous path; you know what I am going to say: 'trim' and 'isa'. Oh 
and 'say'. Of those: 'trim' and 'say' could have been modules leveraging 
prototypes "templating" ability. 'isa' "should have been" a module - but 
it could not because prototypes simply do not support the infix 
notation. To make this point *even* stronger, I looked slack jawed at 
was was required to implement 'trim' in the PR. Assuming this was 
necessary (I can't be the judge there); the complexity of the code was 
unbelievable; almost as if it was necessary to implement a regex parser 
inline. That this was required points more strongly to the failings of 
the perl API to implement something so conceptually simple. But is also 
clearly points to where 'trim' really belongs. In that place where one 
can easily reach for regular expressions.

I think we could avoid (and could have avoided) a tremendous amount 
"misfeatures" and "misappropriations" of the past if there was a clear 
and judicious path to adding features.

In summary:

0) The goal should always tend "up" in terms of where the implementation 
is. The lower level one goes, the less it should be changing. If a 
feature's "place" is clear, but that place doesn't provide sufficiently 
high level support, *THEN* is when you go under the covers. An example 
of this is #4 provides an outlet for things that should act like "perl 
keywords" (like trim) and clearly points to it being in a module. For 
"isa", #4 can't handle infix. So to me, that points to the notion that 
infix operators should be able to be handled by #4; but can't. So go 
under the covers and make it so it can be. E.g.,the natural way to 
extend #4 for infix would be simply to add a template "hook" on the LHS 
of the sub name + maybe even an extension to the overload module:

use overload
   '===' -> \&eqn;

sub ($) eqn ($) {
   my ($a, $b) = @_;
   return $something;

That feels nice and fell out of looking at a little bit of historical 
precedence. As a "template" it would be a violation to something like:

sub eqn ($ SOMESPECIALTHING $); # anti-pattern prototype

That is not a template, that is an incoherent declaration to what is 
actually wanted. So I ask, is it worth time to implement "pluggable 
infix" operators - or to extend prototype template coercion semantics to 
support a parameter list on the LHS of the new op? And all things 
considered equal, is it not consistent with what we already do to allow 
'overload' to take advantage? IMO, *this* is where the effort will be. I 
can't speak on the actual work involved, but I can absolutely foresee 
the additional benefit and whole host of new infix operators. In short, 
we now get to create all sorts of weird things on purpose:

What would this do?

if ($lhs_thing1 '(_8(|)' $rhs_hing2) {
   say 'doh!';

Or better, and I " really"x1000 hope the text spacing is preserved:

if ($lhs_thing1 q{__&__
                  /     \
                 |       |
                 |  (o)(o)
                 C   .---_)
                  | |.___|
                  |  \__/
                 /_____/ \
                /         \} $lhs_thing2) {
                 say 'doh!';

The enhanced ascii perl art alone is worth enabling this all by itself.

1) I find it is the "interface" that that people seem to care most 
about; some interfaces require lower level support than #4+CPAN can give 
-i.e., "perlish" feeling functions and keywords (e.g., try/tiny uses 
prototypes and this is really what demonstrated their power). IOW, most 
are looking for "creating [a] user-defined functions that behave in much 
the same way that Perl's own built-in functions behave with respect to 
their effects upon the parser and upon implicit contexts" - trim/tromp 
is most definitely in this camp. "isa" is not since it is 'infix' and 
now I see why LeoNerd is looking at creating "pluggable" infix - seems 
like what is really needed is an extension to prototypes in #4 to 
provide infix semantics. Maybe a good lead to follow.

2) Another goal of this process would be to shake out those with less 
honorable intents; be the pride, etc. (not accusing anyone here of 
that!!!!! to be clear.). But we are human beings and humans are biased 
by all sorts of things. If done right, we'll have a nice sieve in place 
for eventually and fairly determining where a "feature" in whatever form 
*actually* belongs. This saves everyone time, frustration, and mostly 
time. Best of all, it would greatly improve the "market testing" phase 
of features. It allows us to test more features in more perlish ways; 
and basically wherever the process "stops" for a "feature" is where it 
should stay. Probably CPAN; but the selected few could make it to dual 
life. Really interesting features might demand more low level "under the 
covers" support; etc, etc. Seems pretty harmonious to me.

Incidentally, #4 was a game changer for me; even without the other 
things I enumerated above. I also do wish they had just called 
prototypes, "parameter context templates". Maybe it's not too late to 
change the name; seems to reason they are not mutually exclusive with 
"function signatures" and are in fact much more "perlish" that the 
signature thing, which as far as I can tell has no way to do the "type" 
coercion that "parameter context templates" can. Additionally, 
"parameter context templates" is meant specifically to, "creating 
user-defined functions that behave in much the same way that Perl's own 
built-in functions behave with respect to their effects upon the parser 
and upon implicit contexts". So maybe the reason why "signatures" have 
not been made a default feature is because it seems to not pay proper 
respects to what "parameter context templates" are for. I am not 
prepared to do an analysis on prototypes vs signatures right now, but I 
have started to think about it. Signatures necessary need to be 
compatible with prototypes, even if that condition alone is not 
sufficient to conclude that saga.

If you made it this far, you probably forgot that this was really about 
documenting a sane and humane pathway for features. Let's return to that 
and I am ready to get to work on that.

Thank you,

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About