develooper Front page | perl.perl5.porters | Postings from October 2007

Re: [perl #46987] OO-call failures, autoviv-functions & testing their existence

Thread Previous | Thread Next
October 30, 2007 06:54
Re: [perl #46987] OO-call failures, autoviv-functions & testing their existence
Message ID:
On 10/30/07, Linda W <> wrote:
> This is probably too long a response, but tried to answer all of the
> points brought up.  Sigh.  Focus on forests not trees?
> demerphq wrote:
> > Referencing a subroutine is referencing a slot in a glob in the symbol
> > table. Symbol tables are hashes. Therefore this behavior is expected.
> > The fact that it results in behavior that you find suprising is more a
> > reflection of your (mis)understanding of how subroutine calls and
> > method calls work in a dyanmic language than it is an error in Perls
> > behaviour.
> ---
>         Your explanation is based in Perl's internals.  A bug in
> Perl's internals implementation is not supporting evidence for
> why documented OO calling functionality should be broken.

Its not a bug. Perl behaving the way you dont expect is not a bug.
Perl behaving contrary to the way its documented to work is a bug.
Perl in this case is behaving exactly as it was designed to work. Ergo
its not a bug.

>      It isn't the "reference" that is causing the problem -- though
> you are focusing on it as such because you are focusing on compile time.
> OO-method calling is *dynamic*.  What is the right thing to do when the call
> is made?

Er, no. Im not. I dont understand how you come to this conclusion.

>      It is *only* at execution time that this "situation" can be resolved.
> It is at execution time, that Perl knows whether or not \&boom is defined
> locally.  At execution time, I assert that Perl should look through
> Parent methods for a valid method, and give priority to a defined
> parental method OVER DYING of a FATAL ERROR saying "the "method'
> is not defined".

No, it shouldnt. Because there are no 'valid methods' in perl. There
are just entries in the symbol table. Its that simple.

>      A second problem is that the UNIVERSAL "can()" method is broken
> as well.  According to OO-Perl(Conway), "can only returnes true if an
> object or class can call the method requested".  If the local method
> is "undefined" (as is stated in the error message), then "can()"
> cannot return a reference to that function -- since that function method
> is NOT legally callable.

Yes, this is a consequence of dealing with a dynamic language. If you
dont like it use a different language. Conway glossed over some of the
stickier details in that quote, probably because he didnt want to
confuse things, but Perl has to deal with AUTOLOAD and other dynamic

> >
> >>         I don't think it is the same.  Part of OO functionality --
> >> a MAJOR part, is that if the "method" is not defined local, they it
> >> checks the parent classes.
> >
> > Taking a reference to a subroutine does not involved method dispatch
> > and is NOT documented to take a reference to a parents subroutine.
> > Therefore it isnt a bug.
> ----
>         I was not trying to take a reference to the parent's subroutine.

Well thats sure what your code and your original comments made it look like.

> I am taking a reference to a routine who's location I do not know.  I wanted
> to see if the reference returned "NULL" (or was undefined), or if the reference
> was 'valid'.  I wasn't "blindly" expecting the reference to work.  I wanted
> to see some indication that it wasn't defined, so I wouldn't call it.

So, you decided that by taking a reference to a subroutine it must
return undef if that subroutine wasnt defined. But thats not how
things work.

And you know ill just comment here: You seem to really like using
non-perl terminology in the context of Perl discussion, and you don't
seem to listen when people tell you not to and correct you. This just
makes it hard to help you and indicates that you have gaps in your
perl foundation knowledge which I think just add up to you thinking
things should work in ways that they just don't.

>         The problem is that Perl isn't behaving as defined for OO-calls.

How many times do people have to tell you that it IS behaving as
defined before you will believe them and consider that it is your
understanding that is flawed and not Perl's behaviour?

>         Instead you go off justifying the bad-behavior based on Perl's
> internals.

Umm. No. I merely explain how Perl works. You can try to rationalize
the disconnect between your flawed mental model and Perls actual
behaviour all you wish but its not going to get you anywhere.

>I understand that the perl internals won't return null or undef
> as a "value" when checking the result of \&undefined_routine.  I understand
> that currently, Perl blindly calls a "reference-created" stub.  I'm saying that
> under OO rules, it should not.

Whose OO rules? Yours? C++'es? Java's? Smalltalks? Pythons? Whichever
rules you are speaking of aren't Perl's. There are no apriori OO
rules. They are defined in a particular context. There may be general
conventions that a dispatch mechanism is expected to abide by in order
to be considered to be OO by the general programming public but the
details are entirely within the remit of the language designer.

>Nor should "can" return "true" for methods that are "undefined".

Says you. Graham bar says otherwise. Guess which one most people will listen to?

>         Justifying the current behavior based on Perl's internals doesn't
> mean it isn't a bug.

Well as other people have pointed out it is intentional, and
documented behaviour, so its neither justified by Perl's internals nor
is it a bug.

>         While you said: "that you find suprising is more a
> reflection of your (mis)understanding of how subroutine calls and
> method calls work in a dyanmic language than it is an error in Perls
> behaviour." -- based on how Perl is internally implemented, you have to
> realize, that the converse can be said of you:
>         The fact that you claim non-OO behavior is "desirable" because it is
> compatible with non-OO hash autovivification, is not justifying the bug, but
> may be  --  moroe of a reflection of your (mis)understanding of how OO calls
> are to give *priority* to parent methods OVER throwing a "not found" error.

No. Wrong. That error is there for a reason. The reason being that
Perl allows methods to be created on the fly. If Perl has been told
that a method will be supplied later and the method is not suppled
later the ONLY sane behaviour is to die with an error. Maybe the
autoload routine is broken in some way. Perl cant know whats wrong,
all it knows is that something *is* wrong. Under such a scenario
passing through to a parent method would be wrong, as it may be that
the parent method will do something inappropriate and maybe even

>         I may not be an expert in Perl internals, but the domain of correct
> OO behavior supersedes incorrect function based on language quirks due to
> implementation.

No sorry. Thats just wrong. As i said earlier, there are only a few
behaviours that are generally considered necessary for a dispatch
mechanism to be considered to be OO. The details are necessarily left
up to the language designer.

>         I do not disagree (I agree) that if the call was "funkymethod($self)"
> throwing the error is desired behavior.  I'm saying that Perl has to be
> smart enough to know the difference between "funkymethod($self) and
> "$self->funkymethod", and follow the OO method of searching parent methods
> *before* it throws an error for a non-existent, local method.

It wasnt non-existent. You created it. Perl has no way to know that
you did it accidentally and didnt mean to do it.

> >> But using the method-call syntax "$self->method", it should look through
> >> the parent classes for the method *before* throwing the undefined-error.
> >
> > No, you created a subroutine stub in the symbol table.
> ====
>      A subroutine stub != a method call.  The
> subroutine call wouldn't do OO lookups.  \&func, shouldn't take
> precedence in an OO call.  If I had written "\&$self->method", then I
> _might_ agree the local symbol should be given preference -- but
> \&func isn't a valid way (as many have emphasized to me) to access
> a method name, so it should not be used as a reason for
> "$self->method", an OO call, to use such a "non-OO" reference
> to 'override' a valid parent method.

Look, Perl doesn't work the way you think it does or should. Either
get used to it or don't use it.  But repeating over and over incorrect
assumptions is going to get you nowhere.

> > in the symbol table. Perl has to
> > assume
> ----
>         At execution time, if the local symbol is "undefined", then it
> should look for a valid, defined symbol.  If I had defined the local
> function, then YES, I would expect Perl to override it -- but the function
> isn't *FULLY* defined (the body is missing).  It's a "stub".  Can you
> give 1 way in which calling an empty stub is preferable to looking through
> the parent classes first (that is not based in perl internals or how hard
> it is to get *correct*).


The assumption is that if there is a subroutine stub its expected to
be filled in later by autoload. If autoload doesnt do so then its an
error in the code. Doing anything other than throwing an error would
not make sense.

> > A method call is supposed to look at each entry in the current
> > package, failing finding one, it is supposed to look in  the parents
> > classes.
> ----
>         This is where we differ.  I claim method calls look for
> methods.  You are stuck in Perl's internals talking about 'entries'.
> OO-parlance doesn't talk about implementation-specific things like
> 'entries'.

You are stuck in insisting that there is some ideal form of OO that
all languages must comply with, it isnt so, has never been so and
never will be so.

> >  You created one,
> ---
>      I created a non-existent forward-function reference -- not a
> valid method.  "\&foo" is not an "OO" supported way of referencing
> or creating a "method".

Argh. There is no way for perl to know if its a valid method or not.
There is no difference between methods and functions in perl. Its that

> albeit by an unusual route, and Perl found
> > it. You are muddying the water by bringing up method dispatch to
> > somehow make your incorrect autovivification seem less at fault here.
> ---
>         Autovivification vivifies *function* *stubs*, not OO-methods.
> AutoV. is a perl construct, not an OO construct.  In a compiler, the
> bogus reference would fail at compile time as the referenced "local function"
> would not exist.   In perl, because of dynamic binding, the final
> check for existence isn't done until execution time at the point the call
> is made.  When the call is made, OO->method should give OO-method lookup
> preference over perl's implementation specific _function_ autovivification
> OVER a   *non-existent* function reference created by a non-OO reference.
> Certainly can we agree that \&foo is not a supported OO way of accessing
> methods?  OO gets away from implementation specific details and using
> "addressof" type operators.

No it shouldnt. It should assume that doing anything is inappropriate
and should die, and golly gee, thats exactly what it does.

> > I mean, computers dont normally ignore things that programmers tell
> > them to do. When they do its generally considered a bug. You cant turn
> > around and say that in this case perl should know you didnt really
> > mean for it do something so it should just pretend you didnt.
> ---
>         If Perl had only 1 call mechanism, or if \&func was a
> supported, or proper "OO" way to access methods, I would agree, but
> it wasn't.  I'm saying that \&func isn't "OO" and that "OO" shouldn't
> give preference to "Throwing a Fatal Error", over doing the parent
> method search.


>         Are you saying, at execution time, that when I have "$s->method"
> perl has no idea that this is an OO call that is *different* from "func($s)"?

Of course it knows the difference. One should use method dispatch and
one should use direct dispatch. What im saying is that it doesnt know
the difference between a sub or a method. The only difference is how
it finds the sub. One does what is essentially a hash lookup in only
the current package and the other does a recursive depth first search
through the packages listed in @ISA until it finds a package where the
hash lookup returns something that looks like a subroutine.

> I had several OO-conversant people emphasize that even though methods
> and functions were *similar*, they were not the same and should not
> be treated as such.  If I (or you) agree with that -- then it makes sense
> that "func($s)" is a normal function call and "$s->method" is used
> for OO calls, you can have it both ways.
>         As for perl "doing what I am telling it to do" -- that's from
> your "internal" based perspective -- it is not the perspective of
> an OO implementation.  From the OO perspective -- I would expect that
> \&undef_func would NOT override OO base-class searching because it is
> not an OO construct.  I understand why you believe it the same from
> an internal, standpoint, but I don't feel internal implementation should
> be the "decider" of what is "desired" or what is correct from an
> OO standpoint, since OO was defined before perl ever existed.

OO has changed over time. It doesnt have a definition as you seem to
think, it has a generally accepted set of requirements, which as far
as i know perl satisfies. Thats about it.

> >>
> >>         Well this is the solution others come up with -- but no one
> >> even knew "why" until I reported the bug.  It's confusing and not
> >> behaving as the OO-language lookup is supposed to.
> >
> > Well you were told several times that what you were doing was wrong.
> ---
>         Sorry, I don't accept "wrong" on faith.  I've always to
> know why.  In this case, "why" is because the internally-focus implementation
> produces incorrect behavior when viewed from an OO perspective.

No, viewed from YOUR perspective of OO, which appears to be heavily
influenced by some other languages implementation. However that
perspective of OO at the very least seems to not to take into account
the consequences of Perl being a dynamic language and therefore is out
of sync with the reality.

> >
> > Thats nice. Too bad it doesn't match how it actually works, and given
> > that you don't seem to understand the deeper implications of working
> > in a dynamic language I don't think you are really in a good position
> > to determine how one should work.
> ---
>         I should not have to be a perl-internals expert to write
> in OO.  If one has experience in OO, one might have some
> pre-existing ideas.  Such might be more so if one read a generally-
> -accepted-as-good, OO-perl book. Can you point to an OO reference
> where it says that "Inherited-methods first search for undefined
> function stubs in an local-entry table"?

Does everything have to be spelled out so explicity? The documentation
explains exactly what Perl does, and it explains things like AUTOLOAD
and etc. If you put it together holistically these things are
implicitly explained.

> >
> >>  The user cannot
> >> rely on the built-in "can()" function to determine if the method is
> >> safe to call.  I tried to "protect" the calls -- but 'can' returns true
> >> if there has been a reference to the same-name, but undefined local
> >> function.  You can't use "defined($self->method)" as a replacement for
> >> $self->can('method');
> >
> > There are two issues here. First, you shouldnt create function stubs
> > when you dont intend to fill them in.
> ---
>         As I tried to explain multiple times -- the stubs *were*
> filled in.  The program had NO CLASSES -- all functions started out
> as "local".  Slowly, conceptually-functional areas were broken
> apart into more manageable packages/classes.  As long as all the calls
> were in the same "Package" (or class), it worked.  At some point,
> some of the "referenced" functions were moved from where the 'jump
> table' was used (as called from a Tk-keyboard-event handler).

It doesnt matter what your intentions were. It matters what the code did.

> > The fact that you did so by a
> > rather unusual mechanism is annoying I agree but is not wrong.
> ---
>         I excel in fitting parts together in novel ways. :^)
> >
> > Second, you cant generally rely on references to subroutines,
> ---
>         See above.
> > regardless how they are obtained, through can() or other means, to do
> > the same thing as named dispatch would, be it named method dispatch,
> > or named subroutine dispatch.
> ---
>         Yes...and I see your example, but the *very* str8forward
> translation from reference to named in doing something like
>      $self->"method"(params);
> Obviously wasn't going to work...:-)
>         In fact, the difference between "ref-based calling" and
> 'name-based' calling is, unfortunately one of the problems with most
> of the OO examples I find.  They all seem to use examples where
> "use strict;" is not used (something I tend to always use), and
> end up with examples like "$self->can(foo)" where it isn't
> immediately obvious that 'foo' is effectively treated as a string.

If you see that please flag it as a documentation bug.

> > I say 'generally' here for  a reason BTW, under controlled
> > circumstances you can rely on its behavior, but your situation wasn't
> > really what I would consider a controlled circumstance.
> ---
>         Well, it really was "out of control" considering I was
> going from a complete non-OO case to an OO case in order to
> make it more easily extensible.  But in testing -- if you want
> to find bugs, you can't just do the 'controlled' testing, but
> must also do the things that, as a developer, you know to steer
> around because you aren't sure, based on implementation, whether or
> not it will work.  Most programmers just find the things that work
> and stick with them.  If you want to find errors, you have to do
> the things you wouldn't normally do or think of (!try that in the
> morning before coffee!).
> >
> >>         I acknowledge that this is not as str8tforward a fix as one
> >> might like, but I believe this area is hit more often than the bug
> >> report numbers represent.  Too many perl-experts knew the workaround, but
> >> couldn't say WHY what I was doing failed.
> >
> > You had reams of code, riddled with red flags. In fact the code I
> > analysed was plain and simply wrong, and method dispatch really had
> > nothing to do with it.
> ---
>         Yes -- I had red-flags -- but before I posted the first bit
> of code, I'd gone through tons of variations -- some of which I didn't
> expect to work, but, since the things I expected to work, were not, I
> was trying things that I didn't expect to work, but hoped would
> generate information to point the direction.
> > Its no wonder no one hit on the exact reason
> > that would have satisfied you. If you had listened to the advice given
> > you instead of discounting it as superstitious then the problem would
> > have gone away.
> ---
> I wouldn't have hit on "can" not working.

No, you wouldnt have confused yourself into thinking that what can()
did do was incorrect.

>         You may claim that "can" not working, or that calling local
> functions, KNOWN to be UNDEFINED, should take precedence over searching
> base classes for matches, are not bugs.  But I disagree.  Maybe you
> can get verification from Damian Conway?  Doesn't he work on perl now
> and then?  Can he verify that in an Object Oriented framework, it is
> desirable to to give call-lookup precedence to a local, undefined function
> over valid parent methods and that is desired behavior that "can()"
> return true for "function stubs" which are not callable methods?

Er, im sure that if Damian Conway felt like getting involved in this
disucssion he would.

> > If you had managed to reduce it down to less than 10
> > lines of code *maybe* we would have hit on the explanation that would
> > have satisfied you, but sifting through 50 lines of code riddled with
> > red flags is no fun, especially when the person you are trying to help
> > argues with you about the red flags you point out.
> ---
>         If someone only wants their code to work and doesn't desire
> an understanding of "why" something is broken, I'm sure they won't
> argue with someone giving them an a workaround instead of an answer.
> Eventually I got the help I needed regarding the example you were looking
> for and added in the cases I was concerned with.  The resulting
> code was about 70 lines (or about 20 w/o debug statements showing
> the error at runtime).
>         Sorry, but I'd encountered too many different ways for this
> type of code to fail in too many variations to just accept a workaround.
> I wanted to know what was incorrect: my understanding of OO, a "massive
> braino disruption", misreading the various OO-perl examples and
> references, or an unexpected "bug" ("feature") that threw off how I
> expected things to work vs. how they actually did.
> > And its not a bug. And most people would have realized that and just
> > went "oh, ok, so that doesnt do what I expect" instead of "that should
> > do what I want it to do so it must be a bug because it doesn't".
> ---
>         I believe you are using a perl-implementation based reason as
> justification for the behavior.  I don't agree that in an OO-implementation,
> precedence should be given to a NON-OO reference (you agree it's not
> advisable to do in OO). In fact, I don't know of any OO implementations
> that depend on method addresses between classes -- IT BREAKS the OO paradigm.
> But agreeing that "\&func" is not OO, allows that the OO-specific call method
> should not rely on functions defined that way.  I also do not agree that
> the "can()" function should be returning "true" when a valid _method_ is
> not available (and only a function stub is).
>         Current implementation aside, do you really believe the current
> implementation ("can" unreliable, and local-undef-funcs supersede parent
> method calls) serves *any* valid or reasonable purpose.


> Can you put
> the current implementation aside and think or converse with OO
> experts (not other perl-internal experts). About the functionality
> as is and as suggested from a "functional" (within an Object Oriented
> Context) point of view?

If they are OO experts in non-dynamic languages then their opinion is
irrelevent. If they are OO experts in dynamic languages then their
opinion is probably much the same has been explained. They may quibble
of various aspects of how Perl works, but given the constraints
various language design decisions impose on the functioning I would
bet most open minded and well informed people would come to more or
less similar conclusions.

If you are going to have lazy loading of subroutines (we do) and if
you are going to have intrinsic type orthagonal to class (we do), if
you are going to allow methods and subroutines to be redefined during
normal program flow (we do) then there are various consequences. Its
not unusual for people to find certain aspects of this to be
unexpected and or undesirable, but thats the situation we are in.

Most languages choose to sacrifice some of these features, or to use
alternate implementations to avoid these problems impacting OO. That
fine, but Perls heritage is of a language that started out without any
support for OO and then added various 'neat' features over time. The
end result is that we have a somewhat loose but highly flexible OO
framework which occasionally surprises people who come from other
backgrounds. There is not much we can do about that without designing
a new language. (Which is being done in two separate projects

BTW, this is the last im going to contribute to this thread. Im sorry,
but i dont have time for it. Either get used to perls OO warts and
all, or switch languages. The alternative will just frustrate you and
annoy the rest of us.


perl -Mre=debug -e "/just|another|perl|hacker/"

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About