develooper Front page | perl.perl5.porters | Postings from August 2001

SV: Implementing Callbacks (Was: RE: [PATCH] Adding callbacks to thecore)

Thread Previous | Thread Next
Arthur Bergman
August 22, 2001 13:07
SV: Implementing Callbacks (Was: RE: [PATCH] Adding callbacks to thecore)
Message ID:
> On Wed, 22 Aug 2001, Arthur Bergman wrote:
> > I do not know how many times I have to say this, the above code is
> > faulty, if the next callback gets disabled.
> Right, I should have said this:
>   Disable a previously registered callback.  If the callback is not
>   enabled, does nothing.  A callback may disable itself, but otherwise
>   this function should only be called outside of a callback in the main
>   thread.

And then we don't need the mutex.
> > I think this should be optimized for the first kind. And thus we
> > should optimize for dispatching not for adding or removing callbacks.
> > I understand that we should be flexible, but a 45% overhead instead of
> > a 6% overhead for calling an empty function is pretty wild. Note that
> > even if every callback after it isn't that much slower, you still have
> > the intial 45%.
> Believe it or not, but I actully tried to optimize for the first case.
> One thing I did NOT want to do is iterate over an array, because it's
> difficult to insert & remove elements from an array, and I didn't want to
> pay the cost of iterating over an array in which some callbacks aren't
> even enabled.  I'm not sure why using a linked list is so much slower...
> maybe it's a cache issue?  Or maybe it's the comparisons?

If we have an array of all enabled callbacks then we can just go over it and call them. Since we have already established that optimization should be on calling the cost of insertion or removal can be payed for.

> > I once again suggest an array which you just iter over calling every
> > callback. And no mutex. If the callback needs some kind of data from
> > another thread, I think the callback should take care of it's own
> > protection.
> The mutex isn't the problem here... read on.

So we are getting rid of the mutex?
> > Or we go the other way and make every repeating function notify that
> > it needs to be called and make everyone a oneoff event. The
> > combination of these two seems to create a non optimal solution.
> I see what you're getting at.  I've been down some of these roads, and
> here's what I came up with.
> First, I'll cover the goals that I had in mind with this project.
> 1. I want to come up with a more flexible replacement for pluggable
> runops.  In this so far I've failed because of the performance issues
> (which I cover below).
> 2. I want to be able to start a thread in the background to do some work,
> and have it call a Perl sub when it's done, passing it the result of its
> work.  The callback system I have right now can do that effectively... if
> slowly.
> Next, I'll cover the different implementation methods I've thought about.
> 1. Implementing callbacks as pseudo-signals.
> The way the current signal handler works is that when a signal happens,
> the signal handler ends up scanning an array of all possible signals to
> find out which one happened.  This is bad because most of them will not
> have happened.  In current perl it's OK because signals only happen once
> in a while.  But callbacks can be on constantly.
> Also, how would you add or remove pseudo-signal handlers at runtime?
> One pro to this method is that there is only one comparison to determine
> if an event has happened.
> 2. Having an array of callbacks, seperate from signals.
> Much like using pseudo-signals, but with the benefit that you only have to
> scan an array of callbacks, not signals too.  Same problem though, how
> would you add or remove callbacks?
> I suppose that one could just expand the array for new ones and not ever
> delete old ones; just clear their flags and never set them again.  Or just
> suck it up and splice the array when you need to delete.

Or keep track of freed one and reuse them.
> The main problem with these two methods is that you are scanning a list of
> flags, looking for "true", but you also cover disabled or non-pending
> events, which is a waste... theoretically.

Theoretically, we don't know until we benchmark. Scanning an array is pretty fast, look at how poll() is implmented.

> 3. Linked list of callbacks, seperate from signals.
> This is the (probably misguided) path I took.  My thinking is like this.
> We pay the same price in comparisons that you would with an array of
> callbacks seperate from signals.  But adding and removing callbacks
> becomes easier, and when you scan the list, you only scan callbacks that
> are active.  You don't waste time scanning inactive callbacks.
> But I think the issue is not that it's a linked list.  I think that the
> reason it's slow is the three comparisons.

I think the reason is the linked list and the assignment.
> Solution?  I think that I have to come up with some way to combine signals
> and callbacks, so that we get the speed that we have today (one comparison
> no matter what) and yet still be able to pass data from an asynchronous
> C function to the callback, or from a seperate thread to the callback.

Ah, but to do this we need a mutex, can't the callback fetch the data from somewhere else?

Asynchronous C function notifies callback should be run and stores data at known position, unlocks mutex.
Callback locks mutex and gets the data from known position-


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About