Front page | perl.loop |
Postings from April 2008
Re: benchmarking various event loops with and without anyevent
Thread Previous
|
Thread Next
From:
Rocco Caputo
Date:
April 28, 2008 01:37
Subject:
Re: benchmarking various event loops with and without anyevent
On Apr 27, 2008, at 00:56, Marc Lehmann wrote:
> On Sat, Apr 26, 2008 at 08:13:27AM -0400, Rocco Caputo <rcaputo@pobox.com
> > wrote:
>> each event watcher. Anyone who knows POE can tell you this is one of
>> the least efficient designs possible.
>
> It is the only design that I could get working, even after
> consulting a
> few people and implementing some workarounds for the bugs in POE.
>
> In any case, you have to consider that most people on this planet
> don't know POE, and even if, they don't know it that well. Since the
> documentation for POE is in such a bad state, thats the obvious way
> to fix
> that.
Most people on the planet don't know Perl, or even how to program a
computer. No amount of documentation will help them. :)
In the future, you may wish to include me on your list of people to
consult about designing applications for POE. I may known a thing or
two about the topic. :)
Are you aware that I'm gradually rewriting POE's documentation? If
you could describe what you don't like in a useful way, I may be able
to do something about it.
>> In fact, this design is worse than the average for first-time POE
>> users.
>
> If a better design is possible, it is not known to me, and you haven't
> suggested one either, so talk is cheap. I'd be happy to get a more
> efficient design for POE but nobody could come up with one that also
> worked reliably through multiple iterations of run and also does not
> keep
> the POE kernel from returning.
I would love to have the opportunity to suggest a different design,
but most of my discretionary time is spent addressing the constant
misunderstandings between us. If we can first resolve them, I'll have
that much more time to work on the design.
Obviously I cannot expect you to know everything about POE. Likewise,
you cannot expect me to magically know when you started writing
AnyEvent::Impl::POE. Even if you announced it somewhere, I may not
have been looking. The first I heard of it was here, when you
announced your benchmarks.
In general, if you need someone's attention online, the most effective
and polite way is to contact them directly. If you did contact me
about this, then I'm sorry that I missed it. Are you sure your
message wasn't lost in transit?
As for the design:
First-time POE users tend to design programs where the number of
sessions scales linearly with the number of objects that handle
events. If S(1) is the total overhead imposed by a single session,
then S(N) is the overhead imposed by the average naïve POE user. N is
the number of objects handling events.
AnyEvent::Impl::POE creates a new POE::Session for every event
watcher. It's not uncommon for an object to use more than one event
watcher (I/O and timeout, for example). So we can model the session
overhead in an AnyEvent::Impl::POE program as S(N*M), where N is the
number of objects handling events, and M is the average number of
event watchers per object.
Assuming that N is the same between the equivalent POE and
AnyEvent::Impl::POE program:
S(N*M) > S(N) for M > 1.
QED :P
> If a better design *is* possible (which I don't really doubt), then it
> needs to be vastly more complex, or it needs some non-obvious trick.
> I can
> only imagine making some very complex on-demand instantiating and re-
> check
> wether the session still exists on each watcher creation.
Your imagination comes up with such incredible things. Don't lose
that. :)
The "trick" is to minimize the number of sessions used. Your
benchmarks and comments in your documentation implied that you knew
that sessions imposed overhead. I wrongly assumed the solution would
be obvious.
In your case, I would create a single persistent POE::Session instance
that serviced all the watchers. The watchers themselves would be
small proxies that controlled POE::Kernel watchers within that
session's context.
I use this design in POE::Stage. If you're not averse to looking at
experimental code, you can find it on the CPAN. It also does a lot of
other, unrelated things, so you may have difficulty separating the
magic you need from the voodoo you don't. Comments are welcome, if
they're useful.
> (It is possible that I was fooled by the docs as well, so if there
> is a
> better way, it likely isn't documented, so who could blame me).
I won't blame you. But I will point out that your documentation says
you're already familiar with using undocumented POE features. :)
> Instead of going around and accuse people of making bad designs, it
> would
> be much better to improve the documentation for POE, so that said
> people
> don't have to go around and start guessing...
I don't appreciate your vague, negative comments about the
documentation. Please describe problems in a useful way, or your
expectation that they be fixed is unreasonable.
You seem to be saying that your design for AnyEvent::Impl::POE is
based on guesswork. If so, I overestimated your knowledge of POE
based on your documentation. Therefore, please consider the chance
that you may know less about POE than your documentation implies.
> Oh, and while you fix the docs, could you fix the other bugs as well
I certainly can, but you'll need to describe them in a useful way
first. http://search.yahoo.com/search?p=useful+bug+reports seems to
link to good tips.
And please post them to POE's rt.cpan.org queue. The Perl community
has been kind enough to establish a convenient, central place to
report problems with CPAN distributions. It would be most cooperative
of you to use it.
> (the race condition with sigchld is really annoying,
Please describe the race condition in a useful way, so that I can
investigate and possibly fix it.
POE checks for child processes between calls to application code. If
you fork your process and set sig_child() in the same event handler,
POE won't be able to call waitpid() between them. If this is the race
condition you mean, then it simply shouldn't be possible.
On the other hand, I haven't actually documented this behavior. It
hadn't occurred to me because all my programs follow the reliable
pattern. Based on your feedback, I added this to the description of
sig_child(). Please let me know how/if I can improve it:
"Programs that wish to reliably reap child processes should be sure to
call sig_child() before returning from the event handler that forked
the process. Otherwise POE::Kernel may have an opportunity to call
waitpid() before an appropriate event watcher has been registered."
In the event that you've found an actual bug, I regret that your
description of the problem is not useful. As we say in IRC:
<purl> Look buddy, doesn't work is a strong statement. Does it sit on
the couch all day? Is it making faces at you? Does it want more money?
Is it sleeping with your girlfriend? Please be specific!
> and the nag messages are as well).
I understand from your documentation that you dislike the "ugly" "nag"
messages. I'm sorry, but I cannot help you there. They were added as
the result of useful feedback from other users. And contrary to your
documentation, they can be silenced.
The "run() method was never called" message can be silenced by calling
run(). Not all applications require run(). But if you call it
without any POE::Session instances, it will return immediately and
satisfy POE::Kernel.
For example, this generates the warning:
perl -wle 'use POE; POE::Session->create(inline_states => { _start =>
sub{} });'
While this does not:
perl -wle 'use POE; POE::Kernel->run; POE::Session-
>create(inline_states => { _start => sub{} });'
I hadn't considered to document the technique because it seemed
obvious to me. By now I hope it's clear that a lot more things seem
obvious to me than are actually obvious to the average user. You
should probably call them to my attention if you'd like them to be
documented.
Speaking of which, I added this to the docs. Please let me know how/
if I can improve it:
"POE::Kernel will print a strong message if a program creates sessions
but fails to call run(). If the lack of a run() call is deliberate,
you can avoid the message by calling it before creating a session.
run() at that point will return immediately, and POE::Kernel will be
satisfied."
...
You complain about a message that occurs when a program leaks a child
process. Programs that leak processes are broken in a way that is
difficult to detect without the warning. In extreme cases, process
leakage can quietly strangle a machine to death.
POE can detect this silent but deadly condition. It notifies users
out of courtesy to them, and also because I was tired of handling "POE
crashed my machine!" reports. In this case, the one who smelled it
did not deal it.
If you receive this message, it can be silenced by fixing the process
leak.
>> I have specific issues with your docs. As a courtesy to you and the
>> list, I'll send them to you directly.
>
> Not sure why it would be a courtesy to me, do you have anything to
> hide
> or do you want to insult me even more in private so nobody sees your
> real
> self or something? (just guessing... :)
Wow, you really do expect the worst from me. I don't know where that
came from.
The courtesy to you would be that I brought the issues to you
directly. I could post them somewhere public and unexpected and wait
for you to find them, but that's just rude.
Likewise, posting long, limited-interest e-mail to a public
distribution list is a violation of common etiquette. To those who
are still reading, I'm sorry.
I kind of thought those points would be obvious.
If you still insist on doing this publicly, please indicate your
intent by posting your documentation for AnyEvent::Impl::POE to the
mailing list (new thread, please). That will make it easier for me to
respond. For the record, I still think this is a bad idea.
> I do fix my bugs and am open to suggestions and improvements. But
> correctness comes first, performance second.
OMG, we agree on something! The end times are here:
--
Rocco Caputo - rcaputo@pobox.com
Thread Previous
|
Thread Next