Front page | perl.beginners |
Postings from February 2011
Re: perl's threads
From: Rob Coops
February 9, 2011 05:03
Re: perl's threads
Message ID: AANLkTi=vjgabEoc8Hp_6tmEPNaoh6-Hzf2w4DU+RgCGu@mail.gmail.com
Just adding a little to this discussion then. :-)
Threads regardless of the language are meant to do multiple things in
parallel. For instance I'm at the moment working on a tool that extracts
vast amounts of data from a web service as the calls to that service are
pretty much independent and the data on my side is first stored in a
database before further processing (using separate tables for each of the
calls) there is very little point in doing these actions sequential.
After all both the database and the webservice are fully capable of handling
hundreds or even thousands of requests at a time without fail so why not run
things in parallel?
I found that even with a minimal amount of data I can speed up the process a
lot by simply creating an object that deals with the database and using this
as a base class to build the various API calling objects on. Each object
then is fully self contained deals with the retrieval of its own data
massaging that in a format that fits in its own tables and uses its parents
code to deal with the database insertion.
It depends on your intended usage of threads, if you are thinking along the
lines of a multi threaded process that operates on shared data, where the
threads are constantly sharing outcomes and using the results other threads
have produced to continue their processing so they can supply the yet other
threads with data... think again. Though of course possible in Perl it is
also an very very complex thing to do and the performance will simply not be
there at least not when comparing to other solutions.
But as said for simple parallel and independent actions using threads in
Perl is not such a bad idea.
The idea that threads are always bad regardless of programming language or
are bad i many languages is simply not true. Threads certainly have a place
in programming and in many if not all programing languages. With the many
core processors of to day and tomorrow threads will become a much more
important and useful tool in the programmers arsenal.
The main thing is that for a very long time programmers have had very little
real use for threads as most computers had a single core with 1 or in later
years 2 threads per core. That simple fact has convinced many programmers
that threads are not all that they are hyped up to be. After all you are
still waiting on the hardware to become available and if you run more then 2
threads at a time the time the processor needs for switching between threads
removes many of the performance benefits threads could offer.
Now though that one sees processors that can handle up to 32 threads and
soon even 128 threads at a time multiple threads will prove to be a lot more
useful then most programmers have given them credit for. As the processor
does not need to do all the switching between threads and can simply
continue working on the thread without interruption the additional speed
that literally processing in parallel can bring will be very noticeable. The
main problem that developers will find in terms of raw performance is the
I/O of these systems which will not be able to provide data fast enough to
keep the threads from having to wait for data to become available.
Of course the old monster of how do you deal with multiple threads that do
need to exchange data will be back on the table and certainly if you are
doing funky stuff such as AI where many different inputs are processed at
the same time and all can and should to an extend influence each other then
you will spend many nights waking up screaming in the middle of the night as
you got tangled in the multiple thread nightmare again.
As for Perl the threads implementation on it offers is not to bad and quite
convenient for the simpler parallel run scenario. For serious performance in
the multi threaded arena Perl is simply not the tool for the job, it is as
simple as that.
But any developer that tells you that threads are not worth the trouble they
cause or anything along those lines is clearly an experienced programmer
who unfortunately has not realized yet that the computer world has
changed incredibly over the past few years.
2011/2/9 Shlomi Fish <firstname.lastname@example.org>
> On Tuesday 08 Feb 2011 10:05:47 Dr.Ruud wrote:
> > On 2011-02-07 11:30, Shlomi Fish wrote:
> > > Threads work pretty well in C, though they are extremely tricky to get
> > > right for non-trivial programs
> > That they work "pretty well" is probably about user experience of some
> > heavily interactive system. But only if the user accepts crashes and
> > deadlocks now and then.
> What I meant by work pretty well, is that they are lightweight, cheap,
> well, and can often increase performance.
> > The "tricky to get right" misses the point that systems based on threads
> > have no proper math to back them up.
> . I believe I've seen some mathematical models of threads,
> and you can prove the lack of deadlocks/livelocks/etc. in a model of the
> system mathematically. But like you said, if one is going to implement
> cooperative threads that work using the same resources, he or she is in for
> lot of trouble.
> > Threads are like the first little pig, who built a house of straw.
> Possibly, but they are still implemented and perform in C much better than
> they do in Perl or similar languages.
> In any case, I've done some non-cooperative multi-tasking (or maybe split-
> tasking) where I assign different independent tasks to different threads
> solving a range of Solitaire deals by each thread solving different deals
> in a
> round-robin fashion) and it indeed increased the overall performance of the
> range solver on my Pentium 4 2.4GHz machine with two hyperthreads and on my
> Dual Core x86-64 laptop. I later on implemented a forking range solver, and
> subsequent benchmarks it did not perform as well (the difference was not
> large, but I'm not willing to throw it away either due to what I said here:
> On a discussion I raised on Linux-IL (the Israeli Linux mailing list), I
> told that in order to truly benefit from multiple-cores, one has to make
> the individual tasks execute as closely as possible to one another, which I
> believe implies either multiple threads or alternatively a program where
> fork without calling exec/EXECVE later and just continue execution.
> Shlomi Fish
> Shlomi Fish http://www.shlomifish.org/
> Understand what Open Source is - http://shlom.in/oss-fs
> Chuck Norris can make the statement "This statement is false" a true one.
> Please reply to list if it's a mailing list post - http://shlom.in/reply .
> To unsubscribe, e-mail: email@example.com
> For additional commands, e-mail: firstname.lastname@example.org