develooper Front page | perl.perl5.porters | Postings from July 2002

Re: Thread::Use - use a module inside a thread only

Thread Previous | Thread Next
Nicholas Clark
July 30, 2002 03:22
Re: Thread::Use - use a module inside a thread only
Message ID:
On Tue, Jul 30, 2002 at 12:05:49PM +0200, Elizabeth Mattijsen wrote:
> At 10:59 AM 7/30/02 +0100, Nicholas Clark wrote:
> > > Actually, the thing about PerlIO::gzip isn't true.  You can't create
> > > threads with a PerlIO::gzip layer open currently.  Even if you don't 
> > need a
> > > PerlIO::gzip layer in the thread.  There is no problem if you only want a
> > > PerlIO::gzip layer inside a thread only.  So that is a different problem
> > > altogether.
> >I haven't looked at this thoroughly (actually, it's only been
> >cursory) but I think that cloning a layer open currently strictly requires
> >cloning zlib's struct. In turn, zlib's struct can contain dynamically
> >allocated structures, so they would have to be cloned too. Er, except
> >that zlib's struct is private to zlib. Conundrum.
> Hmmm... do we _really_ want to be reading or writing gzipped files from 
> different threads?  I think that would really be asking for trouble...

Oh, reading or writing should "break" (in that your data gets corrupted by
being randomly split into the two programs, or randomly spliced together
on disk from the two programs) but it should not SEGV (or break your program
otherwise with double free()s)

> The problem right now is that if you want to read or write a gzipped file 
> with the PerlIO::gzip layer, you need to wait until all threads are started 
> and then open the files.  This is because the threads will die because 
> PerlIO::Gzip can't clone.  Even if I don't need PerlIO::gzip inside the thread!

This is bad. I've not tried it, but I know that it's not going to work.

> >I think deflateCopy() may actually be the solution, if it copies all parts
> >of a z_stream whether being for inflation or deflation.
> >But it would have to make the new copy using a malloc() that allocates memory
> >from the new thread, wouldn't it?
> I have no idea.  But I think synchronizing reads and writes from different 
> threads would be a major task...  ;-(

I have no intention of doing that :-)

I mean to find the simplest way to make cloning an new ithread behave in a
way equivalent to (real) forking a program that happens to be using zlib
to read/write a compressed stream: providing one side of the fork doesn't
touch the stream further then all is happy. (all that side does is close and
cleanly discard its z_stream structure at some point) If both sides of the
fork access the stream then their zlib structures in memory are not
corrupted, but garbage data will result.

In theory one could do a "lazy" copy on write style clone to implement the
above - whichever side accesses the stream first wins it, and the other
half has it forcibly discarded (with a warning if it attempts to access it)
but that would mean deferring and storing the information available at CLONE
time about how to copy into the new malloc() pool of the new ithread.

Nicholas Clark

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About