develooper Front page | perl.libwww | Postings from December 2001

Re: sysread() in Net::HTTP times out too quickly

Thread Previous | Thread Next
Gisle Aas
December 6, 2001 09:33
Re: sysread() in Net::HTTP times out too quickly
Message ID: writes:

> recently when we upgraded to the iPlanet 41sp7 release (formerly known as 
> Netscape Enterprise Server), I realized that your otherwise superb LWP 
> library reports errors on certain HTTP requests while common web browsers 
> still work fine.
> I tracked the problem down to the Net::HTTP::Methods module where the 
> sysread() call in the my_readline() function times out immediately if there's 
> no response waiting on the wire already. In order to band-aid the problem, I 
> replaced the sysread() method call by
>             my $n = 0;
>        my $retries = 10;
>        # need to read more data to find a line ending
>        while($n == 0 && $retries--) {
>            $n = $self->sysread($_, 1024, length);
>            last if $n;
>            #print "NEW: Read ", $n ? $n : "0", " bytes\n";
>            sleep(1);
>        }
> And now it works fine again. Experiments showed that it can take up to two 
> rounds to actually get the data that is being sent by the web server. It 
> looks like the sysread doesn't actually block until there's data available. 
> Modern browsers seem to handle this situation gracefully -- any chance you 
> could have LWP handle it, too?

I would sure hope so.  First I would have to understand what is really
broken and why.  Do you have some URL that I can try that fails?  It
might be hard to reproduce if it depends on timing and perhaps the
client OS.  Please also report what version of LWP, Perl you use and
what OS you are running on.

One thing that is different in the new driver code is that the socket
is put into non-blocking mode.  It should still be correct since we
never sysread data from the socket until select(2) has told us that the
socket is readable.  It means that either there should be some data to
read or we have reached EOF.

Try to comment out the call to 'blocking' in LWP::Protocol::http.
Does it work then?


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About