develooper Front page | perl.libwww | Postings from January 2001

Help debugging LWP::Protocol::http problem

From:
Phil Mitchell
Date:
January 25, 2001 09:26
Subject:
Help debugging LWP::Protocol::http problem
Message ID:
4.3.2.7.2.20010125115241.00af1de0@hulmail.harvard.edu
LWP reports a server error when the server is fine, b/c of the following 
obscure problem:

There is an interaction between Solaris and certain web servers (notably 
Microsoft-IIS/4.0) where the response to a GET comes back improperly 
terminated. You can observe this independently of LWP by doing a "telnet 
www.ndu.edu 80",
and then "GET http://www.ndu.edu/ndu/inss/strforum/forum83.html"
You will see telnet receive the response (it's a 404), but then (usually 
but not always) fail to terminate. As far as I know, you will only see this 
on Solaris (I'm on Solaris 7, have heard similar problems w/ 6); I've tried 
same under Linux and haven't seen this problem. I have seen this on a 
number of other sites.

The problem seems to be that LWP reports a server error (500) b/c it times 
out while waiting for the missing EOF terminator. I've turned on LWP::Debug 
and see it hang, apparently in the LWP::Protocol::http->request subroutine, 
or possible in collect(), though I'm not sure. You can see that LWP 
received the response, so it ought to do more than just report a timeout.

Here is the output from a typical run:

LWP::UserAgent::new: ()
HTTP::Cookies::add_cookie_header: Checking www.ndu.edu for cookies
HTTP::Cookies::add_cookie_header: Checking .ndu.edu for cookies
LWP::UserAgent::simple_request: GET 
http://www.ndu.edu/ndu/inss/strforum/forum83.html
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET /ndu/inss/strforum/forum83.html HTTP/1.0
Host: www.ndu.edu
Range: bytes=0-524288
User-Agent: libwww-perl/5.48

LWP::Protocol::http::request: reading response
LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Thu, 25 Jan 2001 16:38:46 GMT
Content-Length: 461
Content-Type: text/html

<html><head><title>Error 404</title>

<meta name="robots" content="noindex">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"></head>

<body>

<h2>HTTP Error 404</h2>

<p><strong>404 Not Found</strong></p>

<p>The Web server cannot find the file or script you asked for. Please 
check the URL to ensure that the path is correct.</p>

<p>Please contact the server's administrator if this problem persists.</p>

</body></html>
LWP::Protocol::http::request: HTTP/1.1 404 Object Not Found
LWP::Protocol::collect: read 461 bytes

[This is where LWP hangs...]






nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About