develooper Front page | perl.libwww | Postings from January 2001

Inconsistent behavior w/ LWP GET in rare cases: timeouts (500)instead of correct response

From:
Phil Mitchell
Date:
January 16, 2001 12:18
Subject:
Inconsistent behavior w/ LWP GET in rare cases: timeouts (500)instead of correct response
Message ID:
4.3.2.7.2.20010116141607.00adb5d0@hulmail.harvard.edu
I am doing GET's on a long list of URL's to detect 404's etc. Out of 1000's 
of URL's, I have seen a few that consistently yield inconsistent behavior. :)

http://www.ndu.edu/ndu/inss/strforum/forum83.html -- This url should return 
a 404, but LWP almost always times out with a server error (500). It's not 
a server problem, though, b/c GET works fine on http://www.ndu.edu. To 
complicate matters, when I run this script from a different machine, it 
consistently gets the 404.

In a related note, on a small number of other URLs (eg., 
http://nces.ed.gov/nationsreportcard/guide/2000456.shtml), I have found LWP 
to be much more likely to time out than to get a correct response, whereas 
Netscape generally gets a prompt response. I have tried 
ua->agent("Netscape/4.7"), but that doesn't seem to be the issue.

In the ndu.edu case, the first machine is a Solaris box on a LAN/ethernet 
connection. When I access it from a linux box via PPP, I get the 404. The 
Solaris setup runs perl 5.005_3 and LWP 5.48; the linux box has perl 5.6.0 
and (I think) LWP 5.49.

Note that I'm checking 1000's of other urls, and the script generally runs 
fine from either box.

Note that my GETs are simple_requests, b/c I need to handle redirects manually.






nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About