develooper Front page | perl.libwww | Postings from March 2013

Trying to diagnose a LWP on FreeBSD (>8.x) issue

Thread Next
From:
Philip Kizer
Date:
March 1, 2013 21:40
Subject:
Trying to diagnose a LWP on FreeBSD (>8.x) issue
Message ID:
CB630484-2938-4BDD-B7C7-B494385F0F92@nostrum.com
I tried mailing the maintainer but no response there has led me to a bit more public forum.


Using any perl > 5.8 (since I really don't have older except for one system about to be upgraded, they're mostly 5.12 for me right now) and LWP 6.04, I see the following behaviour that can be tested as simply a this that mimics what a personal proxy is doing for me:

GET -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)' http://www.gocomics.com/doonesbury/2013/01/27/

When doing that, the output I get is truncated at what is essentially the end of the first data packet.


To verify the server, requesting machine and network, if I merely repeat that action with another tool it succeeds:

curl -H 'User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)' http://www.gocomics.com/doonesbury/2013/01/27/


To further narrow it down, the following copied almost verbatim from the Net::HTTP perldoc also works just fine:

% cat try-net-http.pl 
#!/usr/local/bin/perl -w
use Net::HTTP;

my $s = Net::HTTP->new(Host => "www.gocomics.com") || die $@;
$s->write_request(GET => "/doonesbury/2013/01/27/", 'User-Agent' => "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)");
my($code, $mess, %h) = $s->read_response_headers;

while (1) {
  my $buf;
  my $n = $s->read_entity_body($buf, 1024);
  die "read failed: $!" unless defined $n;
  last unless $n;
  print $buf;
}



The common elements I have found are that I have to be using LWP on FreeBSD-8 or FreeBSD-9 to observe the failure, older 6.x and 7.x systems all produce full output even with the same version of LWP.  For me at least this seems to hold on all 8.x and 9.x systems I have.


Can anyone at least verify that they see the same behaviour?  If so, can anyone suggest where to look for fixes?


Thanks,
Philip




Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About