develooper Front page | perl.libwww | Postings from November 2001

Re: Google denies libwww-perl

Thread Previous | Thread Next
From:
Gisle Aas
Date:
November 14, 2001 22:41
Subject:
Re: Google denies libwww-perl
Message ID:
lr1yj0ilk0.fsf@caliper.ActiveState.com
Tatsuhiko Miyagawa <miyagawa@edge.co.jp> writes:

> Google seems to deny access via LWP, so live/google.t fails.
> I guess the rule is $Header{'User-Agent'} =~ /libwww-perl/

I will remove the 't/live/google.t' from the distribution.  It appears
they did not like it.

Note that Google is very confused when LWP sends it a "Connection"
header that contains something else than "Keep-Alive".  By default LWP
will send "Connection: TE" to something it thinks is a HTTP/1.1
server.  The result is a Google refuse to keep the connection alive.
This is wrong.

The workaround used in 't/live/google.t' was to set the
@EXTRA_SOCK_OPTS to suppress the TE header.

  # Google is confused if we end up sendit it the "Connection: TE"
  # header and will close the connection.  This avoids it.
  push(@LWP::Protocol::http::EXTRA_SOCK_OPTS, SendTE => 0);

I wished there was a way to set up these options on a per server
basis.

Regards,
Gisle

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About