develooper Front page | perl.libwww | Postings from October 2001

Sticky 'Host: ' headers.

Thread Next
John Fessenden
October 12, 2001 17:14
Sticky 'Host: ' headers.
Message ID:

I've searched the Web, (google hopefully would dig throught mail
archives. ) and read over the ChangeLog, and I saw no mention of this
My appologies if this has allready been pointed out:

I've discovered that If you set a Host header

  $r->header('Host', "")

on a request object.  And use LWP::UserAgent::request()
to make the request, and follow redirects to the final result, 

The 'Host: ' header will persist on to each new Redirected request.
This is because to follow the redirect LWP copies the request and modifies
only the URI object it contains, leaving the original set of headers. 
(which of course you would want, except for the 'Host: ' header with the
http protocol. )

This can often go unnoticed as many web servers ignore the 'Host: '
header. However in some cases where the Redirected to server is serving
different content based on the 'Host: ' header it will cause a failure to
retrieve the desired content.

Along these lines, I've noticed that LWP::Protocol::http sets the 'Host: '
header prior to making the actual socket connection, however it does it to
a copy of the headers of the Request object, thereby specifically avoiding
this problem, which would seem to indicate it was thought about. 

So, my question is this: is this expected behavior? if so what is the
recomended way of dealing with it?

my workaround for this issue was to subclass LWP::UserAgent and 
replace the redirect_ok() method with:

sub redirect_ok
    my $self=shift;
    my $request = shift;

    return 0 if $request->method eq "POST";

    # assuming the 'Host:' header got set,  don't let it
    # persist on to redirected connections.
    # maybe it's not the most logical place for this, but
    # it's easier than subclassing LWP::Protocol::http
    # or replacing the $self->request method.

    $request->remove_header( 'Host' );


Thanks for any insight you all may have.

Please reply to my email as I am not yet subscribed to the list.


Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About