develooper Front page | perl.libwww | Postings from April 2003

$ua->parse_head and gzip encoding

Thread Next
From:
Bill Moseley
Date:
April 14, 2003 23:13
Subject:
$ua->parse_head and gzip encoding
Message ID:
Pine.LNX.4.10.10304142217500.28144-100000@mardy.hank.org

I was looking at gzip encoded documents (using mod_gzip on Apache).  I can
add an Accept-encoding: gzip header and get back compressed docs, but I
noticed that in LWP::Protocol that header parsing happens if content type
is text/html regardless of encoding.  Would it be wise to also check for
Content-encoding like below?

I actually use the parsed headers (looking for <meta name="robots" ...>),
so it would be nice to if LWP uncompressed if Compress::Zlib was
available.  Although, I'm not sure how to handle uncompressing in chunks.



--- LWP/Protocol.pm     2001-10-26 12:00:21.000000000 -0700
+++ LWP/Protocol.pm.new 2003-04-14 22:39:04.000000000 -0700
@@ -191,7 +191,7 @@
     my($parse_head, $max_size) = @{$self}{qw(parse_head max_size)};
 
     my $parser;
-    if ($parse_head && $response->content_type eq 'text/html') {
+    if ($parse_head && $response->content_type eq 'text/html' && !$response->header('Content-Encoding')) {
        require HTML::HeadParser;
        $parser = HTML::HeadParser->new($response->{'_headers'});
     }





-- 
Bill Moseley moseley@hank.org



Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About