develooper Front page | perl.libwww | Postings from April 2003

$ua->parse_head and gzip encoding

Thread Next
Bill Moseley
April 14, 2003 23:13
$ua->parse_head and gzip encoding
Message ID:

I was looking at gzip encoded documents (using mod_gzip on Apache).  I can
add an Accept-encoding: gzip header and get back compressed docs, but I
noticed that in LWP::Protocol that header parsing happens if content type
is text/html regardless of encoding.  Would it be wise to also check for
Content-encoding like below?

I actually use the parsed headers (looking for <meta name="robots" ...>),
so it would be nice to if LWP uncompressed if Compress::Zlib was
available.  Although, I'm not sure how to handle uncompressing in chunks.

--- LWP/     2001-10-26 12:00:21.000000000 -0700
+++ LWP/ 2003-04-14 22:39:04.000000000 -0700
@@ -191,7 +191,7 @@
     my($parse_head, $max_size) = @{$self}{qw(parse_head max_size)};
     my $parser;
-    if ($parse_head && $response->content_type eq 'text/html') {
+    if ($parse_head && $response->content_type eq 'text/html' && !$response->header('Content-Encoding')) {
        require HTML::HeadParser;
        $parser = HTML::HeadParser->new($response->{'_headers'});

Bill Moseley

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About