develooper Front page | perl.libwww | Postings from December 2003

RFC: WWW::Mechanize::Compress or LWP patch?

Thread Next
Brian Cassidy
December 2, 2003 09:37
RFC: WWW::Mechanize::Compress or LWP patch?
Message ID:
Hi All,

Today I cooked up a little bit of code [1] to give WWW::Mechanize the
ability to handle compressed content (gzip and deflate). I forwarded it over
to Andy for his comments and he thought that maybe it would be best if this
code was adapted for use directly in LWP.

I would tend to agree since everything is handled behind the scenes.
+ If Compress::Zlib isn't available, we forgo the "Accept_encoding" headers
+ It makes sure the response is compressed before trying to uncompress

The only (freak) edge case would be if you get a response that was encoded
and Compress::Zlib isn't available (thus it croak()s). 

There could be considerable bandwidth savings if LWP users were able to get
compressed content by default (without even knowing it :). Although I guess
therein hides the problem where we force people to accept compressed


-Brian Cassidy ( )


package WWW::Mechanize::Compress;

use strict;
use warnings FATAL => 'all';
use vars qw( $VERSION $HAS_ZLIB );
$VERSION = '0.01';

use base qw( WWW::Mechanize );
use Carp qw( carp croak );

	$HAS_ZLIB = 1 if defined eval "require Compress::Zlib;";

sub _make_request {
	my $self    = shift;
	my $request = shift;

	$request->header( Accept_encoding => 'gzip; deflate' ) if $HAS_ZLIB;

	my $response = $self->SUPER::_make_request( $request, @_ );

	if ( my $encoding = $response->header( 'Content-Encoding' ) ) {
		croak 'Compress::Zlib not found. Cannot uncompress content.'
unless $HAS_ZLIB;

     		$self->{ uncompressed_content } = Compress::Zlib::memGunzip(
$response->content ) if $encoding =~ /gzip/i;
		$self->{ uncompressed_content } =
Compress::Zlib::uncompress( $response->content ) if $encoding =~ /deflate/i;

	return $response;

sub content {
	my $self = shift;

	return $self->{ uncompressed_content } || $self->{ content };

1; - Messaging for educators.

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About