develooper Front page | perl.libwww | Postings from December 2003

RFC: WWW::Mechanize::Compress or LWP patch?

Thread Next
From:
Brian Cassidy
Date:
December 2, 2003 09:37
Subject:
RFC: WWW::Mechanize::Compress or LWP patch?
Message ID:
17351293718870@naldserver.nald.ca
Hi All,

Today I cooked up a little bit of code [1] to give WWW::Mechanize the
ability to handle compressed content (gzip and deflate). I forwarded it over
to Andy for his comments and he thought that maybe it would be best if this
code was adapted for use directly in LWP.

I would tend to agree since everything is handled behind the scenes.
+ If Compress::Zlib isn't available, we forgo the "Accept_encoding" headers
+ It makes sure the response is compressed before trying to uncompress

The only (freak) edge case would be if you get a response that was encoded
and Compress::Zlib isn't available (thus it croak()s). 

There could be considerable bandwidth savings if LWP users were able to get
compressed content by default (without even knowing it :). Although I guess
therein hides the problem where we force people to accept compressed
content.

Comments?

-Brian Cassidy ( brian@nald.ca )

[1]

package WWW::Mechanize::Compress;

use strict;
use warnings FATAL => 'all';
use vars qw( $VERSION $HAS_ZLIB );
$VERSION = '0.01';

use base qw( WWW::Mechanize );
use Carp qw( carp croak );

BEGIN {
	$HAS_ZLIB = 1 if defined eval "require Compress::Zlib;";
}

sub _make_request {
	my $self    = shift;
	my $request = shift;

	$request->header( Accept_encoding => 'gzip; deflate' ) if $HAS_ZLIB;

	my $response = $self->SUPER::_make_request( $request, @_ );

	if ( my $encoding = $response->header( 'Content-Encoding' ) ) {
		croak 'Compress::Zlib not found. Cannot uncompress content.'
unless $HAS_ZLIB;

     		$self->{ uncompressed_content } = Compress::Zlib::memGunzip(
$response->content ) if $encoding =~ /gzip/i;
		$self->{ uncompressed_content } =
Compress::Zlib::uncompress( $response->content ) if $encoding =~ /deflate/i;
	}

	return $response;
}

sub content {
	my $self = shift;

	return $self->{ uncompressed_content } || $self->{ content };
}

1;


http://www.gordano.com - Messaging for educators.

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About