develooper Front page | perl.libwww | Postings from April 2003

Re: $ua->parse_head and gzip encoding

Thread Previous | Thread Next
Mike Simons
April 16, 2003 07:53
Re: $ua->parse_head and gzip encoding
Message ID:
On Wed, Apr 16, 2003 at 06:25:47AM -0700, Bill Moseley wrote:
> If not, then it would be nice to have the header parser code as a
> method.  That way I could decode locally and then do something like
> $ua->parse_html_head( $response, [$content] );


  I don't understand... could you explain this a little more.

- Why don't you want the header parse done on a compressed document?

> > At 11:13 PM 2003-04-14 -0700, you wrote:
> > >I was looking at gzip encoded documents (using mod_gzip on Apache).  I can
> > >add an Accept-encoding: gzip header and get back compressed docs, but I
> > >noticed that in LWP::Protocol that header parsing happens if content type
> > >is text/html regardless of encoding.
> One topic of discussion is if LWP should decode or not.  I have mixed
> feelings if it should be automatic or not.

  I posted a patch to transparently request and decompress (block by
block) gzip style documents a little while ago.  The user who creates
a UserAgent can request the transparent decompression with a option
like "WantCompression => 1".  That code also supports block by block 
decompression of the data as it comes in, instead of handling the
whole decompression in memory.  I have code that handles deflate style
data streams as well but have not made a patch for it.

You can check the list archive if you want it... look for:
# Date: Tue, 25 Mar 2003 02:10:15 -0500
# From: Mike Simons <>
# Subject: Re: Net::HTTP does not use compressed transfers when it should

The current archives are available here:
On Mon, Mar 24, 2003 at 06:18:34PM +1100, Iain 'Spoon' Truskett wrote:
>   -
>   -

  I have been told that the place I implemented the change is not the
best.  The suggestion was to change the code into a request_callback 
method which for UserAgent to call or to move it into a LWP::Decompress 
object which can be attached to a LWP::UserAgent...
  I've not done either of those yet.

      Mike Simons

GPG key:

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About