develooper Front page | perl.libwww | Postings from April 2003

Re: $ua->parse_head and gzip encoding

Thread Previous | Thread Next
From:
Mike Simons
Date:
April 16, 2003 07:53
Subject:
Re: $ua->parse_head and gzip encoding
Message ID:
20030416144228.GA14936@moria.simons-clan.com
On Wed, Apr 16, 2003 at 06:25:47AM -0700, Bill Moseley wrote:
> If not, then it would be nice to have the header parser code as a
> method.  That way I could decode locally and then do something like
> $ua->parse_html_head( $response, [$content] );

Bill,

  I don't understand... could you explain this a little more.

- Why don't you want the header parse done on a compressed document?


> > At 11:13 PM 2003-04-14 -0700, you wrote:
> > >I was looking at gzip encoded documents (using mod_gzip on Apache).  I can
> > >add an Accept-encoding: gzip header and get back compressed docs, but I
> > >noticed that in LWP::Protocol that header parsing happens if content type
> > >is text/html regardless of encoding.
> 
> One topic of discussion is if LWP should decode or not.  I have mixed
> feelings if it should be automatic or not.

  I posted a patch to transparently request and decompress (block by
block) gzip style documents a little while ago.  The user who creates
a UserAgent can request the transparent decompression with a option
like "WantCompression => 1".  That code also supports block by block 
decompression of the data as it comes in, instead of handling the
whole decompression in memory.  I have code that handles deflate style
data streams as well but have not made a patch for it.


You can check the list archive if you want it... look for:
# Date: Tue, 25 Mar 2003 02:10:15 -0500
# From: Mike Simons <msimons@moria.simons-clan.com>
# Subject: Re: Net::HTTP does not use compressed transfers when it should


The current archives are available here:
On Mon, Mar 24, 2003 at 06:18:34PM +1100, Iain 'Spoon' Truskett wrote:
>   - http://nntp.x.perl.org/group/perl.libwww
>   - http://archive.develooper.com/libwww@perl.org/


  I have been told that the place I implemented the change is not the
best.  The suggestion was to change the code into a request_callback 
method which for UserAgent to call or to move it into a LWP::Decompress 
object which can be attached to a LWP::UserAgent...
  I've not done either of those yet.

    Later,
      Mike Simons

-- 
GPG key: http://simons-clan.com/~msimons/gpg/msimons.asc

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About