develooper Front page | perl.module-authors | Postings from November 2010

Re: Using a better compression than .gz for one's CPAN modules

Thread Previous | Thread Next
From:
Shlomi Fish
Date:
November 24, 2010 12:01
Subject:
Re: Using a better compression than .gz for one's CPAN modules
Message ID:
201011242159.59618.shlomif@iglu.org.il
On Friday 19 November 2010 22:02:48 David Cantrell wrote:
> On Fri, Nov 19, 2010 at 08:53:12PM +0200, Shlomi Fish wrote:
> > here is a report on compressing Graph-Easy-0.70.tar with various
> > compression methods:
> > 
> > -rw-r--r-- 1 shlomif shlomif  416916 Nov 14 22:23 Graph-Easy-0.70.tar.gz
> > 
> > -rw-r--r-- 1 shlomif shlomif  329197 Nov  5 12:24 Graph-Easy-0.70.tar.bz2
> > -rw-r--r-- 1 shlomif shlomif  270796 Nov 14 22:21 Graph-Easy-0.70.tar.lrz
> > -rw-r--r-- 1 shlomif shlomif  312844 Nov  5 12:24 Graph-Easy-0.70.tar.xz
> > 
> > As one can see, there are significant savings in size (and bandwidth)
> > by switching to .bz2 and .xz. .lrz (see
> > http://ck.kolivas.org/apps/lrzip/ ) yields even more in its ZPaq
> > preset, but at the cost of longer compression and even decompression
> > times, so it's not preferable. My question is:
> > 
> > 1. Will the CPAN testing and downloading toolchian will handle modules
> > uploaded as .tar.bz2?  (Allow to install them, unpack them, etc.)  How
> > about tar.xz.
> 
> Even if it does, there's not much point.  bzip2 support is nowhere near
> universal, and preventing lots of users from using your code would seem
> to be a poor trade-off for saving an insignificant number of bytes.

One can easily install .bzip2 to unpack that the distribution and as other 
people noted, there is support for it in the toolchain. Furthermore, we should 
expect that users will be required to upgrade their setup to support newer and 
better technologies. Otherwise, we'd all be stuck using the original UNIX 
compress ( http://en.wikipedia.org/wiki/Compress - does anyone still remember 
it?) or maybe no compression at all, because our helpless users cannot be 
bothered installing even compress.

> The *backpan* is so small compared to modern storage that I don't bother
> with a minicpan any more, I just carry a backpan plus indices around
> with me all the time on a bit of plastic the size of a postage stamp.

Well, with my ADSL bandwidth it took quite a long time downloading even 
minicpan.

> 
> As for the others, I've never heard of them.
> 

.xz is http://en.wikipedia.org/wiki/Xz .

http://ck.kolivas.org/apps/lrzip/ - here is information about lrzip, but note 
that its tighest compression (which I've used there) consumes a lot of 
resources, even on decompression.

Welcome to 2010.

> FWIW, there are 166 bzip2 files in my backpan mirror, at least some of
> which have test results, so yes, the toolchain appears to work for them.
> The one I bothered to check is also indexed on search.cpan.org, so that
> important part of the toolchain appears to work with it too.

OK, good.

Regards,

	Shlomi Fish

-- 
-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/

<rindolf> She's a hot chick. But she smokes.
<go|dfish> She can smoke as long as she's smokin'.

Please reply to list if it's a mailing list post.

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About