develooper Front page | perl.cpan.workers | Postings from April 2010

Re: Trimming the CPAN - "Automatic Purging"

Thread Previous | Thread Next
From:
Ask Bjørn Hansen
Date:
April 1, 2010 16:13
Subject:
Re: Trimming the CPAN - "Automatic Purging"
Message ID:
4D6D3AD3-DCAB-42F6-8DE7-34D7666EAB74@perl.org

On Apr 1, 2010, at 19:49, Arthur Corliss wrote:

I can't believe I'm doing this, but ...

>> The main point here is that we can't use 20 inodes per distribution.  It's Just Nuts.   Sure, it's only something like 400k files/inodes now - but at the rate it's going it'll be a lot more soon enough.
> 
> Thats a problem, but not likely the biggest drag on server I/O you're
> suffering.  Might that be <ahem> rsync?

That reply doesn't even make sense.

>> HOWEVER: Right now more of those are wasted on other things (.readme files, symlinks, ...) -- some of which have solutions in progress already.
>> 
>> I don't think anyone is arguing that we NEED to delete the old distributions; only that they do indeed have a cost to keep around in the main CPAN.
> 
> You're right, I'm not arguing the need for the cruft.  I've only pointed out
> the obvious reality that trimming files only postpones the I/O management
> issues that at some time are likely going to have to be addressed, anyway.
> And that you'll get less bang for the buck (or man hour) by treating the
> symptoms, not the disease.
> 
> For the record:  if that's what you want to do, have at it.  Let's just not
> be disingenuous about the fact that we're abrogating our responsibilities as
> technologists by refusing to address the real problems and weaknesses of the
> platform.

You are confusing "we", "I" and "you" again.

....

Yes, I (and I'm guessing everyone else who have thought about it for more than say 5 seconds) agree that having rsync remember the file tree to save the disk IO for each sync sounds like an "obvious solution".  

But reality is more complicated.  If it was such an obviously good solution someone would have done it by now.  (For starters play this question: "What is the kernel cache?").

Andreas' solution is much more sensible -- and as have been pointed out before we DO USE THAT; but the problem here is not with clients who are interested enough to do something special and dedicate resources to their CPAN mirroring.


 - ask


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About