develooper Front page | perl.perl5.porters | Postings from November 2010

why is the perlio buffer hard-wired to 4K?

Thread Next
From:
Craig A. Berry
Date:
November 6, 2010 16:33
Subject:
why is the perlio buffer hard-wired to 4K?
Message ID:
8528CA05-3619-4F15-AF4F-03F2814DBEB0@mac.com
It looks like the perlio layer flushes to the unixio layer whenever  
the perlio buffer fills up (assuming default layers with nothing  
special pushed in between).  That's for writes; for reads, replace  
"flushes to" with "fills from."  But the perlio buffer is hard-wired  
to 4096 bytes, which seems odd both because it's small and because  
it's a naked number with no documentation or justification for the  
chosen size that I could find.  As far as I can tell this specific  
buffer size came in here:

http://perl5.git.perl.org/perl.git/commitdiff/bb9950b796df42e2f824a072ae878c87e977be20

and was never changed.  That was 10 years ago, at a rather early stage  
of PerlIO development (I *think* before layers as such even existed).   
Apparently I am not the only one who has wondered about the small,  
hard-coded size:

http://www.perlmonks.org/?node_id=692386

A larger buffer would result in less frequent flushes or fills and  
presumably save some layer shuffling.  This needs testing, which I  
haven't gotten to yet.   I have seen extra lines show up on a line- 
buffered filehandle where the lines are longer than 4096 bytes, and a  
larger buffer clearly allows longer lines in that situation.  That I  
have tested.

So what is a good size?  I think a pretty safe option would be to make  
it the larger of 4096 and BUFSIZ as below.  That would leave it as is  
where BUFSIZ is very small, but allows a better guess as to what's  
appropriate for the local system if the maintainers of the relevant C  
run-time have increased BUFSIZ to something larger.  Another option  
would be to use st.st_blksize.

I will try to do some proper benchmarking, but does anyone see offhand  
any reason that the following wouldn't be an improvement?

--- perlio.c;-1 2010-11-04 19:23:21 -0500
+++ perlio.c    2010-11-04 21:35:30 -0500
@@ -4133,7 +4133,7 @@ PerlIOBuf_get_base(pTHX_ PerlIO *f)

      if (!b->buf) {
         if (!b->bufsiz)
-           b->bufsiz = 4096;
+           b->bufsiz = BUFSIZ > 4096 ? BUFSIZ : 4096;
         Newxz(b->buf,b->bufsiz, STDCHAR);
         if (!b->buf) {
             b->buf = (STDCHAR *) & b->oneword;
[end]

________________________________________
Craig A. Berry
mailto:craigberry@mac.com

"... getting out of a sonnet is much more
  difficult than getting in."
                  Brad Leithauser


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About