develooper Front page | perl.perl5.porters | Postings from April 2012

Re: How do I enable sfio support in perl?

Thread Previous | Thread Next
From:
Nicholas Clark
Date:
April 5, 2012 08:15
Subject:
Re: How do I enable sfio support in perl?
Message ID:
20120405151542.GN9069@plum.flirble.org
On Fri, Oct 01, 2010 at 01:25:44PM -0400, Andy Dougherty wrote:
> > How do I enable sfio support in perl? A colleague did it a few years
> > who and archived vast performance improvements with it. Now I am
> > tasked with the perl upgrade but do not know where I should start.

My understanding after some digging is that it used to be -Dusesfio
-Duseperlio passed to ./Configure (along with sfio.h found in the include
search path) but Configure was tweaked so that -Dusesfio would automatically
turn on -Duseperlio.

"was tweaked" seems to have been a patch from me. But 12 years ago.
My memory is hazy on these things by now.

Vast performance improvements vs what? stdio? And which version of perl
(significance of this question is below)

> I had fiddled with sfio some years ago, but didn't see any significant 
> performance improvements in any of my work.
> 
> Where does one obtain sfio nowadays?

And here the thread seemed to die.

Right, the backstory is (I think)

perl-5.004 offered, "new" (then)

a) a PerlIO abstraction layer. This wasn't *much* - for stdio, FILE * was
   disguised as PerlIO *, and the functions in perlio.c were just direct
   wrappers around stdio calls
b) experimental support for sfio.
   This used the abstraction layer - the publicly visible type was still
   PerlIO * and the perlio.c functions had the same names.


and at this point, handles were either PerlIO*s or FILE*s, and not much code
cared.

perl-5.005 came and went

perl-5.6.0 was "marketing compatible" with Unicode.

It only missed a couple of minor Unicode features such as Unicode I/O,
Unicode hash keys, and /./ matching Unicode characters

perl-5.8.0 actually could *do* Unicode.

For this, the PerlIO abstraction was turned into real code.
Real ugly code :-(

It's built of "layers" - my understanding is that the code shipped in core
can use either Unix calls such as open()/read()/write() as the source layer,
or ANSI stdio calls. It would be possible to write others, for example a
"Win32" layer to use Win32 APIs directly, etc.


As best I can work out from trying to build on a FreeBSD 7.0 machine:

If sfio's replacement stdio.h is present in the compiler include path, then
commit 2d967e397d11d722 from March 2001 causes Configure to bail out
completely, as it can't compile the gcc version test code, which includes
stdio.h because it uses printf(). The link fails, as sfio's replacement
libstdio.a isn't used.

Remove this, and -Dusesfio works. So 5.8.0 would have built OK with sfio.

However, commit 39f7a87036eb8d13 in April 2003 breaks the build with sfio.
This was merged to the maint-5.8 branch.

This means that 5.8.1 onwards and 5.10.0 onwards do not build with sfio.

I wasn't aware of this. I don't think we've had any reports to RT in the
8.5 years since 5.8.1 shipped. We get more bugs reported on OS/2.


Also, post 5.14.0 Configure has now removed the option to disable useperlio
Everyone gets the perlio layer, either atop stdio or atop "unix".
Directly using stdio "old style" isn't an option offered.

Given that we have PerlIO atop stdio, it may be possible to fix the
existing code to run PerlIO atop Sfio. However, it wasn't obviously
easy, so I tried something else which probably actually less easy in the
end.

If one overrides Configure like this:

--- config.sh~	2012-04-04 17:44:44.000000000 +0200
+++ config.sh	2012-04-04 10:47:17.000000000 +0200
@@ -957,7 +957,7 @@
 usemymalloc='n'
 usenm='false'
 useopcode='true'
-useperlio='define'
+useperlio='undef'
 useposix='true'
 usereentrant='undef'
 usesfio='true'

to disable the PerlIO system completely, perl builds. Even blead still
builds, but fails a lot of tests. This *isn't* a "supported" configuration.


However, it turned out that not much is needed to get blead to build sfio
like this. Specifically

diff --git a/perlio.h b/perlio.h
index f1b5ede..fc68d97 100644
--- a/perlio.h
+++ b/perlio.h
@@ -46,7 +46,7 @@
 #endif
 
 #ifndef USE_PERLIO
-# define USE_STDIO
+/*# define USE_STDIO */
 #endif
 
 #ifdef USE_STDIO

and

diff --git a/perlio.c b/perlio.c
index 79c6fdf..11d9793 100644
--- a/perlio.c
+++ b/perlio.c
@@ -433,7 +433,7 @@ FILE *
 PerlIO_findFILE(PerlIO *pio)
 {
     const int fd = PerlIO_fileno(pio);
-    FILE * const f = fdopen(fd, "r+");
+    FILE * f = fdopen(fd, "r+");
     PerlIO_flush(pio);
     if (!f && errno == EINVAL)
 	f = fdopen(fd, "w");


which undoes an error in

commit de009b76d60bdeb88f1d812ac755ae225805a071
Author: Andy Lester <andy@petdance.com>
Date:   Wed Apr 27 05:02:43 2005 -0500

    perlio-two.patch: More warnings squashed, more consts
    Message-ID: <20050427150243.GA21883@petdance.com>
    
    p4raw-id: //depot/perl@24338


and this which corrects a typo in a refactoring,

diff --git a/util.c b/util.c
index 221dee5..3c2e16b 100644
--- a/util.c
+++ b/util.c
@@ -1516,7 +1516,7 @@ Perl_write_to_stderr(pTHX_ SV* msv)
     else {
 #ifdef USE_SFIO
 	/* SFIO can really mess with your errno */
-	dSAVED_ERRNO;
+	dSAVE_ERRNO;
 #endif
 	PerlIO * const serr = Perl_error_log;


and this patch for Storable which we should probably implement anyway:

diff --git a/dist/Storable/Storable.xs b/dist/Storable/Storable.xs
index ca6f9b4..27966c8 100644
--- a/dist/Storable/Storable.xs
+++ b/dist/Storable/Storable.xs
@@ -37,12 +37,14 @@
  */
 #ifndef USE_PERLIO
 #ifndef PERLIO_IS_STDIO
+#if 0
 #define PerlIO FILE
 #define PerlIO_getc(x) getc(x)
 #define PerlIO_putc(f,x) putc(x,f)
 #define PerlIO_read(x,y,z) fread(y,1,z,x)
 #define PerlIO_write(x,y,z) fwrite(y,1,z,x)
 #define PerlIO_stdoutf printf
+#endif
 #endif	/* PERLIO_IS_STDIO */
 #endif	/* USE_PERLIO */
 

as the code which it disables is for pre-5.004 support, and that's long dead.

With these, blead sort-of-builds sfio.
It bombs out in 4 places where make_ext.pl writes a Makefile.PL and then
tries to use that to generate the Makefile - in each case it fails with an
error about being unable to write to MakeMaker.tmp, and that file is 8192
bytes long. Stealing the correct Makefile from another blead build fixes that.

The test failures are a superset as those for PerlIO-less stdio
(166 vs 122). A test in t/io fails, and Tie::File fails a lot of tests, which
may be related.

Nicholas Clark

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About