develooper Front page | perl.perl5.porters | Postings from February 2003

Re: Smoke [5.9.0] 18609 FAIL(F) MSWin32 5.1 (x86/1 cpu)

Thread Previous | Thread Next
Nick Ing-Simmons
February 2, 2003 03:37
Re: Smoke [5.9.0] 18609 FAIL(F) MSWin32 5.1 (x86/1 cpu)
Message ID:
Enache Adrian <> writes:
>> The attached patch (which creates //depot/perlio/sv.c@18612) implements 
>> all the above suggestions, and also avoids the "short buffered" case 
>> when rslen == 0 (which should now be rare).
>That' still kludgy somehow.

Yes - but it is pragmatic.

>The most concerning point is that mix of low-level and layer-level functions. 
>One may for instance write an i/o layer which has nothing to do with
>file descriptors, (think some kind of 'obstack', or embedding perl into
>another application) and a syscall is there just to get an EBADF.

If that is seen as a problem we can test for fd >= 0 before making the call.

>On the other side, if that layer serves a known size object as a file,
>it'll be nice to get it in one shot too when in slurp mode.

You will hardly ever get it in "one shot". Almost all PerlIO layers buffer 
in chunks of 4K..8K (:mmap is an exception). So even PerlIO_read()
is - under the hood - still going to Copy() into the sv 4K at a time.
The main win of your patch is avoiding lots of malloc()-s, which is 

What I most dislike about the read_record scheme is that can give wrong 
return data if the "guess" is too small - it does not re-try looking 
for more. Also if you look at the the patch you will notice that the 
read-record block also has VMS specific code which bypasses layer stack
to get special file-system record behaviour. So 'goto' that for $/ = undef
breaks :encoding() on VMS.

>My suggestion is to add a new layer-level function 'PerlIO_size()' wired
>to a 'Size' callback from the _PerlIO_funcs structure.

I am not convinced that would be better than the "kludge" as modified 
by my patch. The only layers which "know" the size are non-translating
binary ones associated with disk files (fstat works) or in-memory data
(see below).
Most layers which are problematic (encodings, compress/decompress) will
not know without reading whole file in chunks and doing the sums - 
which is exactly what we would have done without the fix.

The in-memory data case is going to 
be fast anyway. With the patch such streams will presumably fail
the fstat as they will not have a fileno, so we fallback to reading 
in stream-buffer sized chunks. An in-memory stream (e.g. PerlIO_scalar)
will normally own up to whole memory chunk as the buffer, so it 
will in simple case (where it is top of layer stack) read in one go. 
If there are layers on top then incremental is safer anyway.

Nick Ing-Simmons

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About