develooper Front page | perl.vmsperl | Postings from March 2012

Re: what was VMS do here? (was [perl #79960] Setting $/ to read fixedrecords can corrupt valid UTF-8 input)

Thread Previous | Thread Next
From:
Craig A. Berry
Date:
March 2, 2012 06:11
Subject:
Re: what was VMS do here? (was [perl #79960] Setting $/ to read fixedrecords can corrupt valid UTF-8 input)
Message ID:
E5F38775-5418-40B2-9221-C8CBEE1EC876@mac.com

On Mar 2, 2012, at 3:07 AM, Eric Brine wrote:

> On Thu, Mar 1, 2012 at 6:17 PM, Craig A. Berry <craigberry@mac.com> wrote:
> What happens on Unix when you have a pipe buffer that is 8192 bytes and you set $/ to 8193 and read a record containing UTF-8 data through the pipe?
> 
> Perl requests 8K (formerly 4K) chunks until it has received enough. It requests 8K even if it only needs 1 byte.

I think you're thinking of the PerlIO buffer that I increased from 4K to the larger of 8K and BUFSIZ in 5.14, and which only applies to the perlio layer.  But S_sv_gets_read_record calls PerlIO_read, which just retrieves the base layer (formerly stdio, currently unix) and calls its Read method, which is just read().  So there is no buffering under Perl's control.

I was thinking of a situation where something external to Perl limits how much data you can get in one read and thus gives you less than the full amount requested by $/.  I'm pretty sure you'll get mangled UTF-8 if you happen to be mid-character when you hit the end of the device buffer.  To test this, you'd need to know something about the internals of your system's pipe implementation (or other device with a fixed buffer).

________________________________________
Craig A. Berry
mailto:craigberry@mac.com

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About