Re: [perl #129058] Perl_do_vop: resulting string isn't alwaysnull-terminated

Dan Collins via RT wrote:
>The outcome, as I understand it, was that perl strings would usually
>be null-terminated, but XS code could produce SVs with unterminated
>strings. So while this patch fixes one issue by null-terminating this
>particular string, a question: Is it safe, in the general case, for
>Perl_my_atof2() to use strlen?

There is an unresolved conflict between these two.  It's kind-of accepted
for XS modules to generate unterminated strings, particularly by mmapping,
but it's also even more accepted for code using Perl scalars to assume
nul termination.  (That's the point of nul-terminating most strings,
after all.)  Each of these practices can individually coexist with the
bulk of Perl XS-level code (both core and XS modules) that follows the
Postel principle, not assuming nul termination on its inputs while always
generating nul terminated outputs.  This means that code deviating from
the dominant behaviour in one way or the other is seen to work, so there
isn't much pressure on developers to stick to the Postel behaviour.

Obviously the two non-Postel behaviours (assuming nul termination on
input and omitting it on output) can't coexist with each other.  There is
no resolution to this issue that lets us keep both.  We have to make a
once-and-for-all choice, which we've been lamentably unwilling to do.

I think it is more workable to insist on nul termination.  This seems
to be the original intent, that it be safe to pass Perl strings directly
to C library functions taking nul-terminated strings.  We should declare
code generating unterminated strings to be faulty.  This would interfere
with some current mmap tricks, but there are ways to reformulate them
that don't break Perl's string model.  We can enforce nul termination
by having debugging builds check nul termination, as an assertion,
at opportune moments.

To go the other way is also possible, but with more hassle.  Although it
would rescue the mmap crowd, it would require copying of strings (to nul
terminate them) in many more bits of C code, many of which execute more
frequently.  For code receiving strings to not depend on nul termination
can't be enforced by an assertion: it could only be policed by more
specialised means such as valgrind.

-zefram

• [perl #129058] Perl_do_vop: resulting string isn't alwaysnull-terminated by Dan Collins via RT
• Re: [perl #129058] Perl_do_vop: resulting string isn't alwaysnull-terminated by Zefram
• [perl #129058] Perl_do_vop: resulting string isn't alwaysnull-terminated by Dan Collins