develooper Front page | perl.perl5.porters | Postings from April 2021

Re: Perl 7: Fix string leaks?

Thread Previous | Thread Next
From:
Yuki Kimoto
Date:
April 1, 2021 01:06
Subject:
Re: Perl 7: Fix string leaks?
Message ID:
CAExogxOk7kHGZD1y2qMseLzr6wEipMRdzNuZwRg=zQBRVr1Q7g@mail.gmail.com
I have a question about the following code to understand Perl strings.

-----------------------------------------------------
use strict;
use warnings;
use utf8;
use Encode 'encode', 'decode';
use Devel::Peek;

# ASCII range
my $text = 'abc';

# 0
print "A. " . (utf8::is_utf8($text) ? 1 : 0) . "\n";
Devel::Peek::Dump $text;
print "\n";

my $bytes = encode('UTF-8', $text);

# 0
print "B. " . (utf8::is_utf8($bytes) ? 1 : 0) . "\n";
Devel::Peek::Dump $bytes;
print "\n";

my $text_again = decode('UTF-8', $bytes);

# 1
print "C. " . (utf8::is_utf8($text_again) ? 1 : 0) . "\n";
Devel::Peek::Dump $text_again;
print "\n";
------------------------------------------------------

 "use utf8" don't turn on utf8 flag of ascci string.

On the other hand Encode::decode turn on utf8 flag of ascii string.

Is this design mistake or have some intention?



2021年3月27日(土) 20:48 Felipe Gasper <felipe@felipegasper.com>:

> If “gently breaking” changes in Perl 7 are possible, would fixing these
> conspicuous leaks in Perl’s string storage abstraction be possible?
>
> 1) Change the default XS typemap such that `char *` and `const char *` use
> SvPVbyte_nolen. The status quo here uses SvPV_nolen, which leaks the SV
> internals without giving the XS code any means to “recover” from the leak
> (i.e. to query the SV’s UTF8 bit).
>
> 2) Change exec, mkdir, et al. to use SvPVbyte_nolen rather than
> SvPV_nolen. This fixes the same problem. It’s what CPAN Sys::Binmode does
> currently; this would apply that change to core instead.
>
> Both of these will break code that decodes strings without manually
> encoding them for output to a byte layer, so they’re not really workable in
> Perl 5. But they’re also eminent examples of desirable-but-breaking fixes.
>
> I’m happy to work on these when/if it’s decided that they’re agreeable.
>
> cheers,
> -Felipe

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About