develooper Front page | perl.perl5.porters | Postings from July 2017

Re: [perl #131685] Rename utf8::is_utf8() (and other functions)

Thread Next
From:
H.Merijn Brand
Date:
July 1, 2017 16:54
Subject:
Re: [perl #131685] Rename utf8::is_utf8() (and other functions)
Message ID:
20170701185353.4e5d28cd@pc09.procura.nl
On Sat, 01 Jul 2017 09:03:18 -0700, (via RT)
<perlbug-followup@perl.org> wrote:

> # New Ticket Created by   
> # Please include the string:  [perl #131685]
> # in the subject line of all future correspondence about this issue. 
> # <URL: https://rt.perl.org/Ticket/Display.html?id=131685 >
> 
> 
> Hi!
> 
> This is continuation from original discussion about renaming 
> utf8::is_utf8() to utf8::is_upgraded() which can be found at:
> https://www.nntp.perl.org/group/perl.perl5.porters/2017/02/msg243068.html
> 
> Problem is that in more perl modules is used this incorrect code 
> pattern:
> 
>   use utf8;
> 
>   my $value = func();
>   if (utf8::is_utf8($value)) {
>     utf8::encode($value);
>   }
> 
> In most cases module developers think that utf8::is_utf8() returns true 
> when it is needed to manually encode argument into UTF-8 bytes. Which is 
> of course wrong.
> 
> Reason for this is poor name of function utf8::is_utf8() and also poor 
> documentation about this function.
> 
> Functions utf8::is_utf8(), utf8::upgrade() and utf8::downgrade() changes 
> internal string representation, which is fully invisible for pure perl 
> code, and therefore I think all those functions should be in Internals 
> namespace.
> 
> I'm proposing following rename of functions:
> 
> utf8::is_utf8() --> Internals::uses_string_wide_storage()
> utf8::upgrade() --> Internals::upgrade_string_to_wide_storage()
> utf8::downgrade() --> Internals::downgrade_string_from_wide_storage()

I am still objecting, as this will also break code that uses those
functions as intended and correctly.

As these are not XS, Devel::PPPort won't help (assuming authors use
D::P on XS modules to guarantee backward compat)

I'd loath to change/fix every occurrence of code that uses any of these
three correctly, as that code is brittle to start with and probably
hard to fix when broken.

> Plus adding backward compatible aliases to make existing code works like 
> before.

Then why add new functions in the first place?

> As all those functions should be used only for debugging purposes (e.g. 
> test cases for XS code) or when dealing with buggy XS module, I'm 
> proposing starting to throw warning (e.g. since v5.28.0) when those 
> functions are called. For those who are dealing with internals, can turn 
> warning off by no warnings 'experimental::internal';

No, please. Most correct uses will be in dark distant corners, hidden
in modules you don't want to touch anyway.

> I'm attaching patches which:
> 
> * Add new warning category 'experimental::internal'
> * Rename utf8 functions
> * Update perldoc utf8 documentation

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.27   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About