develooper Front page | perl.perl5.porters | Postings from July 2017

Re: [perl #131685] Rename utf8::is_utf8() (and other functions)

Thread Next
H.Merijn Brand
July 1, 2017 16:54
Re: [perl #131685] Rename utf8::is_utf8() (and other functions)
Message ID:
On Sat, 01 Jul 2017 09:03:18 -0700, (via RT)
<> wrote:

> # New Ticket Created by   
> # Please include the string:  [perl #131685]
> # in the subject line of all future correspondence about this issue. 
> # <URL: >
> Hi!
> This is continuation from original discussion about renaming 
> utf8::is_utf8() to utf8::is_upgraded() which can be found at:
> Problem is that in more perl modules is used this incorrect code 
> pattern:
>   use utf8;
>   my $value = func();
>   if (utf8::is_utf8($value)) {
>     utf8::encode($value);
>   }
> In most cases module developers think that utf8::is_utf8() returns true 
> when it is needed to manually encode argument into UTF-8 bytes. Which is 
> of course wrong.
> Reason for this is poor name of function utf8::is_utf8() and also poor 
> documentation about this function.
> Functions utf8::is_utf8(), utf8::upgrade() and utf8::downgrade() changes 
> internal string representation, which is fully invisible for pure perl 
> code, and therefore I think all those functions should be in Internals 
> namespace.
> I'm proposing following rename of functions:
> utf8::is_utf8() --> Internals::uses_string_wide_storage()
> utf8::upgrade() --> Internals::upgrade_string_to_wide_storage()
> utf8::downgrade() --> Internals::downgrade_string_from_wide_storage()

I am still objecting, as this will also break code that uses those
functions as intended and correctly.

As these are not XS, Devel::PPPort won't help (assuming authors use
D::P on XS modules to guarantee backward compat)

I'd loath to change/fix every occurrence of code that uses any of these
three correctly, as that code is brittle to start with and probably
hard to fix when broken.

> Plus adding backward compatible aliases to make existing code works like 
> before.

Then why add new functions in the first place?

> As all those functions should be used only for debugging purposes (e.g. 
> test cases for XS code) or when dealing with buggy XS module, I'm 
> proposing starting to throw warning (e.g. since v5.28.0) when those 
> functions are called. For those who are dealing with internals, can turn 
> warning off by no warnings 'experimental::internal';

No, please. Most correct uses will be in dark distant corners, hidden
in modules you don't want to touch anyway.

> I'm attaching patches which:
> * Add new warning category 'experimental::internal'
> * Rename utf8 functions
> * Update perldoc utf8 documentation

H.Merijn Brand   Perl Monger
using perl5.00307 .. 5.27   porting perl5 on HP-UX, AIX, and openSUSE

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About