develooper Front page | perl.perl5.porters | Postings from September 2017

[perl #131683] Encode::ONLY_PRAGMA_WARNINGS in$PerlIO::encoding::fallback

Thread Previous
From:
Tony Cook via RT
Date:
September 11, 2017 00:01
Subject:
[perl #131683] Encode::ONLY_PRAGMA_WARNINGS in$PerlIO::encoding::fallback
Message ID:
rt-4.0.24-3005-1505088100-1765.131683-15-0@perl.org
On Wed, 30 Aug 2017 02:22:26 -0700, pali@cpan.org wrote:
> On Tuesday 29 August 2017 18:46:43 Tony Cook via RT wrote:
> > b) there's two compilation errors in the build:
> >
> > Unicode.xs: In function ‘XS_Encode__Unicode_decode’:
> > Unicode.xs:342:2: error: ISO C90 forbids mixed declarations and code
> > [-Werror=declaration-after-statement]
> >   U32 flags = 0;
> >   ^
> > Unicode.xs: In function ‘XS_Encode__Unicode_encode’:
> > Unicode.xs:446:2: error: ISO C90 forbids mixed declarations and code
> > [-Werror=declaration-after-statement]
> >   UV ord = utf8n_to_uvuni(s, e-s, &len, flags);
> >   ^
> 
> Ok, that can be fixed.
> 
> > On Mon, 28 Aug 2017 00:50:49 -0700, pali@cpan.org wrote:
> > > All above warning should be sent when user calls Encode with
> > > FB_WARN
> > > bit, independently of lexical warnings. This is how all other
> > > Encode
> > > module works, also in this way is Encode API designed and
> > > documented.
> >
> > The problem is your patch makes these warnings lexically scoped with
> > *only* FB_WARN:
> >
> > $  ./perl -Ilib -MEncode -le 'no warnings qw(surrogate); my $x =
> > encode("utf-16", (my $y = chr(0xdc10)), &Encode::FB_WARN); print "no
> > croak"'
> > no croak
> >
> > $  ./perl -Ilib -MEncode -le 'my $x = encode("utf-16", (my $y =
> > chr(0xdc10)), &Encode::FB_WARN); print "no croak"'
> > UTF-16 surrogate U+DC10 in goto at -e line 1.
> > no croak
> >
> > The utf8n_to_uvuni() function *already* checks that these warnings
> > are lexically enabled.  Also, it only checks them when it actually
> > finds the problem being reported.
> 
> Ah :-( So we need utf8n_to_uvuni() function which report warnings even
> when they are lexically disabled.

> > Your code however calculates flags whether or not there's an error,
> > and currently does it on every character encoded.  When
> > ONLY_PRAGMA_WARNINGS is set this results in three calls to
> > Perl_ckwarn() inside the encoding loop.  (This code should be using
> > ckWARN_d() since these are default on warnings, but it should just
> > set them all and let the API do the checking it does anyway.)
> >
> > An issue with changes to Encode::encode() as you might see above is
> > the opaqueness of the warning:
> >
> > UTF-16 surrogate U+DC10 in goto at -e line 1.
> 
> Yes, this is something which needs to be fixed.
> 
> > My code doesn't contain a goto.  I think the only real fix to this
> > would be to rewrite Encode::encode/decode in C.
> 
> Do you have your implementation? At least I do not see any other patch
> in this ticket.

The "My code" I was referring to is the one-liner above.  I meant it would confuse the user whose code doesn't include a goto (just a call to encode/decode()).

> 
> I used goto in Encode::encode/decode dispatcher functions, so warnings
> from modules would be correctly propagated to caller of the
> Encode::encode/decode function.

I guessed that, but the existing encode/decode functions already has code that handles that:

    ...
    my $octets;
    if ( ref($enc) eq 'Encode::Unicode' ) {
        my $warn = '';
        {
            local $SIG{__WARN__} = sub { $warn = shift };
            $octets = $enc->encode( $string, $check );
        }
        warnings::warnif('utf8', $warn) if length $warn;
    }
    ...


> It is possible to reimplement Encode::encode/decode function in C/XS
> in
> same way how it is implemented in my patch, but without that warning
> from goto?

The most complex part would probably be the call to find_encoding(), the rest I think is relatively simple.

Tony

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=131683

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About