develooper Front page | perl.perl5.porters | Postings from April 2018

[perl #131760] "Wide character" errors are undocumented

From:
Karl Williamson via RT
Date:
April 3, 2018 17:38
Subject:
[perl #131760] "Wide character" errors are undocumented
Message ID:
rt-4.0.24-29940-1522777099-163.131760-15-0@perl.org
On Mon, 17 Jul 2017 16:28:59 -0700, jkeenan wrote:
> On Mon, 17 Jul 2017 22:56:22 GMT, jkeenan wrote:
> > On Mon, 17 Jul 2017 17:29:28 GMT, steve.grazzini@grantstreet.com
> > wrote:
> > > perldiag has an entry for the "Wide character in %s" warning from
> > > doio.c:
> > >
> > > Wide character in %s
> > >         (S utf8) Perl met a wide character (>255) when it wasn't
> > > expecting one.  This
> > >         warning is by default on for I/O (like print).  The easiest
> > > way
> > > to quiet this warning
> > >         is simply to add the ":utf8" layer to the output, e.g.
> > > "binmode
> > > STDOUT, ':utf8'".
> > >         Another way to turn off the warning is to add "no warnings
> > > 'utf8';" but that is often
> > >         closer to cheating.  In general, you are supposed to
> > > explicitly
> > > mark the filehandle
> > >         with an encoding, see open and "binmode" in perlfunc.
> > >
> > > But it doesn't describe the several fatal "Wide character" errors
> > > elsewhere:
> > >
> > > pp_sys.c:2028:  Perl_croak(aTHX_ "Wide character in %s",
> > > OP_DESC(PL_op));
> > > sv.c:3734:          Perl_croak(aTHX_ "Wide character in %s",
> > > sv.c:3737:          Perl_croak(aTHX_ "Wide character");
> > > sv.c:8519:          Perl_croak(aTHX_ "Wide character in $/");
> > >
> > > Those should probably be documented, too. As it is, it's pretty
> > > confusing
> > > to users who
> > > get a fatal error and try to interpret it with the description of
> > > the
> > > unrelated warning.
> > >
> >
> > Thanks for your report.  Not only are these warnings under-
> > documented,
> > they're under-tested as well.
> >
> > I created a branch in which I went to the places you cited in the
> > source code and changed the first letter of the warning from 'W' to
> > 'A..D' respectively.  I then ran the the test suite.  Tests where one
> > of the 4 variant warnings and where we were attempting to match an
> > error message would then fail.
> >
> > This source code point was extensively exercised in the test suite:
> >
> > #####
> > sv.c:3734:          Perl_croak(aTHX_ "Wide character in %s",
> > #####
> >
> > I could not find locations at which the following source code points
> > were exercised in the test suite:
> >
> > #####
> > pp_sys.c:2028:  Perl_croak(aTHX_ "Wide character in %s",
> > OP_DESC(PL_op));
> >
> 
> Appears to have been added in this commit:
> 
> c9cb0f41 (Nicholas Clark              2006-04-29 23:33:36 +0000 2028)
> Perl_croak(aTHX_ "Wide character in %s", OP_DES
> 
> > sv.c:3737:          Perl_croak(aTHX_ "Wide character");
> >
> 
> Appears to have been added in this commit:
> 
> fa301091a (Jarkko Hietaniemi         2000-11-30 20:41:39 +0000  3737)
> Perl_croak(aTHX_ "Wide character");
> 
> 
> > sv.c:8519:          Perl_croak(aTHX_ "Wide character in $/");
> 
> Appears to have been added in this commit:
> 
> 4b3603a49 (Jarkko Hietaniemi         2000-10-17 14:11:31 +0000 2460)
> Perl_croak(aTHX_ "Wide character");

I looked at the code here.  The one at line 8519 happens is the user doesn't specify the :utf8 layer, and sets the slurp $/ to include a wide character.  So the advice given in the diagnostic applies; and we just don't have tests for it.  I suspect that the other ones are valid, but again don't have tests, and that they formerly were exercised, but things have changed to avoid them.  I know, for example, that we no longer accept wide characters in the bitwise operations like &, and so the calls that might have led to these lines of code getting hit are intercepted.

In any event, attached is a generalization to the wording in perldiag that attempts to handle all the cases  If I don't hear objections by April 19, I will apply it (so that it gets into 5.28)
-- 
Karl Williamson

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=131760



nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About