Front page | perl.perl5.porters |
Postings from August 2016
Re: Encode utf8 warnings
Thread Previous
From:
pali
Date:
August 25, 2016 14:35
Subject:
Re: Encode utf8 warnings
Message ID:
20160825085635.GA12156@pali
On Thursday 25 August 2016 00:59:01 Father Chrysostomos wrote:
> pali at cpan.org wrote:
> > My point of view:
> >
> > Encode module is generic and not only UTF-8 related. Warning are enabled
> > only if Encode::FB_WARN or Encode::WARN_ON_ERR check value is passed.
> >
> > So I would expect that warning utf8 pragma does not interference with
> > Encode::encode/decode functions by default (when called without check
> > flags or with Encode::FB_DEFAULT).
> >
> > On the other hand when reading from file handle with :utf8 layer,
> > warnings are managed by warning utf8 pragma. And for me it make sense
> > that this should apply also for :encoding layer with UTF-8 (or other
> > Unicode) encodings.
> >
> > Currently core module PerlIO::encoding enable warnings by default for
> > :encoding layer and those warnings can be controlled only via variable
> > $PerlIO::encoding::fallback. Not via warnings utf8 pragma.
>
> Thank you for looking into the cause.
>
> > My proposal:
> >
> > Add new check flag to Encode module which will follow utf8 pragma
> > warnings state (e.g Encode::FOLLOW_UTF8_PRAGMA_WARN, you can invent
> > better name). And add this flag to PerlIO::encoding module by default.
> >
> > Then patch Encode::utf8 and Encode::Unicode modules (part of Encode
> > package) to use that new (FOLLOW_UTF8_PRAGMA_WARN) flag. That means to
> > show unicode warning if either one of this condition is true:
> > * FOLLOW_UTF8_PRAGMA_WARN is not set && WARN_ON_ERR is set
> > * FOLLOW_UTF8_PRAGMA_WARN is set && utf8 pragma warning is enabled
> >
> > That would means that utf8 warnings would be shown if WARN_ON_ERR is
> > passed to Encode, FOLLOW_UTF8_PRAGMA_WARN is not passed and utf8 pragma
> > warning is disabled.
> >
> > What do you think about it?
>
> I think your proposal will work. As for the name, I think
> FOLLOW_WARNINGS_PRAGMA reads better, but I still do not like it
> very much.
If this option will be chosen then somebody else maybe propose better
name. Basically I'm happy with any reasonable name...
> That said, it may be that we just need to document the $fallback
> variable more prominently somewhere. I certainly was not aware of it,
> but it *is* documented in PerlIO::encoding, so I am to blame for that.
> But how many people make the same mistake?
There are couple of mistakes! Some already reported that setting
"no warnings utf8;" did not turned off those Encode utf8 warnings and
because of that some "hacks" were added into Encode::encode/decode
functions. And that caused another problems as those who expected
warnings via Encode flags are now ignored with no warnings... I pointed
to Encode git commits in previous email... Basically Encode module is
inconsistent between versions.
> It may also be that respecting just the 'utf8' warnings in
> PerlIO::encoding itself (and passing warn flags to Encode if utf8
> warnings are enabled) would suffice.
That would does not work as PerlIO::encoding flags are IIRC passed when
opening file. Not at time when setting warnings.
> I am not sure which is the best way forward.
PerlIO::encoding is core package, not on cpan. So people behind perl
code needs to decide how it should work. Changes in Encode module is not
enough...
I would like to see this fixed in next stable perl version. Who can help
with decision of this warnings mess? Maybe Karl (CCed)?
(PS: Please CC me as I'm not subscribed to this high volume list)
Thread Previous