What to do about smoke failures with locale:en_US.utf8

(or some other locale)

We have smoke failures for configurations that use this.

Here is some irc discussion from yesterday concerning this, that petered 
out.  I'm bringing it up here so that it will be addressed.

tl;dr, what to do with syswrite, among other things, when PERL_UNICODE 
is set.  Should smokes be expecting that setting PERL_UNICODE will pass 
every test?  Could/should harness clear it, like it does PERL5OPT?

<khw> Any ideas about these two tests on an HP box
<khw> Integer overflow in hexadecimal number at t/broken_univ_can.t line 7.
<khw> # Connected to 127.0.0.1
<khw> syswrite() isn't allowed on :utf8 handles at t/Socket.t line 47.
<khw> # Looks like your test exited with 255 just after 2.
<xenu> Socket.t doesn't seem do anything unicode-related
<xenu> is PERL_UNICODE set to something?
<khw> xenu, it doesn't appear in the log 
https://perl5.test-smoke.org/logfile/110471
<dipsy> [ Smoke v5.31.10-25-g5de22a4 FAIL(F) HP-UX B.11.23/64 Intel(R) 
Itanium 2 processor/1600 MHz 2 ]
<xenu> https://perl5.test-smoke.org/report/110471
<dipsy> [ Smoke v5.31.10-25-g5de22a4 FAIL(F) HP-UX B.11.23/64 Intel(R) 
Itanium 2 processor/1600 MHz 2 ]
<xenu> it says that Socket.t fails only with 'locale:en_US.utf8' 
configuration, whatever it means
<khw> I used to know what that meant, but have forgotten
<[Tux]> khw, xenu, 
https://github.com/abeltje/Test-Smoke/blob/master/lib/Test/Smoke/Smoker.pm#L537
<xenu> so it set to an empty string which according to perlrun is 
equivalent to CSDL
<xenu> i'm not sure if we support that
<Grinnz> yeah, it needs to be unset not empty string
<xenu> it was clearly done on purpose
<Grinnz> PERLIO empty string is equivalent to unset, but PERL_UNICODE is not
<Grinnz> i don't think it was
<Grinnz> oh, it does delete $ENV{PERL_UNICODE} after
<Grinnz> if $perlio is not 'locale'
<xenu> anyway, i don't think it's reasonable to expect the whole test 
suite to pass when PERL_UNICODE is set, just like i wouldn't expect 
random PERL5OPT values not to break anything
<khw> [Tux]^^
<xenu> (i also think that the whole idea of PERL_UNICODE is stupid, but 
that's probably just my opinion)
<Grinnz> seems like $perlio would only be 'locale' if 'locale' option is set
<Grinnz> so this is being intentionally tested?
<Grinnz> i don't think changing the behavior of the test script is a 
good test
<Grinnz> and that's what PERL_UNICODE does
<pink_mist> xenu: that's just like .. your opinion man!
<Grinnz> so tldr: don't do those 'locale' configuration tests, all they 
will test is broken configuration
<Grinnz> tbf syswrite *shouldnt* be affected, but that is a cross we 
have to bear for now
<xenu> hmm
<xenu> come to think of it, that syswrite() change made PERL_UNICODE 
*completely* useless
<Grinnz> i have to agree, its not a good idea to set in general, you 
should instead use -C to affect oneliners only
<xenu> previously it only caused minor bugs that probably no one noticed
<xenu> but now it completely breaks all scripts that are using syswrite
<xenu> and "scripts" includes some random cpan dependencies you may not 
be aware of
<xenu> sadly, just like with taint mode, there are tons of resources 
that recommend using it
<xenu> 
https://www.perl.com/pub/2012/05/perlunicook-make-all-io-default-to-utf-8.html/
<xenu> ecample
<xenu> example
<Grinnz> it actually only affects the main program scope
<xenu> oh
<Grinnz> though Path::Tiny apparently checks the caller's scope for the 
setting
<Grinnz> so that's fun
<xenu> so it only partially breaks everything
<Grinnz> setting PERLIO=:utf8 on the other hand would break literally 
everything
<xenu> it's a mess
<Grinnz> and that's why i removed :utf8 from the PERLIO docs
<Grinnz> https://perldoc.pl/blead/perlrun#PERLIO
<dipsy> [ perlrun - how to execute the Perl interpreter - Perldoc Browser ]
<khw> So what's to be done here, now?  An email discussion, a ticket?

• What to do about smoke failures with locale:en_US.utf8 by Karl Williamson