develooper Front page | perl.perl5.porters | Postings from July 2020

Re: Clarification about features on/off by default

Thread Previous
From:
H.Merijn Brand
Date:
July 22, 2020 12:56
Subject:
Re: Clarification about features on/off by default
Message ID:
20200722145554.779507f6@pc09.procura.nl
On Tue, 21 Jul 2020 16:26:18 -0400, Eric Brine <ikegami@adaelis.com>
wrote:

> I was curious as to what was the cause of the errors. The first error
> (test 255) is the result of calling `lc` before decoding the string
> instead of after. This is an existing bug, not one introduced by
> unicode_strings. unicode_strings simply change the outcome of the bug
> (admittedly for the worse).
> 
> TEST (using latest Perl and latest module, unchanged):
> 
> use strict;
> use warnings;
> use feature qw( say );
> 
> use utf8;
> use open ':std', ':encoding(UTF-8)';
> 
> use Text::CSV_XS qw( );
> 
> {
>    my $file = "Eric,ÉRIC\n";
>    utf8::encode($file);
>    open(my $fh, '<:raw', \$file) or die $!;
> 
>    my $csv = Text::CSV_XS->new({ binary => 1, auto_diag => 2 });
>    $csv->header($fh);
> 
>    say sprintf('%1$vX %1$s %2$s', $_, lc($_)) for $csv->column_names;
> }
> 
> OUTPUT:
> 
> 65.72.69.63 eric eric
> C9.72.69.63 Éric éric
> 
> column_names returns partially-lowercased decoded text

THANK YOU!

Found it and fixed it.

Note, $csv->header already returns the column names, so
--8<---
use utf8;
use open ":std", ":encoding(UTF-8)";

use Data::Peek;
use Text::CSV_XS;

{   my $file = "Eric,ÉRIC\n";
    utf8::encode ($file);
    open my $fh, "<:raw", \$file or die $!;

    my $csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 2 });
    my @hdr = $csv->header ($fh);

    DPeek for @hdr;
    say "--";
    say   for @hdr;
    }
-->8---

is probably easier to digest.

All tests now PASS with unicode_strings

I will have to mould above into a test case and I will do a
release asap.

> On Sat, Jul 18, 2020 at 2:31 PM H.Merijn Brand <perl5@tux.freedom.nl> wrote:
> 
> > On Fri, 17 Jul 2020 13:28:46 -0600, Karl Williamson
> > <public@khwilliamson.com> wrote:
> >  
> > > On 7/17/20 12:40 PM, Eric Brine wrote:  
>  [...]  
> > >
> > > It hasn't been decided yet.  There is no one opposed AFAIK to
> > > enabling it if feasible.  Rumor has it that a bunch of code
> > > breaks with this enabled; and I'm trying to track that down.
> > > Each time I'm told person X knows code that breaks, and I've
> > > contacted that person, I'm told, "No, I don't know of any code
> > > that breaks".  The final person X I haven't talked to yet is AFK
> > > for a week.  
> >
> > Text::CSV_XS with feature unicode_strings:
> >
> > t/85_util.t     (Wstat: 26112 Tests: 1448 Failed: 102)
> >   Failed tests:  255, 257, 259, 261, 264, 266, 268, 272
> >                 274, 276, 278, 285, 438, 440, 442, 444
> >                 451, 454, 456, 458, 460, 463, 465, 467
> >                 471, 473, 475, 477, 484, 637, 639, 641
> >                 643, 650, 653, 655, 657, 659, 662, 664
> >                 666, 670, 672, 674, 676, 683, 836, 838
> >                 840, 842, 849, 852, 854, 856, 858, 861
> >                 863, 865, 869, 871, 873, 875, 882, 1035
> >                 1037, 1039, 1041, 1048, 1051, 1053, 1055
> >                 1057, 1060, 1062, 1064, 1068, 1070, 1072
> >                 1074, 1081, 1234, 1236, 1238, 1240, 1247
> >                 1250, 1252, 1254, 1256, 1259, 1261, 1263
> >                 1267, 1269, 1271, 1273, 1280, 1433, 1435
> >                 1437, 1439, 1446
> >   Non-zero exit status: 102
> > Files=28, Tests=52129, 20 wallclock secs ( 3.46 usr  0.28 sys +
> > 19.25 cusr  0.43 csys = 23.42 CPU)
> > Result: FAIL
> > Failed 1/28 test programs. 102/52129 subtests failed.
> >
> > the rest PASSes.
> >
> > FWIW feel free to give me a PR for an updated 85_util.t that PASSes
> > from 5.6.1 to 5.33.0
> >
> > Other than these, Text::CSV_XS is now 'use p7;' and 'use standard;'
> > compliant.

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.31      porting perl5 on HP-UX, AIX, and Linux
https://useplaintext.email  https://tux.nl  http://www.test-smoke.org
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About