develooper Front page | perl.perl6.users | Postings from September 2017

DateTime.Str default formatter with sprintf is pretty slow

From:
=?utf-8?Q?Thor_Michael_St=C3=B8re?=
Date:
September 19, 2017 16:46
Subject:
DateTime.Str default formatter with sprintf is pretty slow
Message ID:
72B7DBF0-8356-4044-949E-CD595B74EBA9@gmail.com
Hey,

When I profiled my "read CSV, munge, write CSV" script to see why it is a bit on the slow side DateTime.Str stood out. I saw the default formatter eventually reached sprintf, which consumed a lot of time. Each output line from my script has one date and time in ISO 8601 format, and when I changed the script so that they were stringified without sprintf the total runtime dropped from ~7 to ~4.5 seconds on a short run that converts 800 lines to 1100.

A test case for this:

#!/usr/bin/env perl6

# Arg: Quick DateTime
sub MAIN( :$qDT = False ) {
    while my $x++ < 1000 {
        say DateTime.new( year => 2017, hour => (^24).roll, minute => (^60).roll,
                          formatter => ( $qDT ?? &formatISO8601 !! Callable ) );
    }
}

sub formatISO8601( DateTime $dt ) {
    ( 0 > $dt.year     ?? '-'~ zPad( $dt.year.abs, 4 ) !!
      $dt.year <= 9999 ?? zPad( $dt.year, 4 )
                       !! '+'~ zPad( $dt.year, 5 ) ) ~'-'~
    zPad( $dt.month, 2 ) ~'-'~ zPad( $dt.day, 2 ) ~'T'~
    zPad( $dt.hour, 2 ) ~':'~ zPad( $dt.minute, 2 ) ~':'~
       ( $dt.second.floor == $dt.second
           ?? zPad( $dt.second.Int, 2 )
           !! $dt.second.fmt('%09.6f') )
     ~
     ( $dt.timezone == 0
       ?? 'Z'
       !! $dt.timezone > 0
          ?? ( '+' ~ zPad( ($dt.timezone/3600).floor, 2 ) ~':'~
                     zPad( ($dt.timezone/60%60).floor, 2 ) )
          !! ( '-' ~ zPad( ($dt.timezone.abs/3600).floor, 2 ) ~':'~
                     zPad( ($dt.timezone.abs/60%60).floor, 2 ) ) )
}

sub zPad($s, $p) {'0' x $p - $s.chars ~ $s}

Running that with —qDT results in a bit less than a 4x speedup for me, with results piped to /dev/null. I wouldn’t know whether doing something like this in general is a good idea, of if sprintf/fmt should be improved.

Putting together that example, I noticed that Date.today also seems slow, taking about twice as long as DateTime.new. Replacing "year => 2017” with "date => Date.today" in the example above slows it down by about 400ms on my (slow) laptop computer. I don't call that in a loop like in this in my actual script anyhow, so that's no biggie for me, but it’s the sort of thing that might get put in a tight loop so it’s something to be aware of.

-------
Thor Michael Støre
thormichael@gmail.com
Database guy in Kongsberg, Norway
Mobile: +47 97 15 14 09




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About