Very Sluggish Code

Hello Folks;

I see an earlier post about sluggish code - I am not really sure what I am doing, so I let me post my entire script here.

++++++++++++++++++++++++++++++++++++++++++++++++++

#!/usr/bin/perl

#use strict;
use DBI;
use XML::XPath;
use XML::XPath::XMLParser;

# Set the input dir where all of the XML files live
my $input_dir = ".";

# Begin reading the directory
opendir(DIR, $input_dir) || die "sorry shit the bed $input_dir: $!";

# Read them into an array
my @files_in_dir = grep { /xml/ } readdir(DIR);
closedir DIR;

# connect to database and create parser object
my $dbh = DBI->connect ("DBI:mysql:can_us_ratecenters",
                           "xxxx", "xxxxxx",
                           { RaiseError => 1, PrintError => 0});

# clear the database - new DIDs coming in
$dbh->do ('TRUNCATE TABLE rc_city_town');

#Now the fun begins - read each file and put it in the database
foreach my $f (@files_in_dir) {
        open IN, "<$f";

my $xp = XML::XPath->new (filename => "./$f");
my $nodelist = $xp->find ("//row");
foreach my $row ($nodelist->get_nodelist ())
   {
       $dbh->do (
           "INSERT IGNORE INTO rc_city_town (state_prov, city_town, did_number) VALUES (?,?,?)",
               undef,
               $row->find ("state")->string_value (),
               $row->find ("ratecenter")->string_value (),
               $row->find ("number")->string_value (),
           );
   }
        close IN;
}
$dbh->disconnect ();


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

As far as I can tell, the script works without any compilation errors. 

I ran it with the Perl debugger ( perl -d scriptname.pl) and stepped through it.

It seems to bottleneck at the first foreach{} loop - probably not the best use case.

To get this script to stop, I have to kill the process - it has been running for  over an hour - it populates the database, but I want to do this right.

However, I am sure there is someone smarter than I and can look at this and say "..oh, there's your problem right there..."

However, I am not a Perl guru. So, I am relying on some assistance from the crowd.

If you have any questions, please feel free. I will take any assistance I can get.

Thanks  - Glen

• Very Sluggish Code by GlenM
• Re: Very Sluggish Code by Jenda Krynicky
• Re: Very Sluggish Code by timothy adigun
• Re: Very Sluggish Code by Ken Slater
• Re: Very Sluggish Code by Ken Slater