develooper Front page | perl.beginners | Postings from March 2002

Cleaning poorly formated csv files

Thread Next
Dave Chappell
March 16, 2002 15:24
Cleaning poorly formated csv files
Message ID:

I’m new to perl and this list. I am trying to create a script that cleans up
csv files in the following ways:

-Remove tab characters
-Remove trailing commas
-Replace ^’ character sequence with a comma
-Want to preserve the CRLF and the end of each line

Then I output the results to another file. I managed to get the tabs removed
but I am having difficulties working on the next item in my task list,
removing trailing commas. Any insight or comments would be appreciated

#Example of whats in the in.txt
ABC,  20020313,"        02:51:47            ",”some text”,,,,,,,,
ABC,  2002     0313,"   02:51:47            ",”more text”,
ABC,  2002031            3,"        02:51:47            ",even^’more^’text,
ABC,  20020313,"        02:51:47            ",,,,,,,,

#My Code
$fileIN = "in.csv"; # Dirty csv file
$fileOUT = "out.csv";

open (IN, $fileIN) or die "Cannot open $fileIN for read :$!";
open (OUT,"> $fileOUT") or die "Cannot open $fileOUT for writing :$!";

while (<IN>) {
            $_=~ tr/\t//d;
            print OUT "$_";
            print "$_\n";


Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About