Memory leak in Text::CSV_XS

Hi porters,

At my company we work on a software which handle a lot of different
type of input file.
The software is a long lasting software running on a server.

We recently had the need to add the possibility to read data from CSV
file and we add a little piece of driver using Text::CSV_XS (we need
speed, a lot of speed).

But using Text::CSV_XS 0.93 made you software blow up under ram
leaking (the script was taking something like 10Gb of memory, but
usualy the script run on 250Mb ~ 500Mb of memory).

We create a little script presenting the issue :

--------------------------------------------------
use Text::CSV;

$i = 0;

foreach $file (@ARGV) {
    open $FILE, "<$file";

    my $CSVFile = Text::CSV->new(
        {   'quote_char'     => '"',
            'sep_char'       => ';',
            'always_quote'   => 0,
            'quote_space'    => 1,
            'quote_null'     => 1,
            'binary'         => 1,
            'blank_is_undef' => 1,
            'empty_is_undef' => 1,
        }
    );

    my $header_ref  = $CSVFile->getline($FILE);
    my $columns_ref = $CSVFile->getline($FILE);

    $CSVFile->column_names( @{$columns_ref} );

    my $lines = $CSVFile->getline_hr_all($FILE);

    close FILE ;

    print "File $i done\n" ;
     $i++ ;

}
---------------------------------------------------

By calling this script with 200 csv file in input (and with
Text::CSV_XS installed), my perl process increase its memory at every
loop, if I remove Text::CSV_XS (and thus using the pure perl
implementation of Text::CSV), my perl process use a fixed quantity of
memory.

We got this issue on different system :
Archlinux, perl v5.16.2 x86_64-linux-thread-multi (from arch
repository),  Text::CSV_XS 0.93
Debian Sqeeze, v5.10. x86_64-linux-gnu-thread-multi (from debian
repository, TexT::CSV_XS from debian repository (libtext-csv-xs-perl).

Thanks for your help and insight.

--
Mayeu
http://6x9.fr

• Memory leak in Text::CSV_XS by Matthieu Maury
• Re: Memory leak in Text::CSV_XS by Alexandr Ciornii
• Re: Memory leak in Text::CSV_XS by Matthieu Maury
• Re: Memory leak in Text::CSV_XS by H.Merijn Brand
• Re: Memory leak in Text::CSV_XS by Kent Fredric