Re: Deleting Lines From File

Hi ExecMan,

On Thu, 5 Apr 2012 13:44:57 -0700 (PDT)
ExecMan <artmerar@yahoo.com> wrote:

> Hi,
> 
> We have a Perl script that sends out emails.  It reads a sequential
> file and sends out the emails.  The file is big, like 500,000 lines.
> 
> Sometimes this process has gone down and it is a pain to see whom
> we've sent to and whom we have not.
> 
> Is there a way within the script to remove the addresses from the file
> as I am processing them?  So, if the process goes down, the file is
> left with only the people who still need to receive the email.
> 
> If not, does anyone have a good idea on how to implement something
> like this?
> 

Well, deleting segments (including lines naturally) from the middle of a file
is quite costly, because in UNIX and UNIX-like operating systems, a file is
nothing but a big bag of bytes, and as a result, you need to move the rest of
the file's contents backwards, which can take a while especially if your file is
large.

One option is to write a new file with modified content. A better option may be
to keep a counter of the processed lines elsewhere on the file system. And if
both of those options are not suitable, please consider using a database of
some sort instead (SQL-based or otherwise).

Regards,

	Shlomi Fish 

-- 
-----------------------------------------------------------------
Shlomi Fish       http://www.shlomifish.org/
List of Portability Libraries - http://shlom.in/port-libs

Trying to block Internet pornography is like climbing a waterfall and trying
to stay dry. (— one of Shlomi Fish’s Internet Friends)

Please reply to list if it's a mailing list post - http://shlom.in/reply .

• Deleting Lines From File by ExecMan
• Re: Deleting Lines From File by Shlomi Fish
• Re: Deleting Lines From File by Binish A.R