develooper Front page | perl.libwww | Postings from November 2001

Re: 500 Read error

Thread Previous | Thread Next
From:
Gisle Aas
Date:
November 29, 2001 08:51
Subject:
Re: 500 Read error
Message ID:
lrvgft5xne.fsf@caliper.ActiveState.com
"Beyond Control Inc." <beyondcontrol@runbox.com> writes:

> Unfortunatly, if I start it now it *will* work for 10 hours, then die.

How many pages has been fetched at this point?  6 each hour for 12
hours are 72, right?

Does it fail quickly if you don't wait for 10 minutes between each
fetch?

Regards,
Gisle

> This is why I find this to be so weird, since I look like IE5, and I always
> create a new UserAgent object so that I won't get cookied or anything like
> that...
> 
> ----- Original Message -----
> From: "William R Ward" <bill@wards.net>
> To: <libwww@perl.org>
> Sent: Wednesday, November 28, 2001 11:55 PM
> Subject: Re: 500 Read error
> 
> 
> > beyondcontrol@runbox.com (Beyond Control Inc.) writes:
> > > I've written a small script that fetches a page every 10 minutes.
> > >
> > > After 12-14 hours, it suddenly stops fetching the page, say there's a
> > > read error (500)
> >
> > Is this something that works once you start it up again?  Or did it
> > work for a while and now you can't get it to work anymore?  If the
> > latter, it could be that the web administrators noticed your script
> > and blocked it.  Some web site owners don't like people slurping their
> > data when the ads or copyright notices might not be slurped as well.

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About