develooper Front page | perl.libwww | Postings from November 2001

Re: 500 Read error

Thread Previous
From:
Beyond Control Inc.
Date:
November 29, 2001 09:52
Subject:
Re: 500 Read error
Message ID:
005801c178ff$8996ce80$59ac003e@nevermind
Yes, when I fail to get the page (simple is->success() test) I try again 5
times, and then give up and go to sleep again.


----- Original Message -----
From: "Gisle Aas" <gisle@ActiveState.com>
To: "Beyond Control Inc." <beyondcontrol@runbox.com>
Cc: <libwww@perl.org>; "William R Ward" <bill@wards.net>
Sent: Thursday, November 29, 2001 6:50 PM
Subject: Re: 500 Read error


> "Beyond Control Inc." <beyondcontrol@runbox.com> writes:
>
> > Unfortunatly, if I start it now it *will* work for 10 hours, then die.
>
> How many pages has been fetched at this point?  6 each hour for 12
> hours are 72, right?
>
> Does it fail quickly if you don't wait for 10 minutes between each
> fetch?
>
> Regards,
> Gisle
>
> > This is why I find this to be so weird, since I look like IE5, and I
always
> > create a new UserAgent object so that I won't get cookied or anything
like
> > that...
> >
> > ----- Original Message -----
> > From: "William R Ward" <bill@wards.net>
> > To: <libwww@perl.org>
> > Sent: Wednesday, November 28, 2001 11:55 PM
> > Subject: Re: 500 Read error
> >
> >
> > > beyondcontrol@runbox.com (Beyond Control Inc.) writes:
> > > > I've written a small script that fetches a page every 10 minutes.
> > > >
> > > > After 12-14 hours, it suddenly stops fetching the page, say there's
a
> > > > read error (500)
> > >
> > > Is this something that works once you start it up again?  Or did it
> > > work for a while and now you can't get it to work anymore?  If the
> > > latter, it could be that the web administrators noticed your script
> > > and blocked it.  Some web site owners don't like people slurping their
> > > data when the ads or copyright notices might not be slurped as well.
>
>


Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About