develooper Front page | perl.libwww | Postings from September 2001

UserAgent hangs. Please help.

Thread Next
Gil Vidals
September 17, 2001 17:20
UserAgent hangs. Please help.
Message ID:
Hello LWP members,

This is my first message to the list, and I am grateful for the list and the benefit that it provides to all of us who rely on LWP.

UserAgent hangs very often upon retrieving certain urls. I have to kill the process. Please show me how to set up LWP so that it will timeout after the number of seconds that I designate. 

I ran the code on the following url and it just hangs for a long time. Longer than my MAX_SECONDS of 30 and ABSMAX_SEC of 40 seconds.

I am using:
    $ua->use_eval(0); ## used to override the disabling of __DIE__ 

I am running RH Linux with version 1.80 of LWP and version 1.1603 of IO::Socket.  IO::Socket::INET has the timeout section commented out by the author. Is this a mistake? Could this be the problem?

Gil Vidals

sub get_page
## Description: gets the source of the page
## Input:  $url (string of url to get the source of)
## Output:  response content if successful
## Effects:  none
## Subroutines used:  none
## Tables used:  none
## Libraries used:  LWP::UserAgent, Carp
   my $self  = shift;
   my ($url) = @_;

    return -1 unless $url;

    ## add http:// if not already on there ##
    $url = "http://" . $url if ($url !~ m@^http://@i);
    carp "Getting page $url";

    my $ua = LWP::UserAgent->new;
    ### How much time we wait before receiving data. According to Jisun this timer might be reset each time
    ### some data comes through for the same page.
    $ua->agent('Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)');
    my $start = time;

    ### allow only X seconds to complete the download of a given page
    my $response = $ua->request(HTTP::Request->new(GET=> "$url"),
                                 sub {
                                    my($data, $response) = @_;
                                    die "ABS-Timeout" if (time - $start) > $ABSMAX_SEC;

    if ($response->is_success)
        my $base = $response->base || '';
        carp "\n\n", $base, '<------------$response->base-----------------------------', "\n\n" if $DEBUG;
        return ($response->content, $base);
    else {
        carp 'Page timed out' if $DEBUG;
        return " ";

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About