develooper Front page | perl.libwww | Postings from February 2003

Using a callback subroutine

Thread Next
Octavian Rasnita
February 15, 2003 08:05
Using a callback subroutine
Message ID:
Hi all,

I want to download the top of an html file, parse it for searching a string,
and if that string is found, I want to cancel downloading the rest of the
If the string is not found, download the second part, and so on.

The problem is that even though I've set the chunk size to 6500, some chunks
have only a few hundread bytes, and this could be a problem because it might
split the string and it won't be found.

Can you tell me if it is a problem with my script, or with LWP?

Thank you very much!


use strict;
use LWP::UserAgent;

my $out = "/var/www/teddy/writable/e1.txt";

my $ua = LWP::UserAgent->new(env_proxy => 0,
timeout => 40,
keep_alive => 1
$ua -> agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)');

my $location = 'http://.../';
my $min = 0;
my $max = 100;

ADDR: for (my $i = $min;$i < $max; $i++) {
my $request = HTTP::Request->new('GET', $location);
$request -> header('REFERER' => $location);
my $response = $ua->request($request, \&callback, 6500);

sub callback {
my($data, $response, $protocol) = @_;

my @a = $data =~ /$string/i;
if ($a[0]) {
open (OUT, ">>$out") or die "Can't write to $out - $!";
print OUT $a[0], "\n";
close OUT;
next ADDR;

Thank you.

Teddy's Center:

Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About