develooper Front page | perl.beginners | Postings from August 2011

Re: Spidering

Thread Previous | Thread Next
From:
Rob Dixon
Date:
August 1, 2011 10:51
Subject:
Re: Spidering
Message ID:
4E36E7A9.6030603@gmx.com
On 01/08/2011 11:03, VinoRex.E wrote:
>
> Hi everyone i am a  beginer for Perl can you give me a psedocode and a
> sample code for a spider program.It will be helpful in understanding web
> interfaces.Thank you

If you can't write your own pseudocode for a web spider then check
Bharathiar University for a more appropriate course. One version goes

   function fetchall(URL)
     content = get(URL)
     loop for it over findlinks(content)
       content = content + fetchall(it)
     return content
   end

Since the purpose of your efforts is to learn Perl, I think a module
like WWW::Mechanize is the wrong choice. To write a program that
accesses the internet, you should install and study the LWP library.

Rob

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About