I would like to use a PERL script to logon to a web site, navigate web
pages, select links for files to be downloaded, then receive the
files. The names of the files will be dynamic changing daily. The
reason I want to use PERL to do this is to automate the process so a
person doesn't have to do it. Does anyone have an example script that
does something like this?
Thank you.
Here I'll give you are break. You have separate opeartions,
the logon, the page retreival, the link extraction, and the individual
retrieval.
Googling and groups.googling will give examples of all those
steps, but here are some basics.
This script will download a single url. But you can get multiple
urls, by getting the page, using something like
#!/usr/bin/perl
use strict;
use warnings;
use Data:
umper;
use WWW::Mechanize;
my $a = WWW::Mechanize->new();
$a->get( '
' );
print $_->url,"\n" for @{ $a->links };
__END__
You can also just use LWP::UserAgent instead of mechanize
to get the page, then extract the links manually, with
HTML::LinkExtor
You can groups.google search for examples of LinkExtor.
As a matter of fact, if you google enough, you will find scripts
that already do what you want, probably using curl or wget.
#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
# don't buffer the prints to make the status update
$| = 1;
my $ua = LWP::UserAgent->new();
my $received_size = 0;
my $url = '
http://zentara.net/zentara1.avi';
print "Fetching $url\n";
my $filename = substr( $url, rindex( $url, "/" ) + 1 );
#print "$filename\n";
open( IN, ">$filename" ) or die $!;
my $request_time = time;
my $last_update = 0;
my $response = $ua->get($url,
':content_cb' => \&callback,
':read_size_hint' => 8192,
);
print "\n";
close IN;
sub callback {
my ($data, $response, $protocol) = @_;
my $total_size = $response->header('Content-Length') || 0;
$received_size += length $data;
print IN $data;
# write the $data to a filehandle or whatever should happen
# with it here.
my $time_now = time;
# this to make the status only update once per second.
return unless $time_now > $last_update or $received_size ==
$total_size;
$last_update = $time_now;
print "\rReceived $received_size bytes";
printf " (%i%%)", (100/$total_size)*$received_size if $total_size;
printf " %6.1f/bps", $received_size/(($time_now-$request_time)||1)
if $received_size;
}
__END__
zentara
