Sipering with javascript.

[TLOlczyk]

I'd like to write a program to spider certain websites
which contain javascript.
One example is a local newspaper providing tv listings.
The page is set up in the following way, it brings up
the listing for half of todays channels for the next
three hours in a grid certain buttons advance the
grid by executing javascript commands ( advance previous
scroll etc, ) Rather than sit there wait half a minute for the next
page to load I thought I would write a script that searches for
the shows and extracts the pages with the shows I want on it.

Another example almost anyone can understand, is the "Look inside
feature " of amazon, where three or four pages of the book plus
toc and index are put on the site. You then use scroll buttons
to move back and forth, and of course they call javascript.

What I've read about spidering with javascript is confusing.
Can anyone help?



The reply-to email address is (e-mail address removed).
This is an address I ignore.
To reply via email, remove 2002 and change yahoo to
interaccess,

**
Thaddeus L. Olczyk, PhD

There is a difference between
*thinking* you know something,
and *knowing* you know something.

 

[John W. Kennedy]

Hmmm. What was your major? African American Studies?

Eat your Luger and burn in Hell, Nazi-boy. We have no time for your
used nightmares.

--
John W. Kennedy
"The bright critics assembled in this volume will doubtless show, in
their sophisticated and ingenious new ways, that, just as /Pooh/ is
suffused with humanism, our humanism itself, at this late date, has
become full of /Pooh./"
-- Frederick Crews. "Postmodern Pooh", Preface

 

[John W. Kennedy]

I'd like to write a program to spider certain websites
which contain javascript.
One example is a local newspaper providing tv listings.
The page is set up in the following way, it brings up
the listing for half of todays channels for the next
three hours in a grid certain buttons advance the
grid by executing javascript commands ( advance previous
scroll etc, ) Rather than sit there wait half a minute for the next
page to load I thought I would write a script that searches for
the shows and extracts the pages with the shows I want on it.

Another example almost anyone can understand, is the "Look inside
feature " of amazon, where three or four pages of the book plus
toc and index are put on the site. You then use scroll buttons
to move back and forth, and of course they call javascript.

What I've read about spidering with javascript is confusing.
Can anyone help?

In order to do that, you first have to have a Perl module that can read
JavaScript code and execute it. See
<URL:http://www.cpan.org/modules/by-module/JavaScript/JavaScript-0.52.readme>
for starters.

I suspect that solving the problem may, in the end, not be worth the effort.

 

[TLOlczyk]

I suspect that solving the problem may, in the end, not be worth the effort.

Thanks. I suspect won't be worth the effort the first time, but the
second third fourth...


The reply-to email address is (e-mail address removed).
This is an address I ignore.
To reply via email, remove 2002 and change yahoo to
interaccess,

**
Thaddeus L. Olczyk, PhD

There is a difference between
*thinking* you know something,
and *knowing* you know something.