Executing Javascript, then reading value

[Melih Onvural]

I need to execute some javascript and then read the value as part of a
program that I am writing. I am currently doing something like this:

import htmllib, urllib, formatter

class myparser(htmllib.HTMLParser):
insave = 0
def start_div(self, attrs):
for i in attrs:
if i[0] == "id" and i[1] == "pr":
self.save_bgn()
self.insave = 1

def end_div(self):
if self.insave == 1:
print self.save_end()
self.insave = 0

parser = myparser(formatter.NullFormatter())

#def getPageRank(self, url):
try:
learn_url = "http://127.0.0.1/research/getPageRank.html?q=http://
www.yahoo.com&"
pr_url = urllib.urlopen(learn_url)
parser.feed(pr_url.read())
except IOError, e:
print e

but the result is the javascript function and not the calculated
value. Is there anyway to get the javascript to execute first, and
then return to me the value? thanks in advance,

Melih Onvural

 

[Melih Onvural]

Thanks, let me check out this route, and then I'll post the results.

Melih Onvural

 

[Diez B. Roggisch]

I need to execute some javascript and then read the value as part of a
program that I am writing. I am currently doing something like this:

import htmllib, urllib, formatter

class myparser(htmllib.HTMLParser):
insave = 0
def start_div(self, attrs):
for i in attrs:
if i[0] == "id" and i[1] == "pr":
self.save_bgn()
self.insave = 1

def end_div(self):
if self.insave == 1:
print self.save_end()
self.insave = 0

parser = myparser(formatter.NullFormatter())

#def getPageRank(self, url):
try:
learn_url = "http://127.0.0.1/research/getPageRank.html?q=http://
www.yahoo.com&"
pr_url = urllib.urlopen(learn_url)
parser.feed(pr_url.read())
except IOError, e:
print e

but the result is the javascript function and not the calculated
value. Is there anyway to get the javascript to execute first, and
then return to me the value? thanks in advance,

Do it in a browser. There are ways to automate one, for example the
webbrowser module, and others.

Then rework your script to work with AJAX.

Diez

 

[bearophileHUGS]

Jean-Paul Calderone:

You might look into the
stand-alone Spidermonkey runtime. However, it lacks the DOM APIs, so
it may not be able to run the JavaScript you are interested in running.
There are a couple other JavaScript runtimes available, at least.

This may be okay too:
http://www.digitalmars.com/dscript/

Bye,
bearophile

 

[John Nagle]

Thanks, let me check out this route, and then I'll post the results.

Melih Onvural

This is getting to be a common problem. One used to be able to
look at web pages from a program by reading the HTML. Now you need to
load the page into a browser-like environment, run at least the
OnLoad JavaScript, and then start looking at the document object module.
This requires a browser emulator, a browser without a renderer.
Useful for spam filters and such.

It's not clear if the original poster needs that much capability,
though.

John Nagle

 

[Melih Onvural]

In fact what you're describing is exactly what I needed. I ended up
finding a way to execute the javascript using Rhino and then capturing
the
result. Not exactly what I wanted to do, but once I found it out, it
works.

Melih Onvural

 

[skip]

Melih> In fact what you're describing is exactly what I needed. I ended
Melih> up finding a way to execute the javascript using Rhino and then
Melih> capturing the result. Not exactly what I wanted to do, but once I
Melih> found it out, it works.

There is an embeddable C implementation as well: SpiderMonkey. It has both
a core engine and a standalone app. There also appears to a an unmaintained
Python interface: python-spidermonkey. Relevant URLs:

http://www.mozilla.org/js/spidermonkey/
http://cheeseshop.python.org/pypi/python-spidermonkey/

Skip