scheduler or infinite loop

[harryos]

hi
I am trying to write a program to read data from a site url.
The program must read the data from site every 5 minutes.

def get_data_from_site(pageurlstr):
h=urllib.urlopen(pageurlstr)
data=h.read()
process_data(data)

At first, I thought of using the sched module ,but then it doesn't
look right adding so many scheduler.enter() statements.The program is
supposed to execute the above function every
5 minutes until the application is shut down by the user.

I created an infinite loop
while True:
print time.asctime()
get_data_from_site('http://somesite.com/')
time.sleep(300)

Is there a better way to do this?Any suggestions ,pointers most
welcome
thanks
harry

 

[Frank Millman]

hi
I am trying to write a program to read data from a site url.
The program must read the data from site every 5 minutes.

def get_data_from_site(pageurlstr):
h=urllib.urlopen(pageurlstr)
data=h.read()
process_data(data)

At first, I thought of using the sched module ,but then it doesn't
look right adding so many scheduler.enter() statements.The program is
supposed to execute the above function every
5 minutes until the application is shut down by the user.

I created an infinite loop
while True:
print time.asctime()
get_data_from_site('http://somesite.com/')
time.sleep(300)

Is there a better way to do this?Any suggestions ,pointers most
welcome
thanks
harry

Here is a technique that allows the loop to run in the background, in its
own thread, leaving the main program to do other processing -

import threading

class DataGetter(threading.Thread):

def __init__(self):
threading.Thread.__init__(self)
self.event = threading.Event()

def run(self):
event = self.event # make local
while not event.is_set():
print time.asctime()
# either call the function from here,
# or put the body of the function here
get_data_from_site('http://somesite.com/')
event.wait(300)

def stop(self):
self.event.set()

In the startup of the main program, insert the following -

data_getter = DataGetter()
data_getter.start()

At the end of the program, terminate the loop like this -
data_getter.stop()

HTH

Frank Millman

 

[harryos]

thanks Frank

 

[Frank Millman]

harryos wrote

thanks Frank

A pleasure.

I left out a line that will usually be desirable.

At the end of the program, you should have -

data_getter.stop()
+ data_getter.join()

This forces the main program to wait until the thread has terminated before
continuing.

Frank

 

[John Nagle]

hi
I am trying to write a program to read data from a site url.
The program must read the data from site every 5 minutes.

def get_data_from_site(pageurlstr):
h=urllib.urlopen(pageurlstr)
data=h.read()
process_data(data)

A key point here is that you're not handling network
failures. The first time you have a brief network
outage, your program will fail.

Consider something like this:



def get_data_from_site(pageurlstr):
try :
h=urllib.urlopen(pageurlstr)
data=h.read()
except EnvironmentError as message :
print("Error reading %s from network at %s: %s" %
(pageurlstr, time.asctime(), str(message))
return(False)
process_data(data)
return(True)


lastpoll = 0 # time of last successful read
POLLINTERVAL = 300.0
RETRYINTERVAL = 30.0
while True:
status = get_data_from_site('http://somesite.com/')
if not status : # if fail
time.sleep(RETRYINTERVAL) # try again soon
print("Retrying...")
continue
now = time.time() # success
# Wait for the next poll period. Compensate for how
# long the read took.
waittime = max(POLLINTERVAL - (now - lastpoll), 1.0)
lastpoll = now
time.sleep(waittime)