Pickle Problem

[MRAB]

Ok, I managed to import the correct error class (was in a non expected
place)

Just as I thought I was finished, I encountered a final problem. I am
running a while loop which is constantly adding search results to a
file, like this (print >> f, res.total_results_available). I have put
an exception in that if it times out, it simply tries again until its
finished. The problem is that the loop re-write all new results
continuing on from the previous searches. I would like to somehow
delete all in that file and start again.

I thought this could simply be achieved by putting f.close() in the
exception and then it re-writes it however I am getting this error:

Traceback (most recent call last):
File "/home/csunix/scs5fjnh/FYProj/Python/pYsearch-3.1/test9.py",
line 17, in <module>
print >> f, res.total_results_available
ValueError: I/O operation on closed file

Is there another way rather than closing the file? Is it possible to
delete all within the file?

You could do:

f.seek(0)
f.truncate()

I hope you don't just discard all the results you've got so far and then
start from the beginning again.

 

[Lie Ryan]

Is there another way rather than closing the file? Is it possible to
delete all within the file?

Thanks

Delete the old file then opening (and creating) it again is the easiest
way? If you need the data from the old file, you can rename the old file
and reopen (and create) a new file in place of it.

 

[Fab86]

You could do:

     f.seek(0)
     f.truncate()

I hope you don't just discard all the results you've got so far and then
start from the beginning again.

MRAB, yes I do, thats the only way I can I get it to work. I wouldnt
know how to make the program read the expection and then carry of
from the last search term.

Is that even possible?

 

[MRAB]

MRAB, yes I do, thats the only way I can I get it to work. I wouldnt
know how to make the program read the expection and then carry of
from the last search term.

Is that even possible?

Your code looks something like this:

srch = WebSearch(app_id=YahooKey)
for lang in langs:
srch.query = "avoir site:.%s" % lang
res = srch.parse_results()
print >> f, res.total_results_available

The exception is raise by, I think, srch.parse_results(), so you could
do something like this:

srch = WebSearch(app_id=YahooKey)
for lang in langs:
srch.query = "avoir site:.%s" % lang
# Use a loop in case we need to try again.
while True:
try:
res = srch.parse_results()
except SearchError:
# Failed. Wait a while before trying again.
time.sleep(SLEEP_IN_SECONDS)
else:
# No exception, so we have the results.
break
print >> f, res.total_results_available

I'm assuming that you can re-use the WebSearch instance for repeated
queries even if it raised the SearchError exception.

If not, just put another srch = WebSearch(app_id=YahooKey) line after
the sleep to create a new WebSearch instance.