Designing a cancellable function

[Leo Breebaart]

I have written a function foo() that iterates over and processes
a large number of files. The function should be available to the
user as library function, via a command-line interface, and
through a GUI.

So, I added a 'config' object as a parameter to foo() that can be
used by the caller to explicitly pass in user-defined settings.
Because the processing foo() does can take such a long time, the
next thing I did was add an 'update_function' callback parameter
that foo() will call regularly, so that the GUI can update a
progress bar, and the command-line version can print dots, etc.

I now would also like to add the possibility to allow the user to
*cancel* the execution of foo() during the processing, and I am
wondering what the best / most Pythonic way to design this is.

One obvious approach seems to me to turn the 'config' object into
something more dynamic, and have foo() regularly inspect it to
see if somebody in the main thread has set e.g. config.abort to
True.

Another approach would be to turn foo() into a proper (threaded)
class with distinct run() and abort() methods. But the caller
would still need to register the update callback somehow, and I
am wondering if this way the whole API for foo() won't become to
complex and overdesigned.

I was wondering if anybody has any insights or best practice
recommendations for me here. Do I keep the function interface? Do
I use a class? Any other solution I am overlooking?

Many thanks in advance for your advice.

 

[Michele]

Hi Leo,

what about one (or more) thread(s) which run the eat() method of
FilesEater object with inheriths from a superclass what's needed to
update the progress attribute (for example a _inc_progress() private
method)? (So you can have a UrlEater class and so on, them all knowing
how to update their progress)
Then you give FilesEater (or maybe a better name) a reference to a
config dict at init time.

You can have multiple FilesEater with different configurations and you
can handle the problem of stopping execution with a stopEating()
public method which sets an internal flag (a Lock object) and you
check it inside the object runloop with something like:

def eat(self):
for f in self.files:
do_some_stuff()
self._eat_for_real()
do_some_other_stuff()

def _eat_for_real(): # you should use the with statement inside here =)
self._eatLock.acquire()
do_the_processing
_write_the_result() # !
self._eatLock.release()

with this you can pause and restart the execution by calling
pause/restart methods that acquire and release the eatLock. It writes
the result before releasing it so you know it finishes the last chunk
when you suspend it.
If then you want to abord, just call the pause() on every *Eater
object and exit.
If you want to exit immediately just exit and design the code that
reads the output of the eat() processing in a way that it can
recognize broken chunks, delete them and go on (if they are written
on-disk obviously).

Hope this helps,
Michele

 

[Gabriel Genellina]

I have written a function foo() that iterates over and processes
a large number of files. The function should be available to the
user as library function, via a command-line interface, and
through a GUI.

So, I added a 'config' object as a parameter to foo() that can be
used by the caller to explicitly pass in user-defined settings.
Because the processing foo() does can take such a long time, the
next thing I did was add an 'update_function' callback parameter
that foo() will call regularly, so that the GUI can update a
progress bar, and the command-line version can print dots, etc.

I now would also like to add the possibility to allow the user to
*cancel* the execution of foo() during the processing, and I am
wondering what the best / most Pythonic way to design this is.

I can't say if this is the "best/more Pythonic way", but a simple way
would be to use the return value from your callback. Consider it an
"abort" function: if it returns True, cancel execution; as long as it
returns False, keep going.
(The somewhat "reversed" meaning is useful in case the user doesn't
provide a callback at all, or it's an empty one, or it contains just
a print ".", statement, all of these returning False; so, to actually
abort the process it must have an explicit "return True" statement).


--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

 

[Giovanni Bajo]

I can't say if this is the "best/more Pythonic way", but a simple way
would be to use the return value from your callback. Consider it an
"abort" function: if it returns True, cancel execution; as long as it
returns False, keep going.

It's even easier if the callback function simply raise an exception, which can
be caught from the outside:


class AbortOperationError(RuntimeError):
pass


def callback(N):
updateProgressBar(N)
processGUIEvents()
if exit_button_pressed:
raise AbortOperationError()


try:
longFunction(callback)
except AbortOperationError()
pass


def longFunction(callback):
for i in xrange(1000000000):
# do something
callback(i / 1000000000.0)