A "scopeguard" for Python

[Alf P. Steinbach]

For C++ Petru Marginean once invented the "scope guard" technique (elaborated on
by Andrei Alexandrescu, they published an article about it in DDJ) where all you
need to do to ensure some desired cleanup at the end of a scope, even when the
scope is exited via an exception, is to declare a ScopeGuard w/desired action.

The C++ ScopeGuard was/is for those situations where you don't have proper
classes with automatic cleanup, which happily is seldom the case in good C++
code, but languages like Java and Python don't support automatic cleanup and so
the use case for something like ScopeGuard is ever present.

For use with a 'with' statement and possibly suitable 'lambda' arguments:


<code>
class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( is_callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" ) from
</code>


I guess the typical usage would be what I used it for, a case where the cleanup
action (namely, changing back to an original directory) apparently didn't fit
the standard library's support for 'with', like

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

Another use case might be where one otherwise would get into very deep nesting
of 'with' statements with every nested 'with' at the end, like a degenerate tree
that for all purposes is a list. Then the above, or some variant, can help to
/flatten/ the structure. To get rid of that silly & annoying nesting.


Cheers,

- Alf (just sharing, it's not seriously tested code)

 

[Mike Kent]

What's the compelling use case for this vs. a simple try/finally?

original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

 

[Alf P. Steinbach]

* Mike Kent:

What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else". "finally" is always
executed. which is incorrect for cleanup

by the way, that's one advantage:

a "with Cleanup" is difficult to get wrong, while a "try" is easy to get wrong,
as you did here


---

another general advantage is as for the 'with' statement generally

original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff

also, the "do other stuff" can be a lot of code

and also, with more than one action the try-else introduces a lot of nesting

finally:
os.chdir(original_dir)
# Do other cleanup

cheers & hth.,

- alf

 

[Robert Kern]

What's the compelling use case for this vs. a simple try/finally?

original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

A custom-written context manager looks nicer and can be more readable.

from contextlib import contextmanager
import os

@contextmanager
def pushd(path):
original_dir = os.getcwd()
os.chdir(path)
try:
yield
finally:
os.chdir(original_dir)


with pushd(somewhere):
...


I don't think a general purpose ScopeGuard context manager has any such benefits
over the try: finally:, though.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

* Mike Kent:

if you thought about it you would mean a simple "try/else". "finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you gave is definitely
equivalent to the try: finally: that Mike posted. The actions are always
executed in your example, not just when an exception isn't raised.

From your post, the scope guard technique is used "to ensure some desired
cleanup at the end of a scope, even when the scope is exited via an exception."
This is precisely what the try: finally: syntax is for. The with statement
allows you to encapsulate repetitive boilerplate into context managers, but a
general purpose context manager like your Cleanup class doesn't take advantage
of this.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Robert Kern:

A custom-written context manager looks nicer and can be more readable.

from contextlib import contextmanager
import os

@contextmanager
def pushd(path):
original_dir = os.getcwd()
os.chdir(path)
try:
yield
finally:
os.chdir(original_dir)


with pushd(somewhere):
...


I don't think a general purpose ScopeGuard context manager has any such
benefits over the try: finally:, though.

I don't think that's a matter of opinion, since one is correct while the other
is incorrect.


Cheers,

- ALf

 

[Alf P. Steinbach]

* Robert Kern:

Eh? Failed execution doesn't require cleanup? The example you gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when action A
succeeds.

With incorrect code cleanup for action A is performed when A fails.

The actions
are always executed in your example,

Sorry, that's incorrect.

[The actions are] not [executed] just when an exception isn't raised.

Sorry, that's incorrect.

From your post, the scope guard technique is used "to ensure some
desired cleanup at the end of a scope, even when the scope is exited via
an exception." This is precisely what the try: finally: syntax is for.

You'd have to nest it. That's ugly. And more importantly, now two people in this
thread (namely you and Mike) have demonstrated that they do not grok the try
functionality and manage to write incorrect code, even arguing that it's correct
when informed that it's not, so it's a pretty fragile construct, like goto.

The with statement allows you to encapsulate repetitive boilerplate into
context managers, but a general purpose context manager like your
Cleanup class doesn't take advantage of this.

I'm sorry but that's pretty meaningless. It's like: "A house allows you to
encapsulate a lot of stinking garbage, but your house doesn't take advantage of
that, it's disgustingly clean". Hello.


Cheers & hth.,

- Alf

 

[Robert Kern]

* Robert Kern:

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an exception."))

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

Sorry, that's incorrect.

Looks like it to me.

You'd have to nest it. That's ugly. And more importantly, now two people
in this thread (namely you and Mike) have demonstrated that they do not
grok the try functionality and manage to write incorrect code, even
arguing that it's correct when informed that it's not, so it's a pretty
fragile construct, like goto.
Uh-huh.


I'm sorry but that's pretty meaningless. It's like: "A house allows you
to encapsulate a lot of stinking garbage, but your house doesn't take
advantage of that, it's disgustingly clean". Hello.

No, I'm saying that your Cleanup class is about as ugly as the try: finally:. It
just shifts the ugliness around. There is a way to use the with statement to
make things look better and more readable in certain situations, namely where
there is some boilerplate that you would otherwise repeat in many places using
try: finally:. You can encapsulate that repetitive code into a class or a
@contextmanager generator and just call the contextmanager. A generic context
manager where you register callables doesn't replace any boilerplate. You still
repeat all of the cleanup code everywhere. What's more, because you have to
shove everything into a callable, you have significantly less flexibility than
the try: finally:.

I will admit that you can put the cleanup code closer to the code that needs to
get cleaned up, but you pay a price for that.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern:

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an
exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.

at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the example I gave.

$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError


Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing, and

2) not even showing anything about your earlier statements, which were
just incorrect.

You're instead showing that my code works as it should for the case that you're
testing, which is a bit unnecessary since I knew that, but thanks anyway.

I'm not sure what that shows, except that you haven't grokked this yet.

Uh-huh.

Yeah. Consider that you're now for the third time failing to grasp the concept
of cleanup for a successful operation.

No, I'm saying that your Cleanup class is about as ugly as the try:
finally:. It just shifts the ugliness around. There is a way to use the
with statement to make things look better and more readable in certain
situations, namely where there is some boilerplate that you would
otherwise repeat in many places using try: finally:. You can encapsulate
that repetitive code into a class or a @contextmanager generator and
just call the contextmanager. A generic context manager where you
register callables doesn't replace any boilerplate. You still repeat all
of the cleanup code everywhere. What's more, because you have to shove
everything into a callable, you have significantly less flexibility than
the try: finally:.

Sorry, but that's meaningless again. You're repeating that my house has no
garbage in it. And you complain that it would be work to add garbage to it. Why
do you want that garbage? I think it's nice without it!

I will admit that you can put the cleanup code closer to the code that
needs to get cleaned up, but you pay a price for that.

Yes, that's an additional point, and important. I forgot to mention it. Thanks!


Cheers & hth.,

- Alf

 

[Jerry Hill]

I'm not sure what that shows, except that you haven't grokked this yet.

Maybe you could give us an example of how your code should be used,
and how it differs from the other examples people have given? And
maybe a quick example of why you would not want to clean up after a
failed operation?

I've been trying to follow along, and I don't get it either. I guess
that makes me at least the third person that doesn't understand what
you're trying to get across.

 

[Robert Kern]

* Robert Kern:

* Robert Kern:
On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
* Mike Kent:
What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else".
"finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.

at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the
example I gave.

$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError


Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing, and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the scope is
exited via an exception" that you talked about in your original post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first chdir() and
before the second "blah blah". Even if an exception is raised in the second
"blah blah", then the cleanup function will still run. This would be equivalent
to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

and not a try: else:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
else:
chdir( original_dir )

Now, I assumed that the behavior with respect to exceptions occurring in the
first "blah blah" weren't what you were talking about because until the chdir(),
there is nothing to clean up.

There is no way that the example you gave translates to a try: else: as you
claimed in your response to Mike Kent.

2) not even showing anything about your earlier statements, which were
just incorrect.

You're instead showing that my code works as it should for the case that
you're testing, which is a bit unnecessary since I knew that, but thanks
anyway.

It's the case you seem to be talking about in your original post. You seem to
have changed your mind about what you want to talk about. That's fine. We don't
have to stick with the original topic, but I do ask you to acknowledge that you
originally were talking about a feature that "ensure some desired cleanup at
the end of a scope, even when the scope is exited via an exception."

Do you acknowledge this?

I'm not sure what that shows, except that you haven't grokked this yet.



Yeah. Consider that you're now for the third time failing to grasp the
concept of cleanup for a successful operation.

Oh, I do. But if I didn't want it to run on an exception, I'd just write the
code without any try:s or with:s at all.

# blah blah #1
chdir( somewhere )
# blah blah #2
chdir( original_dir )

Sorry, but that's meaningless again. You're repeating that my house has
no garbage in it.

No, I'm repeatedly saying that I think your solution stinks. I think it's ugly.
I think it's restrictive. I think it does not improve on the available solutions.

And you complain that it would be work to add garbage
to it. Why do you want that garbage? I think it's nice without it!

And you are entitled to that opinion. I am giving you mine.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern:

On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
* Mike Kent:
What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else".
"finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an
exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.

at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the
example I gave.

$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

The actions are always executed in your example,

Sorry, that's incorrect.

Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing, and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the
scope is exited via an exception" that you talked about in your original
post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first chdir()
and before the second "blah blah". Even if an exception is raised in the
second "blah blah", then the cleanup function will still run. This would
be equivalent to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

Yes, this is equivalent code.

The try-finally that you earlier claimed was equivalent, was not.

and not a try: else:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
else:
chdir( original_dir )

This example is however meaningless except as misdirection. There are infinitely
many constructs that include try-finally and try-else, that the with-Cleanup
code is not equivalent to. It's dumb to show one such.

Exactly what are you trying to prove here?

Your earlier claims are still incorrect.

Now, I assumed that the behavior with respect to exceptions occurring in
the first "blah blah" weren't what you were talking about because until
the chdir(), there is nothing to clean up.

There is no way that the example you gave translates to a try: else: as
you claimed in your response to Mike Kent.

Of course there is.

Note that Mike wrapped the action A within the 'try':


<code author="Mike" correct="False">
original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup
</code>


The 'finally' he used, shown above, yields incorrect behavior.

Namely cleanup always, while 'else', in that code, can yield correct behavior
/provided/ that it's coded correctly:


<code author="Alf" correct="ProbablyTrue" disclaimer="off the cuff">
original_dir = os.getcwd()
try:
os.chdir(somewhere)
except Whatever:
# whatever, e.g. logging
raise
else:
try:
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

It's the case you seem to be talking about in your original post.

What's this "seems"? Are you unable to read that very short post?

You
seem to have changed your mind about what you want to talk about. That's
fine.

And what's this claim about me changing any topic?

We don't have to stick with the original topic

Why not stick with the original topic?

, but I do ask you
to acknowledge that you originally were talking about a feature that
"ensure some desired cleanup at the end of a scope, even when the
scope is exited via an exception."

Yes, that's what it does.

Which is I why I wrote that.

This should not be hard to grok.

Do you acknowledge this?

This seems like pure noise, to cover up that you were sputing a lot of incorrect
statements earlier.

Oh, I do. But if I didn't want it to run on an exception, I'd just write
the code without any try:s or with:s at all.

# blah blah #1
chdir( somewhere )
# blah blah #2
chdir( original_dir )

Yes, but what's that got to do with anything?

No, I'm repeatedly saying that I think your solution stinks. I think
it's ugly. I think it's restrictive. I think it does not improve on the
available solutions.

First you'd have to understand it, simple as it is.

And you are entitled to that opinion. I am giving you mine.

Well, I'm sorry, but while you are entitled to an opinion it's not an opinion
that carries any weight: first you need to get your facts and claims straight.

So far only this latest posting of yours has been free of directly incorrect
statements.

But you still have a lot of statements that just show total incomprehension,
like your example of achieving no cleanup in the case of an exception.


Cheers & hth.,

- Alf

 

[Robert Kern]

* Robert Kern:

* Robert Kern:
On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
* Mike Kent:
What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else".
"finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an
exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.


at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the
example I gave.


$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

The actions are always executed in your example,

Sorry, that's incorrect.

Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing, and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the
scope is exited via an exception" that you talked about in your
original post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first
chdir() and before the second "blah blah". Even if an exception is
raised in the second "blah blah", then the cleanup function will still
run. This would be equivalent to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

Yes, this is equivalent code.

The try-finally that you earlier claimed was equivalent, was not.

Okay, but just because of the position of the chdir(), right?

This example is however meaningless except as misdirection. There are
infinitely many constructs that include try-finally and try-else, that
the with-Cleanup code is not equivalent to. It's dumb to show one such.

Exactly what are you trying to prove here?

I'm just showing you what I thought you meant when you told Mike that he should
have used a try/else instead of try/finally.

Your earlier claims are still incorrect.


Of course there is.

Note that Mike wrapped the action A within the 'try':


<code author="Mike" correct="False">
original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup
</code>


The 'finally' he used, shown above, yields incorrect behavior.

Namely cleanup always, while 'else', in that code, can yield correct
behavior /provided/ that it's coded correctly:


<code author="Alf" correct="ProbablyTrue" disclaimer="off the cuff">
original_dir = os.getcwd()
try:
os.chdir(somewhere)
except Whatever:
# whatever, e.g. logging
raise
else:
try:
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup
</code>

Ah, okay. Now we're getting somewhere. Now, please note that you did not have
any except: handling in your original example. So Mike made a try: finally:
example to attempt to match the semantics of your code. When you tell him that
he should 'mean a simple "try/else". "finally" is always executed. which is
incorrect for cleanup', can you understand why we might think that you were
saying that try: finally: was wrong and that you were proposing that your code
was equivalent to some try: except: else: suite?

What's this "seems"? Are you unable to read that very short post?

I say "seems" because my understandings of what you meant in your original post
and your response to Mike disagreed with one another. Now I see that your later
posts were talking about minor discrepancy about which errors you wanted caught
by the finally: and which you didn't. I was confused because it seemed that you
were saying that try: finally: was completely wrong and that "try/else" was
right. It confused me and at least one other person.

, but I do ask you to acknowledge that you originally were talking
about a feature that "ensure some desired cleanup at the end of a
scope, even when the scope is exited via an exception."

Yes, that's what it does.

Which is I why I wrote that.

This should not be hard to grok.

Do you acknowledge this?

This seems like pure noise, to cover up that you were sputing a lot of
incorrect statements earlier.

No, I'm just trying to clarify what you are trying to say. The above statement
did not appear to accord with your later statement: 'if you thought about it you
would mean a simple "try/else". "finally" is always executed. which is incorrect
for cleanup.' It turns out that what you really meant was that it would be
incorrect for cleanup to be executed when an error occurred in the chdir() itself.

Now, I happen to disagree with that. There are a couple of ways to do this kind
of cleanup depending on the situation. Basically, you have several different
code blocks:

# 1. Record original state.
# 2. Modify state.
# 3. Do stuff requiring the modified state.
# 4. Revert to the original state.

Depending on where errors are expected to occur, and how the state needs to get
modified and restored, there are different ways of arranging these blocks. The
one Mike showed:

# 1. Record original state.
try:
# 2. Modify state.
# 3. Do stuff requiring the modified state.
finally:
# 4. Revert to the original state.

And the one you prefer:

# 1. Record original state.
# 2. Modify state.
try:
# 3. Do stuff requiring the modified state.
finally:
# 4. Revert to the original state.

These differ in what happens when an error occurs in block #2, the modification
of the state. In Mike's, the cleanup code runs; in yours, it doesn't. For
chdir(), it really doesn't matter. Reverting to the original state is harmless
whether the original chdir() succeeds or fails, and chdir() is essentially
atomic so if it raises an exception, the state did not change and nothing needs
to be cleaned up.

However, not all block #2s are atomic. Some are going to fail partway through
and need to be cleaned up even though they raised an exception. Fortunately,
cleanup can frequently be written to not care whether the whole thing finished
or not.

Both formulations can be correct (and both work perfectly fine with the chdir()
example being used). Sometimes one is better than the other, and sometimes not.
You can achieve both ways with either your Cleanup class or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over try: finally:
and has the significant ugliness of forcing cleanup code into callables. This
significantly limits what you can do in your cleanup code.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern:

On 2010-03-03 13:32 PM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
* Mike Kent:
What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else".
"finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you
gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an
exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.


at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the
example I gave.


$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

The actions are always executed in your example,

Sorry, that's incorrect.

Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're testing,
and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the
scope is exited via an exception" that you talked about in your
original post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first
chdir() and before the second "blah blah". Even if an exception is
raised in the second "blah blah", then the cleanup function will still
run. This would be equivalent to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

Yes, this is equivalent code.

The try-finally that you earlier claimed was equivalent, was not.

Okay, but just because of the position of the chdir(), right?

Yes, since it yields different results.

I'm just showing you what I thought you meant when you told Mike that he
should have used a try/else instead of try/finally.


Ah, okay. Now we're getting somewhere. Now, please note that you did not
have any except: handling in your original example. So Mike made a try:
finally: example to attempt to match the semantics of your code. When
you tell him that he should 'mean a simple "try/else". "finally" is
always executed. which is incorrect for cleanup', can you understand why
we might think that you were saying that try: finally: was wrong and
that you were proposing that your code was equivalent to some try:
except: else: suite?

No, not really. His code didn't match the semantics. Changing 'finally' to
'else' could make it equivalent.

I say "seems" because my understandings of what you meant in your
original post and your response to Mike disagreed with one another. Now
I see that your later posts were talking about minor discrepancy about
which errors you wanted caught by the finally: and which you didn't.

It's absolutely not a minor discrepancy whether some code is executed or not. It
can have arbitrarily large effect. And from my point of view the discussion of
that snippet has not been about what errors I "want" caught by the 'finally';
it's been about whether two snippets of code yield the same effect or not:
Mike's code was incorrect not because it did something else, but because as code
that did something else it was not an equivalent to the code that I posted.

I
was confused because it seemed that you were saying that try: finally:
was completely wrong and that "try/else" was right. It confused me and
at least one other person.

, but I do ask you to acknowledge that you originally were talking
about a feature that "ensure some desired cleanup at the end of a
scope, even when the scope is exited via an exception."

Yes, that's what it does.

Which is I why I wrote that.

This should not be hard to grok.

Do you acknowledge this?

This seems like pure noise, to cover up that you were sputing a lot of
incorrect statements earlier.

No, I'm just trying to clarify what you are trying to say. The above
statement did not appear to accord with your later statement: 'if you
thought about it you would mean a simple "try/else". "finally" is always
executed. which is incorrect for cleanup.' It turns out that what you
really meant was that it would be incorrect for cleanup to be executed
when an error occurred in the chdir() itself.

Now, I happen to disagree with that.

Well, I was pretty unclear, almost hint-like, sorry about that, mea culpa, but
you have it slightly wrong. You wrote then "The example you gave is definitely
equivalent to the try: finally: that Mike posted." And it isn't.

There are a couple of ways to do
this kind of cleanup depending on the situation. Basically, you have
several different code blocks:

# 1. Record original state.
# 2. Modify state.
# 3. Do stuff requiring the modified state.
# 4. Revert to the original state.

Depending on where errors are expected to occur, and how the state needs
to get modified and restored, there are different ways of arranging
these blocks. The one Mike showed:

# 1. Record original state.
try:
# 2. Modify state.
# 3. Do stuff requiring the modified state.
finally:
# 4. Revert to the original state.

And the one you prefer:

# 1. Record original state.
# 2. Modify state.
try:
# 3. Do stuff requiring the modified state.
finally:
# 4. Revert to the original state.

These differ in what happens when an error occurs in block #2, the
modification of the state. In Mike's, the cleanup code runs; in yours,
it doesn't. For chdir(), it really doesn't matter. Reverting to the
original state is harmless whether the original chdir() succeeds or
fails, and chdir() is essentially atomic so if it raises an exception,
the state did not change and nothing needs to be cleaned up.

However, not all block #2s are atomic. Some are going to fail partway
through and need to be cleaned up even though they raised an exception.
Fortunately, cleanup can frequently be written to not care whether the
whole thing finished or not.

Yeah, and there are some systematic ways to handle these things. You might look
up Dave Abraham's levels of exception safety. Mostly his approach boils down to
making operations effectively atomic so as to reduce the complexity: ideally, if
an operation raises an exception, then it has undone any side effects.

Of course it can't undo the launching of an ICBM, for example...

But ideally, if it could, then it should.

If you call the possibly failing operation "A", then that systematic approach
goes like this: if A fails, then it has cleaned up its own mess, but if A
succeeds, then it's the responsibility of the calling code to clean up if the
higher level (multiple statements) operation that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed for, and what
the corresponding Python Cleanup class is designed for.

Both formulations can be correct (and both work perfectly fine with the
chdir() example being used). Sometimes one is better than the other, and
sometimes not. You can achieve both ways with either your Cleanup class
or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over try:
finally: and has the significant ugliness of forcing cleanup code into
callables. This significantly limits what you can do in your cleanup code.

Uhm, not really. As I see it.

But for any given task one should use the most practical tool, and I'm certainly
not claiming that Cleanup will always be that: it's just another weapon to
employ in the correctness war -- although I think it's a powerful one.


Cheers,

- Alf

 

[Jean-Michel Pichavant]

You'd have to nest it. That's ugly. And more importantly, now two
people in this thread (namely you and Mike) have demonstrated that
they do not grok the try functionality and manage to write incorrect
code, even arguing that it's correct when informed that it's not, so
it's a pretty fragile construct, like goto.

You want to execute some cleanup when things go wrong, use try except.
You want to do it when things go right, use try else. You want to
cleanup no matter what happen, use try finally.

There is no need of any Cleanup class, except for some technical
alternative concern.

JM

 

[Alf P. Steinbach]

* Jean-Michel Pichavant:

You want to execute some cleanup when things go wrong, use try except.
You want to do it when things go right, use try else. You want to
cleanup no matter what happen, use try finally.

There is no need of any Cleanup class, except for some technical
alternative concern.

Have you considered that your argument applies to the "with" construct?

You have probably not realized that.

But let me force it on you: when would you use "with"?

Check if that case is covered by your argument above.

Now that you've been told about the "with" angle, don't you think it's a kind of
weakness in your argument that it calls for removing "with" from the language?

I recommend that you think about why your argument is invalid.

Or, as I like to say, why your argument is completely bogus.


Cheers & hth.,

- Alf

 

[Robert Kern]

* Robert Kern:

* Robert Kern:
On 2010-03-03 13:32 PM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 11:18 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 09:56 AM, Alf P. Steinbach wrote:
* Mike Kent:
What's the compelling use case for this vs. a simple try/finally?

if you thought about it you would mean a simple "try/else".
"finally" is
always executed. which is incorrect for cleanup

Eh? Failed execution doesn't require cleanup? The example you
gave is
definitely equivalent to the try: finally: that Mike posted.

Sorry, that's incorrect: it's not.

With correct code (mine) cleanup for action A is only performed when
action A succeeds.

With incorrect code cleanup for action A is performed when A fails.

Oh?

$ cat cleanup.py

class Cleanup:
def __init__( self ):
self._actions = []

def call( self, action ):
assert( callable( action ) )
self._actions.append( action )

def __enter__( self ):
return self

def __exit__( self, x_type, x_value, x_traceback ):
while( len( self._actions ) != 0 ):
try:
self._actions.pop()()
except BaseException as x:
raise AssertionError( "Cleanup: exception during cleanup" )

def print_(x):
print x

with Cleanup() as at_cleanup:
at_cleanup.call(lambda: print_("Cleanup executed without an
exception."))

with Cleanup() as at_cleanup:

*Here* is where you should

1) Perform the action for which cleanup is needed.

2) Let it fail by raising an exception.


at_cleanup.call(lambda: print_("Cleanup execute with an exception."))
raise RuntimeError()

With an exception raised here cleanup should of course be performed.

And just in case you didn't notice: the above is not a test of the
example I gave.


$ python cleanup.py
Cleanup executed without an exception.
Cleanup execute with an exception.
Traceback (most recent call last):
File "cleanup.py", line 28, in <module>
raise RuntimeError()
RuntimeError

The actions are always executed in your example,

Sorry, that's incorrect.

Looks like it to me.

I'm sorry, but you're

1) not testing my example which you're claiming that you're
testing, and

Then I would appreciate your writing a complete, runnable example that
demonstrates the feature you are claiming. Because it's apparently not
"ensur[ing] some desired cleanup at the end of a scope, even when the
scope is exited via an exception" that you talked about in your
original post.

Your sketch of an example looks like mine:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

The cleanup function gets registered immediately after the first
chdir() and before the second "blah blah". Even if an exception is
raised in the second "blah blah", then the cleanup function will still
run. This would be equivalent to a try: finally:

# blah blah #1
chdir( somewhere )
try:
# blah blah #2
finally:
chdir( original_dir )

Yes, this is equivalent code.

The try-finally that you earlier claimed was equivalent, was not.

Okay, but just because of the position of the chdir(), right?

Yes, since it yields different results.

I'm just showing you what I thought you meant when you told Mike that
he should have used a try/else instead of try/finally.


Ah, okay. Now we're getting somewhere. Now, please note that you did
not have any except: handling in your original example. So Mike made a
try: finally: example to attempt to match the semantics of your code.
When you tell him that he should 'mean a simple "try/else". "finally"
is always executed. which is incorrect for cleanup', can you
understand why we might think that you were saying that try: finally:
was wrong and that you were proposing that your code was equivalent to
some try: except: else: suite?

No, not really. His code didn't match the semantics. Changing 'finally'
to 'else' could make it equivalent.

Okay, please show me what you mean by "changing 'finally' to 'else'." I think
you are being hinty again. It's not helpful. The most straightforward
interpretation of those words means that you literally just want to remove the
word 'finally' and replace it with 'else' in Mike's example. Obviously you don't
mean that because it is a syntax error. try: else: is not a construct in Python.
There is a try: except: else:, but there is no point to doing that if you don't
have anything in the except: clause. Neither Mike's example nor your original
one have any except: clause. Why do you think that we would interpret those
words to mean that you wanted the example you give just above?

I say "seems" because my understandings of what you meant in your
original post and your response to Mike disagreed with one another.
Now I see that your later posts were talking about minor discrepancy
about which errors you wanted caught by the finally: and which you
didn't.

It's absolutely not a minor discrepancy whether some code is executed or
not. It can have arbitrarily large effect. And from my point of view the
discussion of that snippet has not been about what errors I "want"
caught by the 'finally'; it's been about whether two snippets of code
yield the same effect or not: Mike's code was incorrect not because it
did something else, but because as code that did something else it was
not an equivalent to the code that I posted.

I was confused because it seemed that you were saying that try:
finally: was completely wrong and that "try/else" was right. It
confused me and at least one other person.

, but I do ask you to acknowledge that you originally were talking
about a feature that "ensure some desired cleanup at the end of a
scope, even when the scope is exited via an exception."

Yes, that's what it does.

Which is I why I wrote that.

This should not be hard to grok.


Do you acknowledge this?

This seems like pure noise, to cover up that you were sputing a lot of
incorrect statements earlier.

No, I'm just trying to clarify what you are trying to say. The above
statement did not appear to accord with your later statement: 'if you
thought about it you would mean a simple "try/else". "finally" is
always executed. which is incorrect for cleanup.' It turns out that
what you really meant was that it would be incorrect for cleanup to be
executed when an error occurred in the chdir() itself.

Now, I happen to disagree with that.

Well, I was pretty unclear, almost hint-like, sorry about that, mea
culpa, but you have it slightly wrong. You wrote then "The example you
gave is definitely equivalent to the try: finally: that Mike posted."
And it isn't.

I agree that it does behave differently with respect to when an exception is
raised in chdir(). I was wrong on that point. I thought you were claiming that
it behaved differently when there was an exception in the "# Do other stuff"
block because you were being (and are still being) unclear.

Yeah, and there are some systematic ways to handle these things. You
might look up Dave Abraham's levels of exception safety. Mostly his
approach boils down to making operations effectively atomic so as to
reduce the complexity: ideally, if an operation raises an exception,
then it has undone any side effects.

Of course it can't undo the launching of an ICBM, for example...

But ideally, if it could, then it should.

I agree. Atomic operations like chdir() help a lot. But this is Python, and
exceptions can happen in many different places. If you're not just calling an
extension module function that makes a known-atomic system call, you run the
risk of not having an atomic operation.

If you call the possibly failing operation "A", then that systematic
approach goes like this: if A fails, then it has cleaned up its own
mess, but if A succeeds, then it's the responsibility of the calling
code to clean up if the higher level (multiple statements) operation
that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed for,
and what the corresponding Python Cleanup class is designed for.

And try: finally:, for that matter.

Uhm, not really. As I see it.

Well, not being able to affect the namespace is a significant limitation.
Sometimes you need to delete objects from the namespace in order to ensure that
their refcounts go to zero and their cleanup code gets executed. Tracebacks will
keep the namespace alive and all objects in it.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

* Jean-Michel Pichavant:

Have you considered that your argument applies to the "with" construct?

You have probably not realized that.

But let me force it on you: when would you use "with"?

When there is a specific context manager that removes the need for boilerplate.

Check if that case is covered by your argument above.

Now that you've been told about the "with" angle, don't you think it's a
kind of weakness in your argument that it calls for removing "with" from
the language?

No, it only argues that "with Cleanup():" is supernumerary.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Jean-Michel Pichavant]

* Jean-Michel Pichavant:

Have you considered that your argument applies to the "with" construct?

You have probably not realized that.

But let me force it on you: when would you use "with"?

Check if that case is covered by your argument above.

Now that you've been told about the "with" angle, don't you think it's
a kind of weakness in your argument that it calls for removing "with"
from the language?

I recommend that you think about why your argument is invalid.

Or, as I like to say, why your argument is completely bogus.


Cheers & hth.,

- Alf

I am using python 2.5, so I know nothing about the with statement, and
it may possible my arguments apply to it, you could remove it from the
language, it wouldn't bother me at all.
I just don't see in what you've written (adding a class, with some
__entry__, __exit__ protocol, using a with statement) what cannot be
achieved with a try statement in its simpliest form.

Try except may be lame and noobish, but it works, is easy to read and
understood at first glance.
It looks like to me that 'with' statements are like decorators:
overrated. Sometimes people could write simple readable code, but yet
they're tempted by the geek side of programming: using complex
constructs when there's no need to. I myself cannot resist sometimes ;-)

JM

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern: [snip]

can you
understand why we might think that you were saying that try: finally:
was wrong and that you were proposing that your code was equivalent to
some try: except: else: suite?

No, not really. His code didn't match the semantics. Changing 'finally'
to 'else' could make it equivalent.

Okay, please show me what you mean by "changing 'finally' to 'else'." I
think you are being hinty again. It's not helpful.

[snip middle of this paragraph]

Why do you think that we would interpret those words
to mean that you wanted the example you give just above?

There's an apparent discrepancy between your call for an example and your
subsequent (in the same paragraph) reference to the example given.

But as to why I assumed that that example, or a similar correct one, would be
implied, it's the only meaningful interpretation.

Adopting a meaningless interpretation when a meaningful exists is generally just
adversarial, but in this case I was, as you pointed out, extremely unclear, and
I'm sorry: I should have given such example up front. Will try to do so.


[snip]

I agree. Atomic operations like chdir() help a lot. But this is Python,
and exceptions can happen in many different places. If you're not just
calling an extension module function that makes a known-atomic system
call, you run the risk of not having an atomic operation.


And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a common
fallacy in discussions about programming, to assume that since the same can be
expressed using lower level constructs, those are all that are required.

If adopted as true it ultimately means the removal of all control structures
above the level of "if" and "goto" (except Python doesn't have "goto").

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the namespace in
order to ensure that their refcounts go to zero and their cleanup code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's required):
assigning None is sufficient for that[1].

However, note that the current language doesn't guarantee such cleanup, at least
as far as I know.

So while it's good practice to support it, to do everything to let it happen,
it's presumably bad practice to rely on it happening.

Tracebacks will keep the namespace alive and all objects
in it.

Thanks!, I hadn't thought of connecting that to general cleanup actions.

It limits the use of general "with" in the same way.


Cheers,

- Alf

Notes:
[1] An 'except' clause deletes variables, but since it has no knowledge of the
code it's placed in the only alternatives would be a presumably costly check of
prior existence, or letting it pollute the namespace.