A "scopeguard" for Python

[Robert Kern]

I am using python 2.5, so I know nothing about the with statement,

You can try it out using "from __future__ import with_statement".

and
it may possible my arguments apply to it, you could remove it from the
language, it wouldn't bother me at all.
I just don't see in what you've written (adding a class, with some
__entry__, __exit__ protocol, using a with statement) what cannot be
achieved with a try statement in its simpliest form.

Try except may be lame and noobish, but it works, is easy to read and
understood at first glance.
It looks like to me that 'with' statements are like decorators:
overrated. Sometimes people could write simple readable code, but yet
they're tempted by the geek side of programming: using complex
constructs when there's no need to. I myself cannot resist sometimes ;-)

PEP 343 is a good introduction to the real uses of the with: statement.

http://www.python.org/dev/peps/pep-0343/

Basically, it allows you to package up your initialization and cleanup code into
objects, stick them in your library, unit test them thoroughly, etc. so you
don't have to repeat them everywhere and possibly get them wrong. It's DRY in
action.

Where Alf's Cleanup class goes wrong, in my opinion, is that it does not package
up any code to avoid repetition. You still repeat the same cleanup code
everywhere you use it, so it is no better than try: finally:. It is not a real
use case of the with: statement.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Robert Kern:

When there is a specific context manager that removes the need for
boilerplate.

That's "cleanup no matter what happen".

No, it only argues that "with Cleanup():" is supernumerary.

I don't know what "supernumerary" means, but to the degree that the argument
says anything about a construct that is not 'finally', it says the same about
general "with".

So whatever you mean by supernumerary, you're saying that the argument implies
that "with" is supernumerary.

This is starting to look like some earlier discussions in this group, where even
basic logic is denied.


Cheers,

- Alf

 

[Robert Kern]

* Robert Kern:

* Robert Kern: [snip]
can you
understand why we might think that you were saying that try: finally:
was wrong and that you were proposing that your code was equivalent to
some try: except: else: suite?

No, not really. His code didn't match the semantics. Changing 'finally'
to 'else' could make it equivalent.

Okay, please show me what you mean by "changing 'finally' to 'else'."
I think you are being hinty again. It's not helpful.

[snip middle of this paragraph]

Why do you think that we would interpret those words to mean that you
wanted the example you give just above?

There's an apparent discrepancy between your call for an example and
your subsequent (in the same paragraph) reference to the example given.

But as to why I assumed that that example, or a similar correct one,
would be implied, it's the only meaningful interpretation.

Adopting a meaningless interpretation when a meaningful exists is
generally just adversarial, but in this case I was, as you pointed out,
extremely unclear, and I'm sorry: I should have given such example up
front. Will try to do so.

Thank you. I appreciate it.

[snip]

I agree. Atomic operations like chdir() help a lot. But this is
Python, and exceptions can happen in many different places. If you're
not just calling an extension module function that makes a
known-atomic system call, you run the risk of not having an atomic
operation.


And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a
common fallacy in discussions about programming, to assume that since
the same can be expressed using lower level constructs, those are all
that are required.

If adopted as true it ultimately means the removal of all control
structures above the level of "if" and "goto" (except Python doesn't
have "goto").

What I'm trying to explain is that the with: statement has a use even if Cleanup
doesn't. Arguing that Cleanup doesn't improve on try: finally: does not mean
that the with: statement doesn't improve on try: finally:.

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the namespace in
order to ensure that their refcounts go to zero and their cleanup code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's
required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in that
namespace, either. Not without sys._getframe() hackery, in any case.

However, note that the current language doesn't guarantee such cleanup,
at least as far as I know.

So while it's good practice to support it, to do everything to let it
happen, it's presumably bad practice to rely on it happening.



Thanks!, I hadn't thought of connecting that to general cleanup actions.

It limits the use of general "with" in the same way.

Not really. It's easy to write context managers that do that. You put the
initialization code in the __enter__() method, assign whatever objects you want
to keep around through the with: clause as attributes on the manager, then
delete those attributes in the __exit__(). Or, you use the @contextmanager
decorator to turn a generator into a context manager, and you just assign to
local variables and del them in the finally: clause.

What you can't do is write a generic context manager where the initialization
happens inside the with: clause and the cleanup actions are registered
callables. That does not allow you to affect the namespace.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

* Robert Kern:

That's "cleanup no matter what happen".

For the "# Do stuff" block, yes. For the initialization block, you can write a
context manager to do it either way, as necessary.

I don't know what "supernumerary" means,
http://www.merriam-webster.com/dictionary/supernumerary

but to the degree that the
argument says anything about a construct that is not 'finally', it says
the same about general "with".

He's ignorant of the use cases of the with: statement, true. Given only your
example of the with: statement, it is hard to fault him for thinking that try:
finally: wouldn't suffice.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Michael Rudolf]

Am 04.03.2010 18:20, schrieb Robert Kern:

What I'm trying to explain is that the with: statement has a use even if
Cleanup doesn't. Arguing that Cleanup doesn't improve on try: finally:
does not mean that the with: statement doesn't improve on try: finally:.

Yes, the with-statement rocks

I suggested such a thing a few days ago in another thread, where OP
wanted a "silent" keyword like:

silent:
do_stuff()

would be equivalent to:
try: do_stuff() except: pass

Of course catching *all* exceptions was a bad idea, so I came up with
the code below and now I actually like it and use it myself

---------snip---------
To your first question about a "silenced" keyword: you could emulate
this with context managers I guess.

Something like (untested, just a quick mockup how it could look):



class silenced:
def __init__(self, *silenced):
self.exceptions=tuple(silenced) #just to be explicit
def __enter__(self):
return self #dito
def __exit__(self, type, value, traceback):
for ex in self.exceptions:
if isinstance(value, ex):
return True #supresses exception


So:

with silenced(os.Error):
os.remove(somefile)

Would translate to:

try:
os.remove(somefile)
except os.Error:
pass

One nice thing about this approach would be that you can alias a set of
exceptions with this:

idontcareabouttheseerrors=silenced(TypeError, ValueError, PEBCAKError,
SyntaxError, EndOfWorldError, 1D10T_Error)

with idontcareabouttheseerrors:
do_stuff()

Regards,
Michael

 

[Michael Rudolf]

Am 04.03.2010 17:32, schrieb Jean-Michel Pichavant:

It looks like to me that 'with' statements are like decorators: overrated.

Oh no, you just insulted my favourite two python features, followed
immediately by generators, iterators and list comprehensions / generator
expressions

No, really: they *are* great ;D

Regards,
Michael

 

[Jean-Michel Pichavant]

Am 04.03.2010 17:32, schrieb Jean-Michel Pichavant:

Oh no, you just insulted my favourite two python features, followed
immediately by generators, iterators and list comprehensions /
generator expressions

No, really: they *are* great ;D

Regards,
Michael

They are great, and because of that greatness some of us, including me,
tend to use them where there's no point doing so.
I would never state that decorators are useless anyway, not in this
list, I value my life too much (I'll give it a try in the perl list)

JM

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern:

On 2010-03-03 18:49 PM, Alf P. Steinbach wrote: [snippety]

If you call the possibly failing operation "A", then that systematic
approach goes like this: if A fails, then it has cleaned up its own
mess, but if A succeeds, then it's the responsibility of the calling
code to clean up if the higher level (multiple statements) operation
that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed for,
and what the corresponding Python Cleanup class is designed for.

And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a
common fallacy in discussions about programming, to assume that since
the same can be expressed using lower level constructs, those are all
that are required.

If adopted as true it ultimately means the removal of all control
structures above the level of "if" and "goto" (except Python doesn't
have "goto").

What I'm trying to explain is that the with: statement has a use even if
Cleanup doesn't. Arguing that Cleanup doesn't improve on try: finally:
does not mean that the with: statement doesn't improve on try: finally:.

That's a different argument, essentially that you see no advantage for your
current coding patterns.

It's unconnected to the argument I responded to.

The argument that I responded to, that the possibility of expressing things at
the level of try:finally: means that a higher level construct is superfluous, is
still meaningless.

Both formulations can be correct (and both work perfectly fine with
the chdir() example being used). Sometimes one is better than the
other, and sometimes not. You can achieve both ways with either your
Cleanup class or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over try:
finally: and has the significant ugliness of forcing cleanup code into
callables. This significantly limits what you can do in your cleanup
code.

Uhm, not really. As I see it.

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the namespace in
order to ensure that their refcounts go to zero and their cleanup code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's
required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in
that namespace, either. Not without sys._getframe() hackery, in any case.

However, note that the current language doesn't guarantee such cleanup,
at least as far as I know.

So while it's good practice to support it, to do everything to let it
happen, it's presumably bad practice to rely on it happening.



Thanks!, I hadn't thought of connecting that to general cleanup actions.

It limits the use of general "with" in the same way.

Not really.

Sorry, it limits general 'with' in /exactly/ the same way.

It's easy to write context managers that do that [delete objects from the namespace].

Sorry, no can do, as far as I know; your following example quoted below is an
example of /something else/.

And adding on top of irrelevancy, for the pure technical aspect it can be
accomplished in the same way using Cleanup (I provide an example below).

However, doing that would generally be worse than pointless since with good
coding practices the objects would become unreferenced anyway.

You put
the initialization code in the __enter__() method, assign whatever
objects you want to keep around through the with: clause as attributes
on the manager, then delete those attributes in the __exit__().

Analogously, if one were to do this thing, then it could be accomplished using a
Cleanup context manager as follows:

foo = lambda: None
foo.x = create_some_object()
at_cleanup.call( lambda o = foo: delattr( o, "x" ) )

.... except that

1) for a once-only case this is less code

2) it is a usage that I wouldn't recommend; instead I recommend adopting good
coding practices where object references aren't kept around.

Or, you
use the @contextmanager decorator to turn a generator into a context
manager, and you just assign to local variables and del them in the
finally: clause.

Uhm, you don't need a 'finally' clause when you define a context manager.

Additionally, you don't need to 'del' the local variables in @contextmanager
decorated generator.

The local variables cease to exist automatically.

What you can't do is write a generic context manager where the
initialization happens inside the with: clause and the cleanup actions
are registered callables. That does not allow you to affect the namespace.

If you mean that you can't introduce direct local variables and have them
deleted by "registered callables" in a portable way, then right.

But I can't think of any example where that would be relevant; in particular
what matters for supporting on-destruction cleanup is whether you keep any
references or not, not whether you have a local variable of any given name.

And I think "with" is quite useful even with that restriction.


Cheers,

- Alf

 

[Robert Kern]

* Robert Kern:

* Robert Kern:
On 2010-03-03 18:49 PM, Alf P. Steinbach wrote: [snippety]

If you call the possibly failing operation "A", then that systematic
approach goes like this: if A fails, then it has cleaned up its own
mess, but if A succeeds, then it's the responsibility of the calling
code to clean up if the higher level (multiple statements) operation
that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed for,
and what the corresponding Python Cleanup class is designed for.

And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a
common fallacy in discussions about programming, to assume that since
the same can be expressed using lower level constructs, those are all
that are required.

If adopted as true it ultimately means the removal of all control
structures above the level of "if" and "goto" (except Python doesn't
have "goto").

What I'm trying to explain is that the with: statement has a use even
if Cleanup doesn't. Arguing that Cleanup doesn't improve on try:
finally: does not mean that the with: statement doesn't improve on
try: finally:.

That's a different argument, essentially that you see no advantage for
your current coding patterns.

It's unconnected to the argument I responded to.

The argument that I responded to, that the possibility of expressing
things at the level of try:finally: means that a higher level construct
is superfluous, is still meaningless.

I am attacking your premise that the "with Cleanup():" construct is higher level
than try: finally:. It isn't. It provides the same level of abstraction as try:
finally:.

This is distinct from the accepted uses of the with: statement which *are*
higher level than try: finally: and which do confer practical benefits over
using try: finally: despite being syntactical sugar for try: finally:.

Both formulations can be correct (and both work perfectly fine with
the chdir() example being used). Sometimes one is better than the
other, and sometimes not. You can achieve both ways with either your
Cleanup class or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over
try:
finally: and has the significant ugliness of forcing cleanup code
into
callables. This significantly limits what you can do in your cleanup
code.

Uhm, not really. As I see it.

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the namespace in
order to ensure that their refcounts go to zero and their cleanup code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's
required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in
that namespace, either. Not without sys._getframe() hackery, in any case.

However, note that the current language doesn't guarantee such cleanup,
at least as far as I know.

So while it's good practice to support it, to do everything to let it
happen, it's presumably bad practice to rely on it happening.


Tracebacks will keep the namespace alive and all objects in it.

Thanks!, I hadn't thought of connecting that to general cleanup actions.

It limits the use of general "with" in the same way.

Not really.

Sorry, it limits general 'with' in /exactly/ the same way.

It's easy to write context managers that do that [delete objects from
the namespace].

Sorry, no can do, as far as I know; your following example quoted below
is an example of /something else/.

Okay, so what do you mean by 'the use of general "with"'? I'm talking about
writing a context manager or using the @contextmanager decorator to do some
initialization and then later cleaning up that initialization. That cleaning up
may entail deleting an object. You are correct that the context manager can't
affect the namespace of the with: clause, but that's not the initialization that
it would need to clean up.

Yes, you can write code with a with: statement where you try to clean up stuff
that happened inside of the clause (you did), but that's not how the with:
statement was ever intended to be used nor is it good practice to do so because
of that limitation. Context managers are designed to initialize specific things,
then clean them up. I thought you were talking about the uses of the with:
statement as described in PEP-343, not every possible misuse of the with: statement.

And adding on top of irrelevancy, for the pure technical aspect it can
be accomplished in the same way using Cleanup (I provide an example below).

However, doing that would generally be worse than pointless since with
good coding practices the objects would become unreferenced anyway.



Analogously, if one were to do this thing, then it could be accomplished
using a Cleanup context manager as follows:

foo = lambda: None
foo.x = create_some_object()
at_cleanup.call( lambda o = foo: delattr( o, "x" ) )

... except that

1) for a once-only case this is less code

Not compared to a try: finally:, it isn't.

2) it is a usage that I wouldn't recommend; instead I recommend adopting
good
coding practices where object references aren't kept around.

Many of the use cases of the with: statement involve creating an object (like a
lock or a transaction object), keeping it around for the duration of the "# Do
stuff" block, and then finalizing it.

Uhm, you don't need a 'finally' clause when you define a context manager.

When you use the @contextmanager decorator, you almost always do. See the
Examples section of PEP 343:

http://www.python.org/dev/peps/pep-0343/

Additionally, you don't need to 'del' the local variables in
@contextmanager decorated generator.

The local variables cease to exist automatically.
True.


If you mean that you can't introduce direct local variables and have
them deleted by "registered callables" in a portable way, then right.

But I can't think of any example where that would be relevant; in
particular what matters for supporting on-destruction cleanup is whether
you keep any references or not, not whether you have a local variable of
any given name.

Well, local variables keep references to objects. Variable assignment followed
by deletion is a very readable way to keep an object around for a while then
remove it later. If you have to go through less readable contortions to keep the
object around when it needs to be and clean it up later, then that is a mark
against your approach.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Steve Holden]

* Robert Kern: [...]

No, it only argues that "with Cleanup():" is supernumerary.

I don't know what "supernumerary" means, but to the degree that the
argument says anything about a construct that is not 'finally', it says
the same about general "with".

So rather than look up the meaning if a word you aren't familiar with
you will argue against its use in generic style?

So whatever you mean by supernumerary, you're saying that the argument
implies that "with" is supernumerary.

This is starting to look like some earlier discussions in this group,
where even basic logic is denied.

Why not just stick to the facts and forget about the earlier discussions?

regards
Steve

 

[Mike Kent]

* Mike Kent:


if you thought about it you would mean a simple "try/else". "finally" is always
executed. which is incorrect for cleanup

by the way, that's one advantage:

a "with Cleanup" is difficult to get wrong, while a "try" is easy to get wrong,
as you did here

   ---

another general advantage is as for the 'with' statement generally


also, the "do other stuff" can be a lot of code

and also, with more than one action the try-else introduces a lot of nesting


cheers & hth.,

- alf

Wrong? In what way is my example wrong? It cleanly makes sure that
the current working directory is the same after the try/finally as it
was before it. Suboptimal, perhaps, in that the chdir in the finally
part is always executed, even if the chdir in the try part failed to
change the working directory.

That is a clear advantage to the code you presented, in that you have
the ability to register an 'undo' function only if the 'do' code
succeeded. Your code also avoids a problem with many nested try/
finally blocks. But for the simple chdir example you gave, I think
'wrong' isn't the word you were looking for regarding the try/finally
example I gave.

Anyway, I'll keep your code in mind the next time I want to avoid a
bunch of nested try/finally blocks.

 

[Mike Kent]

He's ignorant of the use cases of the with: statement, true.

<humor> Ouch! Ignorant of the use cases of the with statement, am I?

Given only your
example of the with: statement, it is hard to fault him for thinking that try:
finally: wouldn't suffice.

<humor> Damn me with faint praise, will you? </humor>

I'm kinda amazed at the drama my innocent request for the use case
elicited. From what I've gotten so far from this thread, for the
actual example Mr. Steinbach used, the only disadvantage to my counter-
example using try/finally is that the chdir in the finally part will
always be executed, even if the chdir in the try part did not
succeed. I concede that, and was aware of it when I wrote it. For
the simple example given, I did not consider it compelling. A more
complex example, that would have required multiple, nested try/finally
blocks, would show the advantages of Mr Steinbach's recipe more
clearly.

However, I fail to understand his response that I must have meant try/
else instead, as this, as Mr. Kern pointed out, is invalid syntax.
Perhaps Mr. Steinbach would like to give an example?

 

[Mike Kent]

A custom-written context manager looks nicer and can be more readable.

from contextlib import contextmanager
import os

@contextmanager
def pushd(path):
     original_dir = os.getcwd()
     os.chdir(path)
     try:
         yield
     finally:
         os.chdir(original_dir)

with pushd(somewhere):
     ...

Robert, I like the way you think. That's a perfect name for that
context manager! However, you can clear one thing up for me... isn't
the inner try/finally superfluous? My understanding was that there
was an implicit try/finally already done which will insure that
everything after the yield statement was always executed.

 

[Alf P. Steinbach]

* Mike Kent:

<humor> Ouch! Ignorant of the use cases of the with statement, am I?


<humor> Damn me with faint praise, will you? </humor>

I'm kinda amazed at the drama my innocent request for the use case
elicited. From what I've gotten so far from this thread, for the
actual example Mr. Steinbach used, the only disadvantage to my counter-
example using try/finally is that the chdir in the finally part will
always be executed, even if the chdir in the try part did not
succeed. I concede that, and was aware of it when I wrote it. For
the simple example given, I did not consider it compelling.

Uhm, well.

My example was:

with Cleanup as at_cleanup:
# blah blah
chdir( somewhere )
at_cleanup.call( lambda: chdir( original_dir ) )
# blah blah

It was not intended to compel, rather just to illustrate usage.

And you asked about comparing that with ...

original_dir = os.getcwd()
try:
os.chdir(somewhere)
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

... which does something different, namely, always executing the
os.chdir(original_dir) or more generally the action-specific cleanup.

The action-specific cleanup might be much more costly than a chdir, and/or, in
the case where the action failed, incorrect.

In the same number of lines and with fewer keystrokes you could have written
code that was equivalent to the code I posted and that you wanted to compare
with, e.g. ...

original_dir = os.getcwd()
os.chdir(somewhere)
try:
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

.... so given how easy it is to write such an equivalent code snippet, I assumed
that the different behavior was /not/ intended, that instead, you'd attempted to
write equivalent code but made a slight error in the translation to lower level
construct -- but impossible to say exactly what.

Now you write that you were "aware of [the different behavior] when I wrote it",
and that just baffles me: why not then, as a basis of sound comparision, write
the equivalent code shown above, the same number of lines as what you wrote?

A more
complex example, that would have required multiple, nested try/finally
blocks, would show the advantages of Mr Steinbach's recipe more
clearly.

However, I fail to understand his response that I must have meant try/
else instead, as this, as Mr. Kern pointed out, is invalid syntax.
Perhaps Mr. Steinbach would like to give an example?

OK.

Assuming that you wanted the chdir to be within a try block (which it was in
your code), then to get code equivalent to my code, for the purpose of a
comparision of codes that do the same, you'd have to write something like ...

original_dir = os.getcwd()
try:
os.chdir(somewhere)
except Whatever:
# E.g. log it.
raise
else:
try:
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

.... which would be a more general case.

I've also given this example in response to Robert earlier in the thread.
Although I haven't tried it I believe it's syntactically valid. If not, then the
relevant typo should just be fixed.

I have no idea which construct Robert thought was syntactically invalid. I think
that if he's written that, then it must have been something he thought of.


Cheers & hth.,

- Alf

 

[Alf P. Steinbach]

* Robert Kern:

* Robert Kern:

On 2010-03-04 10:56 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 18:49 PM, Alf P. Steinbach wrote: [snippety]

If you call the possibly failing operation "A", then that systematic
approach goes like this: if A fails, then it has cleaned up its own
mess, but if A succeeds, then it's the responsibility of the calling
code to clean up if the higher level (multiple statements) operation
that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed for,
and what the corresponding Python Cleanup class is designed for.

And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a
common fallacy in discussions about programming, to assume that since
the same can be expressed using lower level constructs, those are all
that are required.

If adopted as true it ultimately means the removal of all control
structures above the level of "if" and "goto" (except Python doesn't
have "goto").

What I'm trying to explain is that the with: statement has a use even
if Cleanup doesn't. Arguing that Cleanup doesn't improve on try:
finally: does not mean that the with: statement doesn't improve on
try: finally:.

That's a different argument, essentially that you see no advantage for
your current coding patterns.

It's unconnected to the argument I responded to.

The argument that I responded to, that the possibility of expressing
things at the level of try:finally: means that a higher level construct
is superfluous, is still meaningless.

I am attacking your premise that the "with Cleanup():" construct is
higher level than try: finally:. It isn't. It provides the same level of
abstraction as try: finally:.

This is distinct from the accepted uses of the with: statement which
*are* higher level than try: finally: and which do confer practical
benefits over using try: finally: despite being syntactical sugar for
try: finally:.

Both formulations can be correct (and both work perfectly fine with
the chdir() example being used). Sometimes one is better than the
other, and sometimes not. You can achieve both ways with either your
Cleanup class or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over
try:
finally: and has the significant ugliness of forcing cleanup code
into
callables. This significantly limits what you can do in your cleanup
code.

Uhm, not really. As I see it.

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the namespace in
order to ensure that their refcounts go to zero and their cleanup code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's
required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in
that namespace, either. Not without sys._getframe() hackery, in any
case.

However, note that the current language doesn't guarantee such cleanup,
at least as far as I know.

So while it's good practice to support it, to do everything to let it
happen, it's presumably bad practice to rely on it happening.


Tracebacks will keep the namespace alive and all objects in it.

Thanks!, I hadn't thought of connecting that to general cleanup
actions.

It limits the use of general "with" in the same way.

Not really.

Sorry, it limits general 'with' in /exactly/ the same way.

It's easy to write context managers that do that [delete objects from
the namespace].

Sorry, no can do, as far as I know; your following example quoted below
is an example of /something else/.

Okay, so what do you mean by 'the use of general "with"'? I'm talking
about writing a context manager or using the @contextmanager decorator
to do some initialization and then later cleaning up that
initialization. That cleaning up may entail deleting an object. You are
correct that the context manager can't affect the namespace of the with:
clause, but that's not the initialization that it would need to clean up.

Yes, you can write code with a with: statement where you try to clean up
stuff that happened inside of the clause (you did), but that's not how
the with: statement was ever intended to be used nor is it good practice
to do so because of that limitation. Context managers are designed to
initialize specific things, then clean them up. I thought you were
talking about the uses of the with: statement as described in PEP-343,
not every possible misuse of the with: statement.

I'm not the one talking about removing variables or that "it's easy to write
context managers that do that".

You are the one talking about that.

So I have really not much to add.

It seems that you're now agreeing with me that former is not good practice and
that the latter is impossible to do portably, but you now argue against your
earlier stand as if that was something that I had put forward.

It's a bit confusing when you argue against your own statements.

Not compared to a try: finally:, it isn't.

Again, this context shifting is bewildering. As you can see, quoted above, you
were talking about a situation where you would have defined a context manager,
presumably because a 'try' would not in your opinion be simpler for whatever it
was that you had in mind. But you are responding to the code I offered as if it
was an alternative to something where you would find a 'try' to be simplest.

Many of the use cases of the with: statement involve creating an object
(like a lock or a transaction object), keeping it around for the
duration of the "# Do stuff" block, and then finalizing it.


When you use the @contextmanager decorator, you almost always do. See
the Examples section of PEP 343:

http://www.python.org/dev/peps/pep-0343/


Well, local variables keep references to objects. Variable assignment
followed by deletion is a very readable way to keep an object around for
a while then remove it later. If you have to go through less readable
contortions to keep the object around when it needs to be and clean it
up later, then that is a mark against your approach.

Sorry, as with the places noted above, I can't understand what you're trying to
say here. I don't recommend coding practices where you keep object references
around, and you have twice quoted that above. I don't have any clue what
"contortions" you are talking about, it must be something that you imagine.


Cheers,

- Alf (three times baffled)

 

[Robert Kern]

Robert, I like the way you think. That's a perfect name for that
context manager! However, you can clear one thing up for me... isn't
the inner try/finally superfluous? My understanding was that there
was an implicit try/finally already done which will insure that
everything after the yield statement was always executed.

No, the try: finally: is not implicit. See the source for
contextlib.GeneratorContextManager. When __exit__() gets an exception from the
with: block, it will push it into the generator using its .throw() method. This
raises the exception inside the generator at the yield statement.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Alf P. Steinbach]

* Steve Holden:

* Robert Kern: [...]

No, it only argues that "with Cleanup():" is supernumerary.

I don't know what "supernumerary" means, but to the degree that the
argument says anything about a construct that is not 'finally', it says
the same about general "with".

So rather than look up the meaning if a word you aren't familiar with
you will argue against its use in generic style?

I haven't argued against the use of the word. I haven't done so in generic
style, and I haven't argued against generic style use of the word, whatever it
is you're trying to say. And I see that you're out trolling again, Steve Holden,
implying all sorts of things that are untrue, as is evidently still your style.

Why not just stick to the facts and forget about the earlier discussions?

For yet another example, here you are implying that I'm not sticking to facts,
which, again, is untrue, a technique that you should be ashamed of, Steve Holden.

And since you're now injecting some Steve Holden'sk noise into this debate,
chances are that in your points- and win/lose fixation you think I have scored a
point immediately upthread, something which you think needs drowning in noise.


Cheers,

- Alf

 

[Robert Kern]

<humor> Ouch! Ignorant of the use cases of the with statement, am I?
Odd, I use it all the time.</humor>

No, I was referring to Jean-Michel, who was not familiar with the with: statement.

<humor> Damn me with faint praise, will you?</humor>

Also talking about Jean-Michel.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

* Mike Kent:

OK.

Assuming that you wanted the chdir to be within a try block (which it
was in your code), then to get code equivalent to my code, for the
purpose of a comparision of codes that do the same, you'd have to write
something like ...

original_dir = os.getcwd()
try:
os.chdir(somewhere)
except Whatever:
# E.g. log it.
raise
else:
try:
# Do other stuff
finally:
os.chdir(original_dir)
# Do other cleanup

... which would be a more general case.

I've also given this example in response to Robert earlier in the
thread. Although I haven't tried it I believe it's syntactically valid.
If not, then the relevant typo should just be fixed.

I have no idea which construct Robert thought was syntactically invalid.
I think that if he's written that, then it must have been something he
thought of.

I was just trying to interpret what you meant by "Changing 'finally' to 'else'
could make it equivalent." As far as I can tell, that has only one possible
interpretation going by the plain meaning of the words, and it isn't yours.
Since you always seem to refer to "try/else" as if it were an independent
construct and not part of "try: except: else:" and no one else introduced
except: clause, I must reiterate that your communications have been fabulously
misleading.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

* Robert Kern:

* Robert Kern:
On 2010-03-04 10:56 AM, Alf P. Steinbach wrote:
* Robert Kern:
On 2010-03-03 18:49 PM, Alf P. Steinbach wrote:
[snippety]

If you call the possibly failing operation "A", then that systematic
approach goes like this: if A fails, then it has cleaned up its own
mess, but if A succeeds, then it's the responsibility of the calling
code to clean up if the higher level (multiple statements) operation
that A is embedded in, fails.

And that's what Marginean's original C++ ScopeGuard was designed
for,
and what the corresponding Python Cleanup class is designed for.

And try: finally:, for that matter.

Not to mention "with".

Some other poster made the same error recently in this thread; it is a
common fallacy in discussions about programming, to assume that since
the same can be expressed using lower level constructs, those are all
that are required.

If adopted as true it ultimately means the removal of all control
structures above the level of "if" and "goto" (except Python doesn't
have "goto").

What I'm trying to explain is that the with: statement has a use even
if Cleanup doesn't. Arguing that Cleanup doesn't improve on try:
finally: does not mean that the with: statement doesn't improve on
try: finally:.

That's a different argument, essentially that you see no advantage for
your current coding patterns.

It's unconnected to the argument I responded to.

The argument that I responded to, that the possibility of expressing
things at the level of try:finally: means that a higher level construct
is superfluous, is still meaningless.

I am attacking your premise that the "with Cleanup():" construct is
higher level than try: finally:. It isn't. It provides the same level
of abstraction as try: finally:.

This is distinct from the accepted uses of the with: statement which
*are* higher level than try: finally: and which do confer practical
benefits over using try: finally: despite being syntactical sugar for
try: finally:.

Both formulations can be correct (and both work perfectly fine with
the chdir() example being used). Sometimes one is better than the
other, and sometimes not. You can achieve both ways with either
your
Cleanup class or with try: finally:.

I am still of the opinion that Cleanup is not an improvement over
try:
finally: and has the significant ugliness of forcing cleanup code
into
callables. This significantly limits what you can do in your
cleanup
code.

Uhm, not really. As I see it.

Well, not being able to affect the namespace is a significant
limitation. Sometimes you need to delete objects from the
namespace in
order to ensure that their refcounts go to zero and their cleanup
code
gets executed.

Just a nit (I agree that a lambda can't do this, but as to what's
required): assigning None is sufficient for that[1].

Yes, but no callable is going to allow you to assign None to names in
that namespace, either. Not without sys._getframe() hackery, in any
case.

However, note that the current language doesn't guarantee such
cleanup,
at least as far as I know.

So while it's good practice to support it, to do everything to let it
happen, it's presumably bad practice to rely on it happening.


Tracebacks will keep the namespace alive and all objects in it.

Thanks!, I hadn't thought of connecting that to general cleanup
actions.

It limits the use of general "with" in the same way.

Not really.

Sorry, it limits general 'with' in /exactly/ the same way.

It's easy to write context managers that do that [delete objects from
the namespace].

Sorry, no can do, as far as I know; your following example quoted below
is an example of /something else/.

Okay, so what do you mean by 'the use of general "with"'? I'm talking
about writing a context manager or using the @contextmanager decorator
to do some initialization and then later cleaning up that
initialization. That cleaning up may entail deleting an object. You
are correct that the context manager can't affect the namespace of the
with: clause, but that's not the initialization that it would need to
clean up.

Yes, you can write code with a with: statement where you try to clean
up stuff that happened inside of the clause (you did), but that's not
how the with: statement was ever intended to be used nor is it good
practice to do so because of that limitation. Context managers are
designed to initialize specific things, then clean them up. I thought
you were talking about the uses of the with: statement as described in
PEP-343, not every possible misuse of the with: statement.

I'm not the one talking about removing variables or that "it's easy to
write context managers that do that".

You are the one talking about that.

So I have really not much to add.

It seems that you're now agreeing with me that former is not good
practice and that the latter is impossible to do portably, but you now
argue against your earlier stand as if that was something that I had put
forward.

No, I'm still saying that sometimes you do need to remove variables that you
initialized in the initialization section. Writing a purpose-built context
manager which encapsulates the initialization and finalization allows you to do
this easily. Putting the initialization section inside the with: clause, as you
do, and requiring cleanup code to be put into callables makes this hard.

It's a bit confusing when you argue against your own statements.


Again, this context shifting is bewildering. As you can see, quoted
above, you were talking about a situation where you would have defined a
context manager, presumably because a 'try' would not in your opinion be
simpler for whatever it was that you had in mind. But you are responding
to the code I offered as if it was an alternative to something where you
would find a 'try' to be simplest.

I have consistently put forward that for once-only cases, try: finally: is a
preferable construct to "with Cleanup():". I also put forward that for
repetitive cases, a purpose-built context manager is preferable. I note that for
both try: finally: and a purpose-built context manager, modifying the namespace
is easy to do while it is difficult and less readable to do with "with
Cleanup():". Since you were claiming that the "generic with:" would have the
same problem as "with Cleanup():" I was trying to explain for why all of the
intended use cases of the with: statement (the purpose-built context managers),
there is no problem. Capisce?

Sorry, as with the places noted above, I can't understand what you're
trying to say here. I don't recommend coding practices where you keep
object references around,

Then how do you clean up locks and transaction objects and similar things if you
don't keep them around during the code suite? Creating an object, keeping it
around while executing a specific chunk of code, and finalizing it safely is a
prime use case for these kinds of constructs.

and you have twice quoted that above. I don't
have any clue what "contortions" you are talking about, it must be
something that you imagine.

These are the contortions:

foo = lambda: None
foo.x = create_some_object()
at_cleanup.call( lambda o = foo: delattr( o, "x" ) )

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco