Unbound names in __del__

[Torsten Bronger]

Hallöchen!

When my __del__ methods are called because the program is being
terminated, I experience difficulties in calling functions that I
need for a clean shutdown of my instances. So far, there has been
only one of these functions, and a class-local alias solved the
problem. However, now there are many of them.

Is there a way to detect whether the program is being terminated?
(In this case, I wouldn't care and return from __del__ immediately.)

Or do you know a cleaner solution? For example, if I have

import vpp43

would it help to say

def __init__(self):
__vpp43 = vpp43
...

to guarantee that I can access all the routines in vpp43 in the
__del__ method?

Tschö,
Torsten.

 

[Terry Reedy]

Is there a way to detect whether the program is being terminated?

See atexit module to register cleanup functions that run *before* the
interpreter starts haphazardly deleting stuff.

Terry J. Reedy

 

[Torsten Bronger]

Hallöchen!

See atexit module to register cleanup functions that run *before*
the interpreter starts haphazardly deleting stuff.

So I could register a function that sets a global variable called
"shutdown_has_begun" to "True". Then I say

def __del__(self):
if shutdown_has_begun: return
...

Alternatively, I could enclose every __del__ contents block with a
"try" whithout catching anything, just to suppress the error
messages.

However, all of this is not pretty pythonic in my opinion. Is it
that exotic to want to call functions from within __del__?

Tschö,
Torsten.

 

[Michael Hoffman]

When my __del__ methods are called because the program is being
terminated, I experience difficulties in calling functions that I
need for a clean shutdown of my instances. So far, there has been
only one of these functions, and a class-local alias solved the
problem. However, now there are many of them.

__del__ is messy and I avoid it whenever possible. Make sure you read
all the caveats and warnings on these pages:

http://www.python.org/doc/ref/customization.html
http://www.python.org/doc/lib/module-gc.html

Most importantly, "It is not guaranteed that __del__() methods are
called for objects that still exist when the interpreter exits."

Or do you know a cleaner solution? For example, if I have

import vpp43

would it help to say

def __init__(self):
__vpp43 = vpp43
...

to guarantee that I can access all the routines in vpp43 in the
__del__ method?

A similar idiom is found in the standard library (e.g. tempfile.py):

# Cache the unlinker so we don't get spurious errors at
# shutdown when the module-level "os" is None'd out. Note
# that this must be referenced as self.unlink, because the
# name TemporaryFileWrapper may also get None'd out before
# __del__ is called.
unlink = _os.unlink

def close(self):
if not self.close_called:
self.close_called = True
self.file.close()
self.unlink(self.name)

def __del__(self):
self.close()

 

[Peter Hansen]

However, all of this is not pretty pythonic in my opinion. Is it
that exotic to want to call functions from within __del__?

Yes, I think it probably is. In the few hundred thousand lines of
Python code I've played a role in developing, we've used __del__ once,
to my knowledge, and I believe it was not a critical use, just an
expedient one.

And I don't recall the last time I saw a __del__ in third-party code I
was examining.

What's your use case for del?

-Peter

 

[Torsten Bronger]

Hallöchen!

[...]

What's your use case for del?

Every instance represents a "session" to a measurement instrument.
After the instance is deleted, the session should be closed to free
resources.

If the program exists, this is actually not necessary, because then
all resources are freed anyway. __del__ is called nevertheless.

Tschö,
Torsten.

 

[Peter Hansen]

Every instance represents a "session" to a measurement instrument.
After the instance is deleted, the session should be closed to free
resources.

You mean like GPIB devices? We've written a lot of software that talks
to instruments, as well as pumps, motors, and sensors of all kinds. I
haven't even needed to "free resources", other than by closing a serial
port, for example. Such simple operations don't require the use of
__del__ and can easily be done with simple .close() type calls as the
application shuts itself down or finishes a test sequence or other
operation.

Use of __del__ is, in my opinion, a bit of a crutch (meaning it seems to
make life easier, but then you start relying on it and find it hard to
do without). Given that it isn't really reliable in non-trivial
situations, I'd recommend pretending __del__ does not exist and
restructuring your system to close these sessions explicitly, under your
direct control, at the appropriate point. This has worked very well for
us so far.

-Peter

 

[Torsten Bronger]

Hallöchen!

Every instance represents a "session" to a measurement instrument.
After the instance is deleted, the session should be closed to free
resources.

You mean like GPIB devices?
Yes.

We've written a lot of software that talks to instruments, as well
as pumps, motors, and sensors of all kinds. I haven't even needed
to "free resources", other than by closing a serial port, for
example. [...]

I've just finished a thin-wrappers implementation of VISA in Python,
see <http://pyvisa.sf.net/>. It calls functions in a proprietary
VISA DLL/SO. The next step is to build a simple-to-use OO layer on
top of it. Therefore, we don't communicate with the device
directly, but via sessions (=handles) within the DLL.

These sessions should be freed when the object instance representing
the device is destroyed by Python. Using the __del__ method is the
natural choice for this in my opinion.

[...] I'd recommend pretending __del__ does not exist and
restructuring your system to close these sessions explicitly,
under your direct control, at the appropriate point. This has
worked very well for us so far.

I'd find this quite sad because so far it's drop-dead simple to use
the OO layer, and I may even convince our in-house HT Basic fans of
Python:

keithley = GpibInstrument(14)
keithley.write("*IDN?")
print keithley.read()

A keithley.close() would be a wart in my opinion; instead I want to
hide the whole session thing from the programmer. Besides, I
haven't yet given up the hope that the issues with __del__ can be
tackled.

Tschö,
Torsten.

 

[Peter Hansen]

keithley = GpibInstrument(14)
keithley.write("*IDN?")
print keithley.read()

A keithley.close() would be a wart in my opinion; instead I want to
hide the whole session thing from the programmer. Besides, I
haven't yet given up the hope that the issues with __del__ can be
tackled.

At least one alternative comes to mind. Have the GpibInstrument class
(or its module) register an atexit() method, and have the constructor
for that class track all instances. On shutdown, the atexit method goes
through all instruments that are still open and issues the .close()
requests, or whatever you do in the __del__ now.

In other words, it would be indistinguishable from __del__ from the
users' point of view, at the cost of a little extra code to make things
explicit, instead of relying on the implicit and, unfortunately,
unreliable nature of __del__. (Which is probably the real wart in
Python, unfortunately.)

-Peter

 

[Torsten Bronger]

Hallöchen!

At least one alternative comes to mind. Have the GpibInstrument
class (or its module) register an atexit() method, and have the
constructor for that class track all instances. On shutdown, the
atexit method goes through all instruments that are still open and
issues the .close() requests, or whatever you do in the __del__
now.

However, this doesn't close sessions while the program is running.
If the programmer has the above code in a function which is called
repeatedly, he may run into trouble. IIRC my current VISA DLL has
only 256 session slots.

Tschö,
Torsten.

 

[Dieter Maurer]

...
And I don't recall the last time I saw a __del__ in third-party code I
was examining.


What's your use case for del?

I had to use one a few days ago:

To call the "unlink" method of a "minidom" object when
its "container" is destroyed.

It would have been possible to let the cyclic garbage
collector find and eliminate the cyclic "minidom" objects.
But, DOMs may be large and I create lots of them in
a short time (each in its own container)...


Dieter

 

[Peter Hansen]

[on using atexit]
However, this doesn't close sessions while the program is running.
If the programmer has the above code in a function which is called
repeatedly, he may run into trouble. IIRC my current VISA DLL has
only 256 session slots.

Hmm... it sounds to me as though the design of this DLL is such that one
is expected to hold a session open for the duration of the application.
So instead of creating new GpibInstrument objects as needed, then just
sort of dropping them, the programmers would have to create the required
instrument objects at the start of the program and reuse them when needed.

But that's as far as my thoughts take me on this, without working
directly with you. I guess you'll have to explore the __del__ approach
through to its conclusion before you'll be confident it won't work (or
perhaps you'll find a solution after all!).

-Peter

 

[Terry Reedy]

However, this doesn't close sessions while the program is running.
If the programmer has the above code in a function which is called
repeatedly, he may run into trouble. IIRC my current VISA DLL has
only 256 session slots.

Deleting during a run is a different issue from deleting during program
shutdown, which is where you started us. In CPython, 'del keithley' will,
if keithley is the last reference to the object, dependably delete and
object, calling __del__ on the way. For other implementations, deleting
the last reference makes the object eligible to be deleted 'sometime' (if
and when garbage collection happens). So an explicit close is the only
dependable cross-implementation method that I know of.

Terry J. Reedy

 

[Torsten Bronger]

Hallöchen!

[...]

However, this doesn't close sessions while the program is
running. If the programmer has the above code in a function
which is called repeatedly, he may run into trouble. IIRC my
current VISA DLL has only 256 session slots.

Deleting during a run is a different issue from deleting during
program shutdown, which is where you started us.

Well, I talked about shutdown in the OP because only *then* I had
experienced problems. Cleaning up is important in every phase of
the program, though.

[...] So an explicit close is the only dependable
cross-implementation method that I know of.

I agree, but I could live with the little danger of a Python
implementation that frees so slowly that the DLL runs out of
handles. One could add a close() method for emergency use.
However, this __del__ thingy is so damn tempting ... ;-)

Tschö,
Torsten.