__del__ doesn't work

[Peter Abel]

I have an application, which is an instance of a class
with a deeply nested object hierarchy. Among others one
method will be executed as a thread, which can be stopped.
Everything works fine except that when deleting the main
instance - after the thread has been stopped -
the __del__ method will not be carried out.

Tough a simple example works as expected:.... def __init__(self):
.... print 'Constructor of A()'
.... def __del__(self):
.... print 'Destructor of A()'
.... Destructor of A()

After all I know I can't expect that the __del__ method
is executed imediatly when deleting the instance
e.g. "del a" as in the example.

But the statements
import gc

and

gc.collect()

should assure that when the reference counter goes to zero
some time the __del__ method is executed.
But it never does.

So am I expecting something wrong
or am I doing something not in the right way?

The code is too complex to post it here or even a snippet.
But I think it's not a matter code but a matter of philosophy
that I may not have understood.
Any help would be appreciated.

Regards
Peter

 

[Aahz]

I have an application, which is an instance of a class with a deeply
nested object hierarchy. Among others one method will be executed as a
thread, which can be stopped. Everything works fine except that when
deleting the main instance - after the thread has been stopped - the
__del__ method will not be carried out.

It's almost certainly the case that you've still got a reference to that
instance somewhere. Try holding a binding to the instance; if
sys.getrefcount() returns >2, you know there's some other binding in
existence that you need to track down.

 

[Graham Dumpleton]

I have an application, which is an instance of a class
with a deeply nested object hierarchy. Among others one
method will be executed as a thread, which can be stopped.
Everything works fine except that when deleting the main
instance - after the thread has been stopped -
the __del__ method will not be carried out.

Does your deeply nested object hierarchy contain references back
onto the top level class instance itself? For example, run the following:

print "start"

class A:
def __init__(self):
print "init"
def __del__(self):
print "del"

a = A()
del a

class B:
def __init__(self):
print "init"
self.a = self
def __del__(self):
print "del"

b = B()
del b

print "finish"

And you get:

start
init
del
init
finish

Ie., you don't see "del" for class B. This is because B had a
member variable which looped back onto the object itself.

Same thing can happen where a class holds a reference back to
a method of the class, ie., a callback.

I don't believe that Python can cope with this and the object
will never actually be deleted. Please correct me if I am wrong.

In certain circumstances where I have had this problem and I
needed to know the object would be destroyed as soon as
possible, I have had to add a method to the class which could
be called to undo any references the class held which pointed
back onto itself.

I have looked at weak references, but have not been satisfied
that it could be used in the situation where I had to overcome the
problem.

 

[Josiah Carlson]

I don't believe that Python can cope with this and the object

will never actually be deleted. Please correct me if I am wrong.

You are incorrect.

From http://www.python.org/doc/current/lib/module-gc.html:
"The gc module is only available if the interpreter was built with the
optional cyclic garbage detector (enabled by default). If this was not
enabled, an ImportError is raised by attempts to import this module."

It is enabled in Windows builds, and I believe it is also enabled in all
other distributed builds.

In general, cycle detection is costly, and is only done when a large
amount of memory is being used.

In certain circumstances where I have had this problem and I
needed to know the object would be destroyed as soon as
possible, I have had to add a method to the class which could
be called to undo any references the class held which pointed
back onto itself.

That is smart, but it does beg the question: Why would a class instance
need to have a reference to itself? I can understand certain reasons,
but many can be translated into weakrefs with little difficulty.

I have looked at weak references, but have not been satisfied
that it could be used in the situation where I had to overcome the
problem.

Weak referenced objects are not required to be deleted immediately
following the final strong reference removal. Perhaps that is where
your confusion lies.

Nowhere does it claim to delete immediately following the final strong
reference removal, the garbage collector collects when it is ready to.
One thing to remember is that sometimes references hide.
.... def __init__(self):
.... print "init"
.... self.me = weakref.ref(self)
.... def __del__(self):
.... print "del"
....Traceback (most recent call last):
del
"I wonder why a hasn't been deleted yet..."('Ahhh', "I wonder why a hasn't been deleted yet...")


- Josiah

 

[Duncan Booth]

(e-mail address removed) (Peter Abel) wrote in

But the statements
import gc

and

gc.collect()

should assure that when the reference counter goes to zero
some time the __del__ method is executed.

If the class is involved in a cycle, then the __del__ method is never
called.

The garbage collector will simply refuse to collect any objects which have
__del__ methods. The reason for this is quite simple: if A and B refer to
each other, and both have __del__ methods, which class should be released
first? Python refuses to guess, so neither is released.

The garbage collector can trigger a __del__ method when some objects are in
a cycle and none of them have __del__ methods; then the cycle may be
released and if this causes some other objects to lose their last reference
then they will be released and associated __del__ methods called.

 

[Peter Abel]

(e-mail address removed) (Peter Abel) wrote in message
Thank you guys for your quick answers.
Back reference is the problem. It's done in a not quite obvious way,
so I didn't realize it immediately. I'll try to find a workaround
either by weakref or anything else to avoid this.

Many Thanks
Peter

 

[Graham Dumpleton]

You are incorrect.

From http://www.python.org/doc/current/lib/module-gc.html:
"The gc module is only available if the interpreter was built with the
optional cyclic garbage detector (enabled by default). If this was not
enabled, an ImportError is raised by attempts to import this module."

...

Hmmm, that isn't quite how I read it. If objects in a cycle have
__del__
methods, you have to break the cycles somehow still. See:

garbage

A list of objects which the collector found to be unreachable but
could not
be freed (uncollectable objects). By default, this list contains
only objects
with __del__() methods. Objects that have __del__() methods and
are part
of a reference cycle cause the entire reference cycle to be
uncollectable,
including objects not necessarily in the cycle but reachable only
from it.
Python doesn't collect such cycles automatically because, in
general, it isn't
possible for Python to guess a safe order in which to run
the__del__() methods.
If you know a safe order, you can force the issue by examining the
garbage list,
and explicitly breaking cycles due to your objects within the
list. Note that these
objects are kept alive even so by virtue of being in the garbage
list, so they
should be removed from garbage too. For example, after breaking
cycles, do
del gc.garbage[:] to empty the list. It's generally better to
avoid the issue by
not creating cycles containing objects with __del__() methods, and
garbage
can be examined in that case to verify that no such cycles are
being created.

That is smart, but it does beg the question: Why would a class instance
need to have a reference to itself? I can understand certain reasons,
but many can be translated into weakrefs with little difficulty.

Because the Python class was a wrapper around a C++ class which
provided callbacks from an event system. In the Python wrapper, one
specified in the code dynamically what method of the Python class
you wanted the event delivered to. Thus, the Python class has a
reference back not to the class, but a method of the class. This has
the same effect of creating a loop. Ie.,

class A:

def __init__(self):
print "init"
self.a = self._a

def __del__(self):
print "del"

def _a(self):
pass

a = A()
del a

Will print "init" only.

Or for an actual example:

import netsvc

class Handler(netsvc.Agent):

def __init__(self):
netsvc.Agent.__init__(self)
print "init"
self.startTimer(self.timeout,30)

def __del__(self):
print "del"

def timeout(self):
print "timeout"

o = Handler();
del o

netsvc.Dispatcher().run()

If an attempt was made to delete the Handler object before the timer
expired, it wouldn't. Instead it would hang around until the timer
expired, which isn't what was wanted. Thus need to do:

o.Handler()
o.destroyAgent()
del o

Where the destroyAgent() method undoes any cycles caused by the
registered callbacks.

Weak referenced objects are not required to be deleted immediately
following the final strong reference removal. Perhaps that is where
your confusion lies.

Nowhere does it claim to delete immediately following the final strong
reference removal, the garbage collector collects when it is ready to.
One thing to remember is that sometimes references hide.

That things aren't deleted immediately is why I wasn't satsified that
it
could be used in my situation. I needed the immediate deletion that
would normally occur in CPython when a final reference was deleted.
Without it, the wrapped C++ class wouldn't get destroyed and the
registrations it had made on behalf of the Python wrapper to events
generated by the event system wouldn't be removed. Thus per example
above, even though you had deleted the local reference to the object,
the object was still active and still receiving events. If the handler
did
some sort of permanent event registration against some aspect of the
event system, it would never die.