dumping in destructor

[íÉÔÑ]

I have a class which I want to save it's data automatically on disc,
when it's destroyed. I have following code:

from cPickle import dump

class __Register(object):
def __init__(self):
self.dict = {}
def __del__(self):
fh = open('aaa', 'w')
dump(self.dict, fh)
fh.close()

g_register = __Register() # global instance. I do not destroy it
manually, so destructor is called on iterpreter exit

But when g_register is being destroyed, dump seems to be already dead,
so I get:

Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound method __Register.__del__ of <MyWiki.Register.__Register object
at 0x835a74c>> ignored

can I somehow save my data from destructor?

 

[Marc 'BlackJack' Rintsch]

I have a class which I want to save it's data automatically on disc,
when it's destroyed. I have following code:

from cPickle import dump

class __Register(object):
def __init__(self):
self.dict = {}
def __del__(self):
fh = open('aaa', 'w')
dump(self.dict, fh)
fh.close()

g_register = __Register() # global instance. I do not destroy it
manually, so destructor is called on iterpreter exit

The call to `__del__()` is not guaranteed.

can I somehow save my data from destructor?

Reliably? No! Change your design, it won't work this way.

Ciao,
Marc 'BlackJack' Rintsch

 

[robert]

I have a class which I want to save it's data automatically on disc,
when it's destroyed. I have following code:

from cPickle import dump

class __Register(object):
def __init__(self):
self.dict = {}
def __del__(self):
fh = open('aaa', 'w')
dump(self.dict, fh)
fh.close()

g_register = __Register() # global instance. I do not destroy it
manually, so destructor is called on iterpreter exit

But when g_register is being destroyed, dump seems to be already dead,
so I get:

Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound method __Register.__del__ of <MyWiki.Register.__Register object
at 0x835a74c>> ignored

can I somehow save my data from destructor?

the order of __del__ execution is quite unreliable (depending on
implementation/version of Python).
Also there is problematic/circular garbage (gc.garbage).

=>

1. For algorithmic use cases see 'with' statement
http://python.about.com/od/gettingstarted/qt/py25WITH.htm


2. For data structure trees implement your own register/deregister
book-keeping. Often a kind of "second" container-refcount
attribute does it at core in cases with multiple but non-circular
linking.
Yet, to get the effect in a cheap way, one can often do something
like this:
Add those objects also to a global container

g_reg_containter.add( myObject )

Then periodically and at critical times (and maybe finally
sys.exitfunc) after gc.collect() check the sys.getrefcount(obj)
(or the "second" refcount) to fall below "1 plus number of extra
local refs". In case execute your obj.__deregister() or so ...


Robert

 

[Michele Simionato]

But when g_register is being destroyed, dump seems to be already dead,
so I get:

Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound method __Register.__del__ of <MyWiki.Register.__Register object
at 0x835a74c>> ignored

can I somehow save my data from destructor?

The best thing is to use the 'with' statement, but it requires you to
rewrite all
of your code. Alternatively you can use the atexit module. I wrote
once a
recipe that may be of interest to you:

http://code.activestate.com/recipes/523007/

 

[íÉÔÑ]

Thanks for your answers!

my g_register is a global register, wich contains all my objects and
lives all the program lifetime. So 'with' is not appliable. Am I
right?

But using atexit sounds to be a good solution

 

[íÉÔÑ]

Thank you for your answers!

my g_register is a global object, and it lives all the program's
lifetime, so 'with' is not appliable. Am I right?

I tried to use atexit and wrote following:

class _Register(object):
def dump(self):
....

class Registerable(object):
....

g_register = _Register()
atexit.register(g_register.dump)
....
....
g_register.add(Registerable('aa'))

But now I get:

cPickle.PicklingError: Can't pickle <class '__main__.Registerable'>:
attribute lookup __main__.Registerable failed

Does that mean that by the time of atexit execution my Registerable
class is already dead?

 

[Gabriel Genellina]

Thank you for your answers!

my g_register is a global object, and it lives all the program's
lifetime, so 'with' is not appliable. Am I right?

Why not? You could use a with statement (or try/finally) around your main
entry point.

I tried to use atexit and wrote following:

class _Register(object):
def dump(self):
....

class Registerable(object):
....

g_register = _Register()
atexit.register(g_register.dump)
...
...
g_register.add(Registerable('aa'))

But now I get:

cPickle.PicklingError: Can't pickle <class '__main__.Registerable'>:
attribute lookup __main__.Registerable failed

Does that mean that by the time of atexit execution my Registerable
class is already dead?

No, at least not due to using atexit. When atexit functions are executed,
the interpreter is still in a fully working state. From pythonrun.c,
function Py_Finalize:

/* The interpreter is still entirely intact at this point, and the
* exit funcs may be relying on that. In particular, if some thread
* or exit func is still waiting to do an import, the import machinery
* expects Py_IsInitialized() to return true. So don't say the
* interpreter is uninitialized until after the exit funcs have run.
* Note that Threading.py uses an exit func to do a join on all the
* threads created thru it, so this also protects pending imports in
* the threads created via Threading.
*/

Probably you have another problem in your code; try to use pickle alone
(not within atexit) and see what happens.

 

[íÉÔÑ]

Thank you! I have already implemented custom load/save operations
without pickle. And they work fine on atexit.

Just for information: pickle.dump worked OK when called manually, but
being called by atexit it produeced the above described error. I don't
know why.

 

[Gabriel Genellina]

Thank you! I have already implemented custom load/save operations
without pickle. And they work fine on atexit.

Just for information: pickle.dump worked OK when called manually, but
being called by atexit it produeced the above described error. I don't
know why.

This small example (based on your code) worked fine for me:

import atexit
from pickle import dump

class _Register(object):
def dump(self):
dump(self.items, open("x.xxx","wb"), -1)

class Registerable(object):
pass

g_register = _Register()
atexit.register(g_register.dump)
g_register.items = [Registerable(),Registerable()]