Why derivated exception can not be pickled ?

[Mathieu Courtois]

Here is my example :


import cPickle

ParentClass = object # works
ParentClass = Exception # does not

class MyError(ParentClass):
def __init__(self, arg):
self.arg = arg

def __getstate__(self):
print '#DBG pass in getstate'
odict = self.__dict__.copy()
return odict

def __setstate__(self, state):
print '#DBG pass in setstate'
self.__dict__.update(state)

exc = MyError('IDMESS')

fo = open('pick.1', 'w')
cPickle.dump(exc, fo)
fo.close()

fo = open('pick.1', 'r')
obj = cPickle.load(fo)
fo.close()


1. With ParentClass=object, it works as expected.

2. With ParentClass=Exception, __getstate__/__setstate__ are not called.

Does anyone explain me why ?
Thanks.

 

[Steven D'Aprano]

Here is my example :


import cPickle

ParentClass = object # works
ParentClass = Exception # does not [...]
1. With ParentClass=object, it works as expected.

2. With ParentClass=Exception, __getstate__/__setstate__ are not called.

Does anyone explain me why ?

I think it is a bug. According to the documentation, if your class is
unpickleable, an exception should be raised. If it is pickleable,
__getstate__ should be called. I can't see anything to explain that what
you are seeing is expected behaviour.


Exceptions should definitely be pickleable:

http://bugs.python.org/issue1692335

so __getstate__ should be called. I think you should report this as a bug.

 

[Dieter Maurer]

Here is my example :


import cPickle

ParentClass = object # works
ParentClass = Exception # does not

class MyError(ParentClass):
def __init__(self, arg):
self.arg = arg

def __getstate__(self):
print '#DBG pass in getstate'
odict = self.__dict__.copy()
return odict

def __setstate__(self, state):
print '#DBG pass in setstate'
self.__dict__.update(state)

exc = MyError('IDMESS')

fo = open('pick.1', 'w')
cPickle.dump(exc, fo)
fo.close()

fo = open('pick.1', 'r')
obj = cPickle.load(fo)
fo.close()


1. With ParentClass=object, it works as expected.

2. With ParentClass=Exception, __getstate__/__setstate__ are not called.

The pickle interface is actually more complex and there are several
ways an object can ensure picklability. For example, there is
also a "__reduce__" method. I suppose, that "Exception" defines methods
which trigger the use of an alternative picklability approach (different
from "__getstate__/__setstate__").

I would approach your case the following way: Use "pickle" instead
of "cPickle" and debug picking/unpickling to find out what
happens in detail.

 

[Mathieu Courtois]

Thanks for your reply

The pickle interface is actually more complex and there are several

ways an object can ensure picklability. For example, there is

also a "__reduce__" method. I suppose, that "Exception" defines methods

which trigger the use of an alternative picklability approach (different

from "__getstate__/__setstate__").

You're right: Exception has __reduce__ & __reduce_ex__ methods. Always read carefully the manual ;-)

I must override these methods.


MC

 

[Mathieu Courtois]

Thanks for your reply

The pickle interface is actually more complex and there are several

ways an object can ensure picklability. For example, there is

also a "__reduce__" method. I suppose, that "Exception" defines methods

which trigger the use of an alternative picklability approach (different

from "__getstate__/__setstate__").

You're right: Exception has __reduce__ & __reduce_ex__ methods. Always read carefully the manual ;-)

I must override these methods.


MC

 

[Mathieu Courtois]

Hello,

The simple example works fine using __reduce__:

class MyError(Exception):
def __init__(self, arg):
self.arg = arg

def __reduce__(self):
return (MyError, (self.arg, ))