Crash in thread on program termination

[Alec Wysoker]

I have several programs that have two or more threads. The threads that are not the main thread are daemon threads, i.e. the fact that they are running should not stop the program from terminating if the main thread (the only non-daemon thread) terminates. I do not join to these threads, but just run off the end of the main module to terminate the program. I'm using Python 2.3.4. I occasionally get the following error:

Traceback (most recent call last):
File "C:\Python23\lib\threading.py", line 451, in __bootstrap
self.__stop()
File "C:\Python23\lib\threading.py", line 460, in __stop
self.__block.notifyAll()
File "C:\Python23\lib\threading.py", line 256, in notifyAll
self.notify(len(self.__waiters))
File "C:\Python23\lib\threading.py", line 238, in notify
currentThread() # for side-effect
TypeError: 'NoneType' object is not callable


What seems to be happening is that at the time the thread is stopped, the threading module object is no longer in an unhealthy state. I.e. threading.currentThread is None. I hacked around this problem by substituting my own code for _Condition.notifyAll() that would not call notify() if there were no waiters to be notified. However, that merely allowed the code to get a little further, before crashing with a similar problem, i.e. threading._StringIO is None instead of being a class object.

Traceback (most recent call last):
File "C:\Python23\lib\threading.py", line 443, in __bootstrap
s = _StringIO()
TypeError: 'NoneType' object is not callable


I turned on threading.__debug__ and threading._VERBOSE to get some debug output, and one thing that is strange is that I don't get the debugging output I expect.

# From threading.Thread.__bootstrap():
try:
self.run()
except SystemExit:
if __debug__:
self._note("%s.__bootstrap(): raised SystemExit", self)
except:
if __debug__:
self._note("%s.__bootstrap(): unhandled exception", self)
s = _StringIO()
_print_exc(file=s)
_sys.stderr.write("Exception in thread %s:\n%s\n" %
(self.getName(), s.getvalue()))
else:
if __debug__:
self._note("%s.__bootstrap(): normal return", self)
finally:
self.__stop() # Line 451


I.e. I would expect to get one of the messages ('raised SystemExit', 'unhandled exception', or 'normal return') before self.__stop() is called at line 451. However, I do not see any of those messages.

Any ideas or suggestions? I am baffled.

Thanks,

Alec Wysoker

 

[Peter Hansen]

I have several programs that have two or more threads. The threads
> that are not the main thread are daemon threads, i.e. the fact
> that they are running should not stop the program from terminating
> if the main thread (the only non-daemon thread) terminates.

Traceback (most recent call last): [...]
File "C:\Python23\lib\threading.py", line 238, in notify
currentThread() # for side-effect
TypeError: 'NoneType' object is not callable

What seems to be happening is that at the time the thread is stopped, the threading
> module object is no longer in an unhealthy state.
I.e. threading.currentThread is None.

In a nutshell, the interpreter is dismantling itself while the
daemon threads merrily continue to run -- or attempt to. During
one of the steps in this dismantling process, the interpreter
goes through and rebinds all the globals in all modules to the
None object. You're seeing the effect of a daemon thread
encountering the inevitable bad effects of this.

For background, search the list archives and focus on messages
written by Tim Peters. Also try Googling the web for, say,
"python interpreter daemon thread shutdown nonetype" or
something... there are various not entirely conclusive threads
on the subject here and there.

I'm not sure there is a generally acceptable solution. You'll
notice that the ultimate response of the code in __bootstrap()
is still to try to print to stderr... which is a harmless but
otherwise perhaps distracting thing to do.

In my own code where this was a bother, I have something like
the following at the highest levels of the run() method of
my daemon threads:

def run(self):
try:
# do real stuff here
except Exception, ex:
if (self.isDaemon() and
isinstance(ex, AttributeError) and
ex[0].startswith("'NoneType'")):
pass

In other words, I'm attempting to catch those specific types of
spurious error and avoid letting __bootstrap() have its say.
At the moment, 98% of the spurious messages are gone and that's
good enough for my purposes...

-Peter