Why is SETUP_FINALLY uninterruptable[sic]?

[David Pokorny]

Whilst swimming through Python/ceval.c, I came upon the following:

PyEval_EvalFrame(...) {
....
<main interpreter loop>
....
if (--_Py_Ticker < 0) {
if (*next_instr == SETUP_FINALLY) {
/* Make the last opcode before
a try: finally: block
uninterruptable. */
goto fast_next_opcode;
}
_Py_Ticker = _Py_CheckInterval;
tstate->tick_counter++;
.... <do "occasional" things like let other threads run, ... occurs every
sys.getcheckinterval() bytecodes>
}

I've been trying to find a good reason _why_ the main loop shouldn't perform
the check at the usual time in the (highly rare)
situation where SETUP_FINALLY occurs as the last instruction before a check.
So far, I've managed to confuse myself even further and
come up with another stumper: what makes SETUP_FINALLY more special than
SETUP_LOOP and SETUP_EXCEPT?

The best answer I can think of is this: if another thread ran in between the
SETUP_FINALLY and the next first instruction of the
try block, then this other thread might step on some globals that
SETUP_FINALLY uses to communicate to the try block. As far as I can tell, no
such communication takes place and this isn't the reason.

Could a Python guru please enlighten me?

David

 

[Jeff Epler]

Well, it's not an answer per se, but this block of code was added at the
same time as PyThreadState_SetAsyncExc, which is a C API to allow
raising an exception from an arbitrary thread.

http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Python/ceval.c#rev2.366

I suspect that this has something to do with code like

l = some_lock()
l.acquire()
# MARK
try:
something with l held
finally:
l.release()
If an asynchronous exception happens betwen l.acquire() and the
SETUP_FINALLY opcode (at "MARK") , there's no guarantee that l.release()
is executed.

This state of affairs always existed with KeyboardInterrupt and/or
signals, and it's hard to see how this fixes the problem if l.acquire()
is Python code, but I think that's what's going on.

The special magic is only applied to SETUP_FINALLY because try: except:
isn't used for this kind of resource management.

This is all guesswork, though.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBIrdeJd01MZaTXX0RArqkAJ49yzUNPscYJ3M0ARIAzgFp7HNYVwCgmrA2
MRtRV7JqB6Wi5/GgaNIGYPY=
=uG8Q
-----END PGP SIGNATURE-----

 

[David Pokorny]

I suspect that this has something to do with code like

l = some_lock()
l.acquire()
# MARK
try:
something with l held
finally:
l.release()
If an asynchronous exception happens betwen l.acquire() and the
SETUP_FINALLY opcode (at "MARK") , there's no guarantee that l.release()
is executed.

This state of affairs always existed with KeyboardInterrupt and/or
signals, and it's hard to see how this fixes the problem if l.acquire()
is Python code, but I think that's what's going on.

I would belive you, but there is a trivial modification that circumvents
this problem:

---begin---
mylock = some_lock()

try:
mylock.acquire()
something with mylock held
finally:
mylock.release()
---end---

Now if you're saying that lots of people wrote code as above and
uninterruptibility was added to ensure that it wouldn't break, that's one
thing, but is this the ONLY reason SETUP_FINALLY is uninterruptible? This
seems improbable, especially since putting statements adjacent to #MARK will
break this code.

David

 

[Michael Hudson]

I would belive you, but there is a trivial modification that circumvents
this problem:

---begin---
mylock = some_lock()

try:
mylock.acquire()
something with mylock held
finally:
mylock.release()
---end---

Huh? If you typo the first mylock, or the acquire() call fails for
some reason, that doesn't do what you want.

Cheers,
mwh

 

[Tim Peters]

[David Pokorny]

Whilst swimming through Python/ceval.c, I came upon the following:

PyEval_EvalFrame(...) {
....
<main interpreter loop>
....
if (--_Py_Ticker < 0) {
if (*next_instr == SETUP_FINALLY) {
/* Make the last opcode before
a try: finally: block uninterruptable. */
goto fast_next_opcode;
}
_Py_Ticker = _Py_CheckInterval;
tstate->tick_counter++;
.... <do "occasional" things like let other threads run, ... occurs every
sys.getcheckinterval() bytecodes>
}

I've been trying to find a good reason _why_ the main loop shouldn't perform
the check at the usual time in the (highly rare) situation where SETUP_FINALLY
occurs as the last instruction before a check.
So far, I've managed to confuse myself even further and come up with another
stumper: what makes SETUP_FINALLY more special than SETUP_LOOP and
SETUP_EXCEPT?

Jeff Epler's answer was correct: it has to do with asynchronous
exceptions. However, it's extremely obscure, and deserves a much
better comment than it got (I won't hesitate much to rip that code
out, given that cryptic comment). It's also unclear (as developed
later in this message thread) exactly what it does and doesn't protect
against.

See:

http://mail.python.org/pipermail/python-dev/2003-June/036318.html

for the only explanation that appears to exist.

 

[David Pokorny]

Thanks! This has been very illuminating.

David