Reentrancy of Python interpreter

[Brad Johnson]

I have embedded a single threaded instance of the Python interpreter in my
application.

I have a place where I execute a Python command that calls into C++ code which
then in turn calls back into Python using the same interpreter. I get a fatal
error which is "PyThreadStage_Get: no current thread."

I guess I understand why I can't have two invocations of the same interpreter
thread in one call stack, but how would I go about solving this?

 

[bvukov]

I have embedded a single threaded instance of the Python interpreter in my
application.

I have a place where I execute a Python command that calls into C++ code which
then in turn calls back into Python using the same interpreter. I get a fatal
error which is "PyThreadStage_Get: no current thread."

I guess I understand why I can't have two invocations of the same interpreter
thread in one call stack, but how would I go about solving this?

Looks like ( from PyThreadStage_Get error ) that you lost the GIL. You
probably
entered some C++ code and encapsulated you're work in the

Py_BEGIN_ALLOW_THREADS
<code>
Py_END_ALLOW_THREADS

but you're <code> is calling back the Python function, and you forgot
to acquire
back the GIL.

 

[Brad Johnson]

Looks like ( from PyThreadStage_Get error ) that you lost the GIL. You
probably
entered some C++ code and encapsulated you're work in the

Py_BEGIN_ALLOW_THREADS
<code>
Py_END_ALLOW_THREADS

but you're <code> is calling back the Python function, and you forgot
to acquire
back the GIL.

I didn't explicitly allow for threads in Python. It should just be single
threaded. So I don't understand why I should have to acquire the GIL for the
second call into the interpreter, or how I lost the GIL in the first place.

Maybe I just need the second call to be placed into its own thread...?

 

[Hrvoje Niksic]

I have a place where I execute a Python command that calls into C++
code which then in turn calls back into Python using the same
interpreter. I get a fatal error which is "PyThreadStage_Get: no
current thread."

Does the C++ code call into the interpreter from a different thread?

 

[Brad Johnson]

Does the C++ code call into the interpreter from a different thread?

No.

But, problem has been solved. I enabled thread support for the Python
interpreter and now wrap the code that caused the problem in PyGILState_Ensure
and PyGILState_Release. I'm not sure why this fixed it. My theory is that the
Python interpreter is not reentrant and requires another thread if you call into
it more than once in the same call chain.