How to release the GIL from C?

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I am trying to release the GIL in a multi-threaded program (efforts
detailed below) without any success.

In the main thread during startup, I do the following:

Py_InitializeEx(0); /* Python shouldn't handle signals */
PyEval_InitThreads();

/* release the GIL */
... various pieces of code tried here ...

Note that no other threads exist yet when the code above is run.

After startup, there are around 30 threads and my handler routines do
work in them.

void handler(...)
{
PyGILState_STATE gilstate=PyGILState_Ensure();

... do the interesting stuff ...

PyGILState_Release(gilstate);
}

No matter what I try, the PyGILState_Ensure() hangs trying to acquire
the GIL:

#0 0x00002afd50ce243d in sem_wait () from /lib/libpthread.so.0
#1 0x00002afd5471557d in PyThread_acquire_lock () from
/usr/lib/libpython2.5.so.1.0
#2 0x00002afd546e44d4 in PyEval_RestoreThread () from
/usr/lib/libpython2.5.so.1.0
#3 0x00002afd5470bb8a in PyGILState_Ensure () from
/usr/lib/libpython2.5.so.1.0

Here are the various things I have tried to release the GIL (the
documentation says they all do, but that doesn't appear to be the case
:-( Unfortunately the doc is all geared around saving the interpreter
state and then later restoring it. I have nowhere to save it in the
main thread nor do I know when the main thread is having handlers
invoked in it again anyway in order to restore it.

1. (Based on Py_BEGIN_ALLOW_THREADS)
PyThreadState_Swap(NULL);
PyEval_ReleaseLock();

2. (Based on high level Py_BEGIN_ALLOW_THREADS)
PyEval_SaveThread();

3. Random code I found on the web
PyThreadState *tstate = PyThreadState_Get();
PyThreadState_Clear(tstate);
PyEval_ReleaseThread(tstate);

4. Something simple
PyEval_ReleaseLock();

5. Other code I found on the web
PyThreadState_Swap(NULL);

Does anyone know exactly which sequence of calls I should be making?

Thanks,

Roger

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGVKKSmOOfHg372QQRAi4VAJ9KO8z/WnSgPOlCjEOOJgDCf20nCwCeJHUE
kkhvMsy6p3qflvGKolttWTo=
=4Z8h
-----END PGP SIGNATURE-----

 

[Roger Binns]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I am trying to release the GIL in a multi-threaded program (efforts
detailed below) without any success.

The ultimate cause was that the program forked to go into daemon mode.
I had called PyOS_AfterFork() as the documents directed. What I hadn't
realised (and isn't documented) is that AfterFork() reinitialises the
Python threading state by making a new GIL and acquires it. The
acquiring bit meant none of my other threads could ever get it.

It is still unclear from the docs exactly which combination of functions
dealing with threadstate, threads and lock you need to call to just
unilaterally give up the GIL (ie when you don't have following block
that wants to reacquire the GIL)

Roger
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGVNW0mOOfHg372QQRAuyOAJ4pCCIK779XgvaUdKBtSa+nHElrHQCgiueP
n/0uMFCSH3SrQhMXdm2Jb/o=
=uh4D
-----END PGP SIGNATURE-----