The GIL, callbacks, and GPFs

[Francois De Serres]

Hello,

I'm chasing a GPF in the interpreter when running my extension module.
It's not very elaborated, but uses a system (threaded) callback, and
therefore the GIL.
Help would be mucho appreciated. Here's the rough picture:

win32_spam.c
------------
/* code here is unit-tested OK */

typedef struct Bag {
char data[128];
site_t size;
} Bag;
typedef void (*PCallback)(const Bag * const bag);

Bag bag;
PCallback user_callback = NULL;

/* called by windoz */
static void CALLBACK win32_call_me(void) {
memcpy(bag.data, somestuff, 100);
bag.size = 100;
SPAWN_THREAD(user_callback, & bag);//pseudo-code
}



spam_module.c
-------------
/* most of the code here is pasted from doc */

static PyObject * my_callback = NULL;

static PyObject *
setCallback(PyObject *dummy, PyObject *args) {
PyObject *result = NULL;
PyObject *temp;
if (PyArg_ParseTuple(args, "O:miSetCallback", &temp)) {
if ((temp != Py_None) && (!PyCallable_Check(temp))) {
PyErr_SetString(PyExc_TypeError, "parameter must be callable");
return NULL;
}
Py_XINCREF(temp);
Py_XDECREF(my_callback);
my_callback = temp;
Py_INCREF(Py_None);
result = Py_None;

/* set the actual callback in win32_spam.c */
user_callback = & external_callback;
}
return result;
}

static void external_callback(const Bag * const bag) {
if (my_callback && (my_callback != Py_None)) {
PyObject * arglist = NULL;
PyObject * result = NULL;
arglist = Py_BuildValue("(s#)", bag->data, bag->size);

/* do it */
PyGILState_STATE gil = PyGILState_Ensure();
result = PyEval_CallObject(my_callback, arglist);
PyGILState_Release(gil);

Py_DECREF(arglist);
Py_DECREF(result);
}
}


blowup_spam1.py
-------------
# This throws a GPF on callback.
# Why, since the GIL is acquired by the callback?

import spam.py

def callback(stuff):
print stuff

if __name__ == '__main__':
spam.setCallback(callback)
raw_input()


blowup_spam2.py
-------------
# This throws a GPF as well
# Why, since we're using a threadsafe queue?

import spam
import Queue

q = Queue.Queue()

def callback(stuff):
q.put(stuff)

if __name__ == '__main__':
spam.setCallback(callback)
while True:
print q.get()



nice_spam.py
-------------
# This works
# Why, since the rest did not?

import spam
import Queue
import threading

q = Queue.Queue()

def callback(stuff):
q.put(stuff)

def job():
while True:
print q.get()

if __name__ == '__main__':
spam.setCallback(callback)
t = threading.Thread(job)
t.start()
raw_input()



Please point me to what I'm doing wrong...

TIA,
Francois


PS: I've already submitted my issue under "(Win32 API) callback to
Python, threading hiccups", but I guess it was poorly formulated.

 

[Thomas Heller]

Hello,

I'm chasing a GPF in the interpreter when running my extension module.
It's not very elaborated, but uses a system (threaded) callback, and
therefore the GIL.
Help would be mucho appreciated. Here's the rough picture:

static void external_callback(const Bag * const bag) { if
(my_callback && (my_callback != Py_None)) {
PyObject * arglist = NULL;
PyObject * result = NULL;
arglist = Py_BuildValue("(s#)", bag->data, bag->size);
/* do it */
PyGILState_STATE gil = PyGILState_Ensure();

You're calling Py_BuildValue without holding the GIL. Not sure if
that's the reason for your problems, but I don't think its ok.

result = PyEval_CallObject(my_callback, arglist);
PyGILState_Release(gil);
Py_DECREF(arglist);
Py_DECREF(result); }
}

Thomas