Cyclic garbage collection and segfaults...

[Thomas Mailund]

Hi group.

I have a problem with some C extensions I am working with and
hope that some of you can help. Basically, I am wrapping a a tree
structure from C where I have python methods for extracting either the
entire tree or subtrees; since I don't want the full tree to be
deallocated while python has references to the subtrees, I INCREF the
full tree whenever I hand out a subtree reference. I don't want any of
the subtrees to be deallocated while the full tree lives, either, so I
want the fulltree to have a reference to each of the subtrees. Naturally,
this gives me a cyclik reference structure, and I want to be able to
garbage collect it. But this is where my problems begin...

For some reason, when I turn on the cyclic garbage collection for my
types, a deallocation automatically gives me a segfault. That is, if my
type new type contains any data whatsoever, I cannot garbage collect
without dumping core. I have boiled the problem down to the code shown
below. It contains no cyclic structure, it is as simple as it gets, but
it still seqfaults for me. If I comment out the magic void-pointer (which
isn't used for anything) I don't get the segfault.


#include <Python.h>

struct SimpleObject;

static PyObject * Simple_new (PyTypeObject *type,
PyObject *args,
PyObject *kwds);
static int Simple_init (struct SimpleObject *self,
PyObject *args,
PyObject *kwds);
static int Simple_traverse(struct SimpleObject *self,
visitproc visit,
void *arg);
static void Simple_dealloc (struct SimpleObject *self);




typedef struct SimpleObject {
/* object stuff */
PyObject_HEAD

/* dark magic */
void *magic;

} SimpleObject;


static PyTypeObject simple_SimpleType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"simple.Simple", /*tp_name*/
sizeof(SimpleObject), /*tp_basicsize*/
0, /*tp_itemsize*/
(destructor)Simple_dealloc, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /*tp_flags*/
"Simple objects", /*tp_doc*/
(traverseproc)Simple_traverse, /*tp_traverse*/
0, /*tp_clear*/
0, /*tp_richcompare*/
0, /*tp_weaklistoffset*/
0, /*tp_iter*/
0, /*tp_iternext*/
0, /*tp_methods*/
0, /*tp_members*/
0, /*tp_getset*/
0, /*tp_base*/
0, /*tp_dict*/
0, /*tp_descr_get*/
0, /*tp_descr_set*/
0, /*tp_dictoffset*/
(initproc)Simple_init, /*tp_init*/
0, /*tp_alloc*/
Simple_new, /*tp_new*/

0 /*...the rest...*/
};


static PyObject *
Simple_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
SimpleObject *self = (SimpleObject *)type->tp_alloc(type, 0);
return (PyObject*)self;
}

static int
Simple_init(SimpleObject *self, PyObject *args, PyObject *kwds)
{
return 0;
}

/* cyclic gc */
static int
Simple_traverse (SimpleObject *self, visitproc visit, void *arg)
{
fprintf(stderr,"Simple_traverse...\n");
return 0;
}

static int
Simple_clear(SimpleObject *self)
{
fprintf(stderr,"Simple_clear...\n");
return 0;
}

static void
Simple_dealloc(SimpleObject *self)
{
fprintf(stderr,"Simple_dealloc %p\n", self);
self->ob_type->tp_free((PyObject*)self); /* <= segfault here */
return;
}



static PyMethodDef simple_methods[] = {
{0} /* sentinel */
};



void
initsimple(void)
{
if (PyType_Ready(&simple_SimpleType) < 0) return;
Py_INCREF(&simple_SimpleType);

PyObject* m = Py_InitModule3("simple", simple_methods, "Simple module.");
PyModule_AddObject(m,"Simple",(PyObject*)&simple_SimpleType);
}


A script sufficient to provoke the fault is this:

import simple
simple.Simple()


Can anyone explain what I'm doing wrong? Or perhaps suggest a better
solution to my "real" problem, if I'm approaching the problem completely
wrong

Yours,
/mailund

 

[Simon Burton]

Can anyone explain what I'm doing wrong? Or perhaps suggest a better
solution to my "real" problem, if I'm approaching the problem completely
wrong

Yours,
/mailund

In my extensions python seems to segfault in the GC most of all...
Just yesterday I removed a malloc that was (why??) causing a segfault.

Have you seen pyrex ? It's absolutely brilliant. Take 2 hours to try it
out if you have the time.

Simon.

 

[Michael Hudson]

Hi group.

I have a problem with some C extensions I am working with and
hope that some of you can help.
[snippety]

static void
Simple_dealloc(SimpleObject *self)
{
fprintf(stderr,"Simple_dealloc %p\n", self);
self->ob_type->tp_free((PyObject*)self); /* <= segfault here */

Well, you're calling tp_free from a tp_dealloc. That doesn't *sound*
sensible to me.

Can anyone explain what I'm doing wrong? Or perhaps suggest a better
solution to my "real" problem, if I'm approaching the problem completely
wrong

There are docs on this sort of thing.

Cheers,
mwh

 

[Thomas Mailund]

Hi group.

I have a problem with some C extensions I am working with and
hope that some of you can help.
[snippety]

static void
Simple_dealloc(SimpleObject *self)
{
fprintf(stderr,"Simple_dealloc %p\n", self);
self->ob_type->tp_free((PyObject*)self); /* <= segfault here */

Well, you're calling tp_free from a tp_dealloc. That doesn't *sound*
sensible to me.

I'm suprised to hear that; the documentation I was working from,
<URL:http://www.python.org/doc/current/ext/node22.html>, does exactly
that. Of course, this is from the section that uses reference counting,
not cyclic gc, but later on, when the garbage collector is introduced, the
deallocator still calls tp_free, it just calls clear first
<URL:http://www.python.org/doc/current/ext/node24.html>.

If I shouldn't free self in this way, how should I do it?

There are docs on this sort of thing.

About the garbage collection? In that case, I thought I *was* following
the documentation ;-)

If about the general problem with accessing the whole and parts of a C
structure from python, if you have any specific references in mind, I
would be very grateful if you would post them. I know that I am not the
first to have this problem, and that very likely there are general
patterns for solving the problem, but my googling didn't find anything
(which probably means I wasn't asking the right questions, but never the
less...)

Yours,
/mailund