Memory debugging tool for Python/C API?

[Robert Latest]

Hello,

extending Python in C ist just the coolest thing since sliced bread (and
I'm particularly happy because I really had started to miss C when I did
more and more things in Python).

I've got one question though. Tha C/API documentation is not quite clear
(to me, anyway) on what happens exactly to ressources between the
C/Python side. One example is this: I wrote a function that assembles a
string into a dynamically-allocated buffer, then creates a PyString from
it which it returns to the Python side:

PyObject *dyn_string(void)
{
char *buffer = malloc(20);
PyObject *pystr;
strcpy(buffer, "Hello, World");
pystr = PyString_FromString(buffer);
free(buffer);
return pystr;
}

This works, but I'm not sure if PyString...() really makes a new copy of
the data (ellowing me to use free()), or if this will segfault one day.
OTOH I wouldn't know how to safely dispose of the string data otherwise.

Another example (all on the C side of things) is a function where I
build a dictionary from a set of keys, putting PyNone into each value
(Py_INCREF()ing PyNone each time). At another point some of the values
are replaced by other PyObjects. At first I used PyDECREF() on each
value before setting the new value, but that botched up the dictionary
beyond repair. Without the PyDECREF() calls everything worked fine.

Now to my actual question: Is there some sort of debugging tool that I
could use to analyze my code to see if everything gets properly
allocated and deallocated and if the reference counting works right?

Thanks,
robert

 

[Martin v. Löwis]

This works, but I'm not sure if PyString...() really makes a new copy of

the data (ellowing me to use free())

See the documentation:

http://docs.python.org/c-api/string.html#PyString_FromString

# Return a new string object with a *copy* of the string v as value

Another example (all on the C side of things) is a function where I
build a dictionary from a set of keys, putting PyNone into each value
(Py_INCREF()ing PyNone each time).

Assuming you use PyDict_SetItem(String): you shouldn't INCREF Py_None;
the SetItem will itself incref the value (and decref any old value
that might be already stored under that key).

The only exception where you, as a caller, have to INCREF is when the
reference is documented as "stolen"; very few API functions steal
references.

At another point some of the values
are replaced by other PyObjects. At first I used PyDECREF() on each
value before setting the new value, but that botched up the dictionary
beyond repair. Without the PyDECREF() calls everything worked fine.

See above.

Now to my actual question: Is there some sort of debugging tool that I
could use to analyze my code to see if everything gets properly
allocated and deallocated and if the reference counting works right?

You should compile Python in pydebug mode; this will perform some
additional checks each time. In addition, you can look at
sys.getrefcount repeatedly. If you create a test that allocates stuff
and then releases it, then sys.getrefcount should be unchanged. If
it does change, you might have a reference leak.

Regards,
Martin