C extension - new and init functions

[Paul Moore]

I have a C extension type, "Pattern", where my type manages a
dynamically allocated memory block (of "instructions", the details
aren't important).

The basic constructor builds a pattern with a single instruction
("End") as follows:

static PyObject *
Pattern_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Pattern *self;

self = (Pattern *)type->tp_alloc(type, 0);
self->prog = NULL;
return (PyObject *)self;
}

static int
Pattern_init(Pattern *self, PyObject *args, PyObject *kwds)
{
self->prog = newpatt(0);
if (self->prog == NULL)
return -1;

return 0;
}

This is fine, but in a lot of cases I want to allocate a Pattern with
a larger buffer (which will be filled in C before being passed back to
Python code). I want to allow the type to be subclassed, so I believe
that any allocation I do in C should be by calling the type
constructor - so my code to build instances should have the form (for
a Pattern classmethod):

static PyObject *
Pattern_XXX(PyObject *cls, PyObject *arg)
{
...

result = PyObject_CallFunction(cls, "");
if (result == NULL)
return NULL;

...
}

OK, but that allocates the initial buffer, which I will then throw
away. In practice, this isn't a significant issue (the overhead of one
allocation and one deallocation isn't going to wreck things!) but it
feels somehow clumsy. However, I can't see an obvious way to refactor
the code so that building the object and assigning the buffer are
separate - so that I can write a function like Pattern_init, which
allocates a differently sized buffer (well, I could, but if I did, I'd
have no way of using it that respects subclassing...

Can anybody offer me any suggestions on how to code this (or
confirmation that there's no way, and I should just live with the
unwanted allocation)?

Thanks,
Paul.

 

[Hrvoje Niksic]

OK, but that allocates the initial buffer, which I will then throw
away.

Why don't you use a constructor argument that allows specifying the
initial buffer size?

In practice, this isn't a significant issue (the overhead of one
allocation and one deallocation isn't going to wreck things!) but it
feels somehow clumsy. However, I can't see an obvious way to
refactor the code so that building the object and assigning the
buffer are separate - so that I can write a function like
Pattern_init, which allocates a differently sized buffer (well, I
could, but if I did, I'd have no way of using it that respects
subclassing...

Another way would be to avoid allocating the buffer at all until it is
actually needed.

 

[Paul Moore]

Why don't you use a constructor argument that allows specifying
the initial buffer size?

That's an option, but I don't want to expose that argument to Python
code (as it would allow the user to create an object which wasn't
fully initialised - I only use the extra argument myself when there's
additional initialisation to do in C).

Another way would be to avoid allocating the buffer at all
until it is actually needed.

The problem then is that I have to include null checks in a lot of
code that can otherwise assume that objects passed in have a valid
buffer.

Thinking it through, I think the extra dealloc/alloc and minor code
duplication probably isn't enough of an issue to compromise safety
over.

Thanks,
Paul.