Passing python list from C to python

[hartley]

I'm very new at wrapping Python/C, and I have run into some problems.

I have one python module that provides me with a list (provideBuffer
in provideBuff.py):

Py_Initialize();
pName = PyString_FromString("provideBuff");
pModule = PyImport_Import(pName);

pFunc = PyObject_GetAttrString(pModule,"provideBuffer");

pValue = PyObject_CallObject(pFunc,NULL);

pValue is now a PyList - i've even verified this with:

int a = PyList_Check(pValue);
printf("%d\n", a);

However, I want to send this PyList to another python module, but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.

pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");
pValue2 = PyObject_CallObject(pFunc2,pValue);

pValue2 is now False! So i guess i cannot pass pValue as an argument
to PyObject_CallObject when i want to pass an python list as an
argument. But how must a go about to make this work?

 

[John Machin]

I'm very new at wrapping Python/C, and I have run into some problems.

I have one python module that provides me with a list (provideBuffer
in provideBuff.py):

 Py_Initialize();
        pName = PyString_FromString("provideBuff");
        pModule = PyImport_Import(pName);

        pFunc = PyObject_GetAttrString(pModule,"provideBuffer");

        pValue = PyObject_CallObject(pFunc,NULL);

pValue is now a PyList - i've even verified this with:

int a = PyList_Check(pValue);
        printf("%d\n", a);

However, I want to send this PyList to another python module,

Please explain "send" ... do you mean the C equivalent of the Python
statement C_embedding.buff = the_pylist ?

BTW C-embedding would trigger a syntax error in Python source; best to
avoid ...

but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.

pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");

Get?? Do you want Set? Is buff a Python function? Or is it the
destination of the "sending"? Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]

pValue2 = PyObject_CallObject(pFunc2,pValue);

CallObject?? You used this before because you had a function and
wanted to call it because it returned you a value .... now you want to
do one of (in Python terms)

value = amodule.anattr
value = getattr(amodule, "anattr")

or

amodule.anattr = value
setattr(amodule, "anattr", value)

pValue2 is now False!

False?? Do you mean the C NULL?

So i guess i cannot pass pValue as an argument
to PyObject_CallObject when i want to pass an python list as an
argument. But how must a go about to make this work?

It's mainly just a matter of (1) knowing what you want to do (2)
picking the API that does what you want (3) checking the returned
value for error after every call.

HTH,
John

 

[hartley]

I'm very new at wrapping Python/C, and I have run into some problems.

Please explain "send" ... do you mean the C equivalent of the Python
statement C_embedding.buff = the_pylist ?

BTW C-embedding would trigger a syntax error in Python source; best to
avoid ...

I'm sorry I'm not using the proper terms when trying to describe this
- i've never really learnt those terms, I guess i should do that one
day.


Imagine that the python function C-embedding.buff looks like this:

def buff(a):
if isinstance(a,list):
print "success"

I want to send, pass, call, whatever a list argument from the C code
onto this function. As simple as this - how can you call a function in
python from C, and providing a python list as an argument?

As I wrote earlier - I already have a PyList, called pValue. But I
have not been able to use this as an argument for C-embedding.buff
using the code I described last time.

but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.

pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");

Get?? Do you want Set? Is buff a Python function? Or is it the
destination of the "sending"? Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]

pValue2 = PyObject_CallObject(pFunc2,pValue);

CallObject?? You used this before because you had a function and
wanted to call it because it returned you a value .... now you want to
do one of (in Python terms)

value = amodule.anattr
value = getattr(amodule, "anattr")

or

amodule.anattr = value
setattr(amodule, "anattr", value)

pValue2 is now False!

False?? Do you mean the C NULL?

So i guess i cannot pass pValue as an argument
to PyObject_CallObject when i want to pass an python list as an
argument. But how must a go about to make this work?

It's mainly just a matter of (1) knowing what you want to do (2)
picking the API that does what you want (3) checking the returned
value for error after every call.

HTH,
John

 

[hartley]

I'm very new at wrapping Python/C, and I have run into some problems.

I have one python module that provides me with a list (provideBuffer
in provideBuff.py):

 Py_Initialize();
        pName = PyString_FromString("provideBuff");
        pModule = PyImport_Import(pName);

        pFunc = PyObject_GetAttrString(pModule,"provideBuffer");

        pValue = PyObject_CallObject(pFunc,NULL);

pValue is now a PyList - i've even verified this with:

int a = PyList_Check(pValue);
        printf("%d\n", a);

However, I want to send this PyList to another python module,

Please explain "send" ... do you mean the C equivalent of the Python
statement C_embedding.buff = the_pylist ?

BTW C-embedding would trigger a syntax error in Python source; best to
avoid ...

but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.

pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");

Get?? Do you want Set? Is buff a Python function? Or is it the
destination of the "sending"? Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]

pValue2 = PyObject_CallObject(pFunc2,pValue);

CallObject?? You used this before because you had a function and
wanted to call it because it returned you a value .... now you want to
do one of (in Python terms)

value = amodule.anattr
value = getattr(amodule, "anattr")

or

amodule.anattr = value
setattr(amodule, "anattr", value)

pValue2 is now False!

False?? Do you mean the C NULL?

So i guess i cannot pass pValue as an argument
to PyObject_CallObject when i want to pass an python list as an
argument. But how must a go about to make this work?

It's mainly just a matter of (1) knowing what you want to do (2)
picking the API that does what you want (3) checking the returned
value for error after every call.

HTH,
John

I tried to keep this simple, but I realise that since I'm not using
the terminology correctly, it may be hard understand what I mean.

I'll do something even simpler:

Say you have the function:

C-embedding.buff

def buff(s):
if isinstance(s,list):
return s

how can i call (use,invoke,?? - sorry, about the terminology again)
this function from C so that I call this function with an argument
that is (accepted as) a python list?

foo.bar

def bar():


pName = PyString_FromString("foo");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule,"bar");
pValue = PyObject_CallObject(pFunc,NULL);

 

[John Machin]

[snip]

Please explain "send" ... do you mean the C equivalent of the Python
statement C_embedding.buff = the_pylist ?

BTW C-embedding would trigger a syntax error in Python source; best to
avoid ...

I'm sorry I'm not using the proper terms when trying to describe this
- i've never really learnt those terms, I guess i should do that one
day.

Today would be a good day to start

Imagine that the python function C-embedding.buff looks like this:

def buff(a):
    if isinstance(a,list):
        print "success"

I want to send, pass, call, whatever a list argument from the C code
onto this function. As simple as this - how can you call a function in
python from C, and providing a python list as an argument?

As I wrote earlier - I already have a PyList, called pValue. But I
have not been able to use this as an argument for C-embedding.buff
using the code I described last time.

but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.
pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");

Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]

pValue2 = PyObject_CallObject(pFunc2,pValue);

Your problem is here, in the second arg of PyObject_CallObject. It
should be either NULL if no args are being supplied, or a tuple
containing the args if 1 or more args are being supplied. pValue is
the arg itself; you need to wrap it in a tuple.

See http://docs.python.org/c-api/object.html#PyObject_CallObject

Actually you might like to use this instead (no tuple hassles):

http://docs.python.org/c-api/object.html#PyObject_CallFunctionObjArgs

What are you using as a source of information? Have you read this:

http://docs.python.org/extending/embedding.html#pure-embedding ?

It contains a very thorough example of calling a Python function from
C, with all the reference-counting and error-checking stuff and it's
quite runnable -- I've just now compiled it and run it (1) as-is (2)
changed to pass a list instead of multiple ints (3) changed again to
emulate you trying to pass the list directly instead of wrapped in a
tuple ... here's the result:

TypeError: argument list must be a tuple
Call failed

Oh by the way, you can get away with C-embedding as a module name if
you are importing it from C, but importing it from Python [so that you
could test it independently of your C program] presents a difficulty
(syntax error) ... better to change it to c_embedding.

HTH,
John

 

[hartley]

[snip]

I'm sorry I'm not using the proper terms when trying to describe this
- i've never really learnt those terms, I guess i should do that one
day.

Today would be a good day to start

Imagine that the python function C-embedding.buff looks like this:

def buff(a):
    if isinstance(a,list):
        print "success"

I want to send, pass, call, whatever a list argument from the C code
onto this function. As simple as this - how can you call a function in
python from C, and providing a python list as an argument?

As I wrote earlier - I already have a PyList, called pValue. But I
have not been able to use this as an argument for C-embedding.buff
using the code I described last time.

but I
don't know how to do this. Initially I though I could just do like
above, only swapping NULL with pValue, but that is not working.
pName2 = PyString_FromString("C-embedding");
pModule2 = PyImport_Import(pName2);
pFunc2 = PyObject_GetAttrString(pModule2,"buff");
 Any good reason for not checking the
return value for an error? [Rhetorical question; answer == "No"]
pValue2 = PyObject_CallObject(pFunc2,pValue);

Your problem is here, in the second arg of PyObject_CallObject. It
should be either NULL if no args are being supplied, or a tuple
containing the args if 1 or more args are being supplied. pValue is
the arg itself; you need to wrap it in a tuple.

Seehttp://docs.python.org/c-api/object.html#PyObject_CallObject

Actually you might like to use this instead (no tuple hassles):

http://docs.python.org/c-api/object.html#PyObject_CallFunctionObjArgs

What are you using as a source of information? Have you read this:

http://docs.python.org/extending/embedding.html#pure-embedding?

It contains a very thorough example of calling a Python function from
C, with all the reference-counting and error-checking stuff and it's
quite runnable -- I've just now compiled it and run it (1) as-is (2)
changed to pass a list instead of multiple ints (3) changed again to
emulate you trying to pass the list directly instead of wrapped in a
tuple ... here's the result:

TypeError: argument list must be a tuple
Call failed

Oh by the way, you can get away with C-embedding as a module name if
you are importing it from C, but importing it from Python [so that you
could test it independently of your C program] presents a difficulty
(syntax error) ... better to change it to c_embedding.

HTH,
John

Thank you, that works! You're a genius

As for what I'm reading, I'm familiar with both


http://docs.python.org/c-api/
and
http://docs.python.org/extending/embedding.html#pure-embedding

but i still find it hard to read this and find the proper API. The
Python/C API reference is 162 pages!

I have a second issue that I've also struggled with, namely using
class object (don't shoot me if my terminology is off again). With the
little toy class below, i want to instansiate it, boost it, and return
it, similar to:

s = ObjectTest()
s.boostList()
return s.dispList()

but in C, of course.

so far I have:

pName = PyString_FromString("ob-test");
pModule = PyImport_Import(pName);
pFunc = PyObject_GetAttrString(pModule,"ObjectTest");
if (pFunc && PyCallable_Check(pFunc)) {

printf("Callable\n");
}
pObj = PyObject_CallObject(pFunc,NULL);
if (pObj != NULL) {
printf("WAS NOT NULL!\n");
}

At least this seems to work. However, as to how to actually
instantiate and use this class the way I described, I have no idea
which API to use. Even though I've spent lots of time reading it! Do
you have any clue here, as well?



(ob-test.py)
class ObjectTest:

def __init__(self):
self.list = []


def boostList(self):
self.list.append(1)

def dispList(self):
return self.list

 

[John Machin]

[snip]

/* the first telling */
/* the second telling */

Oh by the way, you can get away with C-embedding as a module name if
you are importing it from C, but importing it from Python [so that you
could test it independently of your C program] presents a difficulty
(syntax error) ... better to change it to c_embedding.

As for what I'm reading, I'm familiar with both

http://docs.python.org/c-api/
andhttp://docs.python.org/extending/embedding.html#pure-embedding

You must have a very strange definition of familiarity! That section
has as I pointed out a complete working example of a C program that
calls a Python function with 1 or more arguments, complete with error
checking and reference counting. To me, familiarity would include
reading that program, understanding what it is doing (there are
copious explanations), compiling it, running it, then start making
changes (like I did) to do things like what you want to do.

but i still find it hard to read this and find the proper API. The
Python/C API reference is 162 pages!

It's a reference, not a novel. You needed no API that was not in the
sample program. What you should do is when you are going through the
example programs, look up each API in the reference manual to see what
the arguments are supposed to be, what other possibilities there are
(e.g. PyObject_CallObject 2nd arg is NULL or a tuple of your args),
what the return value is, what if anything is said about reference
counts, what similar APIs are there (e.g. a few called
PyObject_Callxxxxxxx)

I have a second issue that I've also struggled with, namely using
class object (don't shoot me if my terminology is off again). With the
little toy class below, i want to instansiate it, boost it, and return
it, similar to:

s = ObjectTest()
s.boostList()
return s.dispList()

Let's break that down:

alist = s.dispList()
return alist

s.dispList (despite its name indicating "display") returns a reference
to a list. Your C program is going to return that to whom? The shell?

but in C, of course.

so far I have:

pName = PyString_FromString("ob-test");

/* the third telling */
s/-/_/

"What I tell you three times is true". Just do it.

pModule = PyImport_Import(pName);

You've imported the module. So far, I'm making the charitable
assumption that the lack of error checking is for brevity in your
posting, but is there in your actual code.

pFunc = PyObject_GetAttrString(pModule,"ObjectTest");

You've obtained a reference to the class.

if (pFunc && PyCallable_Check(pFunc)) {

            printf("Callable\n");
                }

And if there's a problem, you'll fall through and use the dud
reference to the class??

Get into the habit NOW of doing the error checking properly, after
each API call; otherwise debugging will become a nightmare. Follow the
examples in the sample program. In this case, it is necessary only to
test for NULL, and get the correct error message displayed for that.
Checking if the returned object is callable in the same if-test clouds
the issue. Checking for callability separately is quite unnecessary --
just call the object and you'll find out then if it's not callable.

pObj = PyObject_CallObject(pFunc,NULL);
        if (pObj != NULL) {
        printf("WAS NOT NULL!\n");

Again instead of printing congratulatory messages on success, put in
proper checking for failure, get the error message printed and bale
out.

}

At least this seems to work. However, as to how to actually
instantiate and use this class the way I described, I have no idea
which API to use.

You don't know how to instantiate the class? What do you think was the
purpose of the PyObject_CallObject() that you just did??

Even though I've spent lots of time reading it! Do
you have any clue here, as well?

Let's trawl through the Python equivalent of what you want to do:

s = amodule.ObjectTest()
(1) import the module
(2) get the "objectTest" attribute of the module
(3) call it, park the result in s

s.boostList()
(1) get the "boostList" attribute of s
(2) call it ... but it's a method, maybe we need to use a different
PyObject_Callxxxxxx ... no result to park.

x = s.dispList()
(1) get the "displist" attribute of s
(2) call it (see above), park result in x
(3) do something sensible with x

(ob-test.py)
class ObjectTest:

        def __init__(self):
                self.list = []

        def boostList(self):
                self.list.append(1)

        def dispList(self):
                return self.list

HTH,
John

 

[hartley]

/* the first telling */
(...)
/* the third telling */
(...)
/* the third telling */


-
If you had loosened up on the sarcasm I would probably have read what
you wrote more thoroughly instead of just skimming through it. Thanks
for the help, but you should seriously consider doing something with
your patronizing attitude.

HTH,
Hartley

 

[hartley]

/* the first telling */
(...)
/* the second telling */
(...)
/* the third telling */

-
If you had loosened up on the sarcasm I would probably have read what
you wrote more thoroughly instead of just skimming through it. Thanks
for the help, but you should seriously consider doing something with
your patronizing attitude.

HTH,
Hartley

 

[John Machin]

/* the first telling */
(...)
/* the second telling */
(...)
/* the third telling */

-
If you had loosened up on the sarcasm

That wasn't sarcasm, it was an attempt at humourous watering-down the
expression of exasperation at continued ignoring of advice and wonder
that I was continuing to bother ...

HTH,

It didn't.

 

[Aahz]

If you had loosened up on the sarcasm I would probably have read what
you wrote more thoroughly instead of just skimming through it. Thanks
for the help, but you should seriously consider doing something with
your patronizing attitude.

http://www.mikeash.com/getting_answers.html
http://www.catb.org/~esr/faqs/smart-questions.html

 

[hartley]

http://www.mikeash.com/getting_answers.htmlhttp://www.catb.org/~esr/faqs/smart-questions.html

From "http://www.mikeash.com/getting_answers.html":

How To Ask Questions The Smart Way is pretty condescending, especially
if you already feel insulted by somebody telling you that you need
help asking questions. If you're pointed at a guide with a filename of
smart-questions, that means this person thinks you have stupid
questions, and who needs that?

Cheers mate

-Hartley

 

[Aahz]

smart-questions.html

From "http://www.mikeash.com/getting_answers.html":

How To Ask Questions The Smart Way is pretty condescending, especially
if you already feel insulted by somebody telling you that you need
help asking questions. If you're pointed at a guide with a filename of
smart-questions, that means this person thinks you have stupid
questions, and who needs that?

Cheers mate

Not really. Feel free to chide people trying to help you, eventually
people won't care. <shrug>