distutils and ctypes

[jtravs]

Hi all,

I suspect that I'm doing something stupid, I would like some other
opinions though.
I'm getting started with ctypes and am trying to use distutils to help
build my module. At the moment I simply want distutils to build a
shared c library (not a python extension!). Under linux, the following
works, under windows xp id doesn't (which I guess is obvious, but the
linux success lead me on).
I have two files at the moment...

/* begin test.c */
int test(int i)
{
return i*i;
}
/* end test.c */

#begin setup.py
from distutils.core import setup, Extension
setup(name="test", version="0.0", ext_modules = [Extension("test",
["test.c"])])
# end setup.py

If I run:

python setup.py build

under linux, I get a nice shared c library under my build dir, which
can be imported by ctypes.
If I run it under windows I get the following:

running build
running build_ext
building 'test' extension
C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\link.exe
/DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Python25\libs
/LIBPATH:C:\Python25\PCBuild /EXPORT:inittest
build\temp.win32-2.5\Release\test.obj /OUT:build\lib.win32-2.5\test.pyd
/IMPLIB:build\temp.win32-2.5\Release\test.lib
LINK : error LNK2001: unresolved external symbol inittest
build\temp.win32-2.5\Release\test.lib : fatal error LNK1120: 1
unresolved externals
LINK : fatal error LNK1141: failure during build of exports file

I can see the problem: python is trying to build an extension module
and is telling the linker to export "inittest", which doesn't exist.
This didn't happen under linux as you don't need to export interfaces
in shared c libraries on linux.

So finally, my question is, is there a way to get distutils to simply
build a shared library on windows so that I can use ctypes with them???

Thanks for your patience with this post, and thanks for any replies.
Best regards,
John Travers

 

[Robert Kern]

So finally, my question is, is there a way to get distutils to simply
build a shared library on windows so that I can use ctypes with them???

Not out-of-box, no. The OOF2 project has added a bdist_shlib command which
should do most of what you want, though. It's somewhat UNIX-oriented, and I
think it tries to install the shared library to a standard location (e.g.
/usr/local/lib). You might want to modify it to install the shared library in
the package so it is easy to locate at runtime.

http://www.ctcms.nist.gov/oof/oof2/
http://www.ctcms.nist.gov/oof/oof2/source/oof2-2.0.1.tar.gz

The code is in the shlib/ subdirectory.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

Not out-of-box, no. The OOF2 project has added a bdist_shlib command which
should do most of what you want, though. It's somewhat UNIX-oriented, and I
think it tries to install the shared library to a standard location (e.g.
/usr/local/lib). You might want to modify it to install the shared library in
the package so it is easy to locate at runtime.

http://www.ctcms.nist.gov/oof/oof2/
http://www.ctcms.nist.gov/oof/oof2/source/oof2-2.0.1.tar.gz

The code is in the shlib/ subdirectory.

And if you do so, please let us know about it! This would be quite useful for
many other ctypes-using projects.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I suspect that I'm doing something stupid, I would like some other
opinions though.
I'm getting started with ctypes and am trying to use distutils to help
build my module. At the moment I simply want distutils to build a
shared c library (not a python extension!). Under linux, the following
works, under windows xp id doesn't (which I guess is obvious, but the
linux success lead me on).

Not sure it's stupid, but I wonder why you want to use ctypes. What's
wrong with extension modules?

Regards,
Martin

 

[Robert Kern]

Not sure it's stupid, but I wonder why you want to use ctypes. What's
wrong with extension modules?

What's wrong with ctypes? They're both valid, useful approaches to connect to C
libraries.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Guest]

What's wrong with ctypes? They're both valid, useful approaches to connect to C
libraries.

See the original posting. Distutils doesn't support building arbitrary
shared libraries, but does support building extension modules.

Furthermore, extension modules are more type-safe and more expressive
than loading shared libraries through ctypes. IMO, you may consider
using ctypes as a last resort - if you have the chance for a
well-engineered solution, write a compiled wrapper.

Regards,
Martin

 

[Robert Kern]

See the original posting. Distutils doesn't support building arbitrary
shared libraries, but does support building extension modules.

Furthermore, extension modules are more type-safe and more expressive
than loading shared libraries through ctypes. IMO, you may consider
using ctypes as a last resort - if you have the chance for a
well-engineered solution, write a compiled wrapper.

To which I say that doing the type-checking and error handling is much easier in
Python than using the C API. Add to that the tediousness of the edit-compile-run
cycle of C and the finickiness of refcounting.

There's nothing *wrong* with either approach. They're just different and have
different strengths and weaknesses. Some of those weaknesses are remediable (say
by using the bdist_shlib command that the OOF2 project implemented) and some aren't.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Guest]

To which I say that doing the type-checking and error handling is much easier in
Python than using the C API. Add to that the tediousness of the edit-compile-run
cycle of C and the finickiness of refcounting.

Sure: edit-compile-run is tedious, but in the given case, it is
necessary anyway. Also, refcounting is finicky, but with ctypes,
you often have to use just as finicky memory management APIs.
(such as using specific allocation and deallocation routines,
or having to do memory management at all when in C you would
do stack allocation).

My main concern with ctypes is that you have to duplicate
information from the header files, which is error-prone,
especially when the header files may change (either across
versions or across target systems).

Regards,
Martin

 

[jtravs]

Not out-of-box, no. The OOF2 project has added a bdist_shlib command which
should do most of what you want, though. It's somewhat UNIX-oriented, and I
think it tries to install the shared library to a standard location (e.g.
/usr/local/lib). You might want to modify it to install the shared library in
the package so it is easy to locate at runtime.

http://www.ctcms.nist.gov/oof/oof2/
http://www.ctcms.nist.gov/oof/oof2/source/oof2-2.0.1.tar.gz

The code is in the shlib/ subdirectory.

Thank you very much - this looks like exactly what I want.
John

 

[jtravs]

Sure: edit-compile-run is tedious, but in the given case, it is
necessary anyway. Also, refcounting is finicky, but with ctypes,
you often have to use just as finicky memory management APIs.
(such as using specific allocation and deallocation routines,
or having to do memory management at all when in C you would
do stack allocation).

My main concern with ctypes is that you have to duplicate
information from the header files, which is error-prone,
especially when the header files may change (either across
versions or across target systems).

I have looked at: building an extension module, using pyrex and using
ctypes. Initially ctypes won because I was under the impression that to
use pyrex or build an extension module required the same compiler as
python was compiled with. This is not a problem on linux, but under
windows this is much too great a problem for potential users (who will
also need to compile the module). I now have discovered that I can use
mingw32 (without patching) under windows (with python2.5) though I'm
not clear if this is by chance or has been implemented as a new feature
(can't find suitable documentation).

Anyway, at this point I think I will stick with ctypes purely out of
simplicity. My c library is for some legacy lab hardware control. I
only need to call three functions. With ctypes, I can check the data in
python (which I do anyway to validate it) and pass it onto the library
in about 10 lines of code. Writing an extension module I believe would
be much more difficult.

Anyway,
Thanks both for your help.
John