python extension modules within packages not loading

[tyler]

I've written a small python extension but I'm having difficulty loading
it at runtime. The source for my extension is a module which is a
member of a package is organized as follows.

test/setup.py
test/myutils/__init__.py
test/myutils/netmodule.c

my setup.py file for building / installing looks like this

setup(ext_package = 'myutils',
ext_modules = [ Extension('net', sources = [ 'myutils/netmodule.c'
]) ],
packages = [ 'myutils'])

as per the faq I've been building and installing it with

python setup.py build
python setup.py install

which results in the following being installed into my site-packages
directory.

/usr/lib/python2.4/site-packages/myutils/__init__.py
/usr/lib/python2.4/site-packages/myutils/__init__.pyc
/usr/lib/python2.4/site-packages/myutils/net.so

when I run the interpreter and try to import the modules however I get
no joy.
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ImportError: No module named net

now annoyed I strace the interpreter during the loading of the module
and see.

stat64("myutils", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("myutils/__init__.py", {st_mode=S_IFREG|0644, st_size=0, ...}) =
0
stat64("myutils/__init__", 0xbf8d3a6c) = -1 ENOENT (No such file or
directory)
open("myutils/__init__.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or directory)
open("myutils/__init__module.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No
such file or directory)
open("myutils/__init__.py", O_RDONLY|O_LARGEFILE) = 3

clearly the above corresponds to my first import statement and it
appears to have been successful. A little further along in the trace
however I see the following.

stat64("myutils", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat64("myutils/net", 0xbf8d3ebc) = -1 ENOENT (No such file or
directory)
open("myutils/net.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file
or directory)
open("myutils/netmodule.so", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or directory)
open("myutils/net.py", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file
or directory)
open("myutils/net.pyc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file
or directory)

clearly the above corresponds to my second import statement but for
some reason it fails with ENOENT when trying to open myutils/net.so
which definately exists.

Noticing it appears to be using a relative path I decided to try again
and this time I chdir() to /usr/lib/python2.4/site-extensions directory
before launching the interpreter and it then it works perfectly fine.
Of course I shouldn't have to set my cwd to the site-extensions
directory before using my extension module..

How do I fix this? What have I done wrong?

Help very much appreciated, thanks.

Tyler

 

[Matthew Woodcraft]

I've written a small python extension but I'm having difficulty loading
it at runtime. The source for my extension is a module which is a
member of a package is organized as follows.

test/setup.py
test/myutils/__init__.py
test/myutils/netmodule.c
[...]

Noticing it appears to be using a relative path I decided to try again
and this time I chdir() to /usr/lib/python2.4/site-extensions directory
before launching the interpreter and it then it works perfectly fine.
Of course I shouldn't have to set my cwd to the site-extensions
directory before using my extension module..

It looks to me like the problem is that you have a 'myutils'
subdirectory inside the directory that you're starting the interpreter
in. Are you running it from the 'test' directory?

When you use the interactive interpreter, Python's default search path
for modules and packages includes the current directory as the first
entry, so it's finding 'myutils' there rather than in
/usr/lib/python2.4/site-packages.

If that's right, then to avoid this problem you can either run Python
from any other directory, or put a copy of your 'net.so' in the local
'myutils' too.

-M-