package search

[boris]

I have two directories, lib1 and lib2, that both contain the package
foo, one with the submodule mod1
and the other with the submodule mod2:

$ ls lib*/foo/
lib1/foo/:
__init__.py __init__.pyc mod1.py mod1.pyc

lib2/foo/:
__init__.py __init__.pyc mod2.py mod2.pyc
$

Now this script:

import sys
sys.path.append("lib1")
sys.path.append("lib2")
import foo.mod1

will find the module foo.mod1, while the same script with the two
append-lines interchanged will not:

import sys
sys.path.append("lib2")
sys.path.append("lib1")
import foo.mod1

The error is:

import foo.mod1
ImportError: No module named mod1

So I guess that when Python finds the package foo, it will not look for
it again using the rest of
the search path, even if at that first place the module could not be
found.

Other languages like e.g. Java would find the module mod1 in the second
case too (if Java doesn't
find it in the first place, it will also look for it in the second
place).

This behaviour of Python is very unfortunate, because you always have
to merge your package
trees. In my cvs I have several packages, each in its own cvs module,
all starting with
de.science_computing. Now if I need two of them in a new project, I
will have two package trees
in my cvs sandbox, both starting with de.science_computing. Leaving the
sandbox that way, cvs
will work but Python imports will fail while if I merge all the package
trees, Python will work but cvs
won't.

My questions:
- Have I done anything wrong?
- If not, is this behaviour of Python a bug or intentional?
- If it it intentional, what is the intent?

Thank you for your help
Boris

 

[Sybren Stuvel]

boris enlightened us with:

So I guess that when Python finds the package foo, it will not look
for it again using the rest of the search path, even if at that
first place the module could not be found.

That is because, when importing foo.mod1, it first imports foo. When
that is done, package foo is "asked for" mod1.

This behaviour of Python is very unfortunate, because you always
have to merge your package trees.

Which sounds like sound advice to me.

In my cvs I have several packages, each in its own cvs module, all
starting with de.science_computing. Now if I need two of them in a
new project, I will have two package trees in my cvs sandbox, both
starting with de.science_computing. Leaving the sandbox that way,
cvs will work but Python imports will fail while if I merge all the
package trees, Python will work but cvs won't.

Then try a better versioning system, like Mercurial, Bazaar or
Subversion.

- Have I done anything wrong?

Yes, development of a single directory structure in several CVS trees.
CVS is very directory-oriented.

- If not, is this behaviour of Python a bug or intentional?
Intentional.

- If it it intentional, what is the intent?

Simplicity and explicitness. You have two packages 'foo', and it
simply loads the first one it finds.

Sybren

 

[boris]

Simplicity and explicitness. You have two packages 'foo', and it
simply loads the first one it finds.

Yes, here you are right. It is indeed simple.

Boris

 

[imho]

boris ha scritto:

I have two directories, lib1 and lib2, that both contain the package
foo, one with the submodule mod1
and the other with the submodule mod2:
[...]
Now this script:

import sys
sys.path.append("lib1")
sys.path.append("lib2")
import foo.mod1

will find the module foo.mod1, while the same script with the two
append-lines interchanged will not:

import sys
sys.path.append("lib2")
sys.path.append("lib1")
import foo.mod1

The error is:

import foo.mod1
ImportError: No module named mod1
> [...]

You just have to put in "__init__.py" in "lib2" (the package directory
you are "extending"), the following lines:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

"__path__", in each __init__ module, is a list initialized with the
module's path, but you can extend it by appending paths where you want
the interpreter to look for further modules.
pkgutil.extend_path automatically appends to __path__ all subdirectories
of directories on sys.path named after the package.

HTH

Diego.

 

[boris]

You just have to put in "__init__.py" in "lib2" (the package directory
you are "extending"), the following lines:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

"__path__", in each __init__ module, is a list initialized with the
module's path, but you can extend it by appending paths where you want
the interpreter to look for further modules.
pkgutil.extend_path automatically appends to __path__ all subdirectories
of directories on sys.path named after the package.

HTH

Diego.

COOL! You just saved me an awful lot of work.

Thanks, Diego!

Boris

 

[imho]

boris ha scritto:

COOL! You just saved me an awful lot of work.

Thanks, Diego!

Boris

;-)

Bye