Importing Modules

[Walter Brunswick]

What is the best way to import all modules in a directory (and possibly a subdirectory/subdirectories), possibly including
conditionals, such as regexes?

 

[Peter Hansen]

What is the best way to import all modules in a directory
> (and possibly a subdirectory/subdirectories), possibly including
conditionals, such as regexes?

The "best" was, as always, depends on what your use case is. Why do you
want to do this? What will you do with the modules once they are imported?

Also: the second part of the question doesn't mean anything to me. What
do you mean by "conditionals, such as regexes", in the context of
importing modules?

-Peter

 

[Walter Brunswick]

The purpose is rather irrelevant. The modules could be used for an assortment of tasks.

By conditionals I mean if the name of a module contains a substring, such as "asdf" (i.e. "asdf" in module) or matches a pattern of
some sort, for example, all modules which match the regex "module[\d]+\.py", which could be "module5.py" or "module11199.py" or
"module345.py".

 

[Sam Pointon]

On the second point, a combination of sys.path, os.listdir and
__import__ should do what you're after, although sifting through the
whole of sys.path and subfolders from Python, rather than the
interpreter itself, could be slow. (And it'll be redundant as well -
__import__ will have do the same thing, though you could fix that by
using the imp module).

-Should- work, but not tested, so don't blame me if it doesn't:

import sys, os, re

def find_modules(str_pat):
pat = re.compile(str_pat + '.py[co]?')
init_pat = re.compile('__init__.py[co]?')
for dir in sys.path:
files = [n for n in os.listdir(path) if pat.search(n)]
[__import__(name.rspit('.')) for name in files]
for possible_dir in os.listdir(path):
try:
if True in [init_pat.match(n) for n in os.listdir(name
+ '/' + possible_dir):
find_modules(str_pat)
except OSError: #does os.listdir raise this?
pass

 

[Devan L]

On the second point, a combination of sys.path, os.listdir and
__import__ should do what you're after, although sifting through the
whole of sys.path and subfolders from Python, rather than the
interpreter itself, could be slow. (And it'll be redundant as well -
__import__ will have do the same thing, though you could fix that by
using the imp module).

-Should- work, but not tested, so don't blame me if it doesn't:

__import__(modulename) is not equivalent to import modulename;
__import__ returns a module object
[QUOTE][/QUOTE]

 

[Peter Hansen]

The purpose is rather irrelevant.

The purpose of something is not only relevant but essential when someone
asks for the "best" way to do it.

For example, without knowing anything of your purpose, I could simply
say that writing a module with a series of import statements, one per
module in the target subdirectory, is the "best" way, and I could easily
defend that as "best" against all other possible approaches until we
knew what the real reason for doing this was.

(Your extra detail on the regex thing makes it clear that the above
approach would not be "best", but I think that just goes to show that
until we know why you want this, we cannot possibly answer your question
properly.)

-Peter

 

[Terry Hancock]

The purpose of something is not only relevant but essential when someone
asks for the "best" way to do it.

For example, without knowing anything of your purpose, I could simply
say that writing a module with a series of import statements, one per
module in the target subdirectory, is the "best" way, and I could easily
defend that as "best" against all other possible approaches until we
knew what the real reason for doing this was.

Indeed for any situation in which this solution is possible, I would
regard it as the best way to import modules: "explicitly".

The need to import modules implicitly usually implies some kind of
plugin pattern. A simple application of os.listdir(), some use of
glob or simple string methods, and the __import__ built-in function
will usually do what you want in the implicit case.

But don't do the implicit version unless you have a very good
reason for doing so (such as a plugin model where the code will
be maintained by someone other than the author of the main program).
In such a situation you should also take reasonable precautions to
prevent major collisions, even if you expect it to be the plugin
author's responsibility to avoid them (e.g. establish interface
rules for plugins and a formal API for interacting with your
program).