Automatic import of submodules

[Massi]

Hi everyone,

in my project I have the following directory structure:

plugins
|
-- wav_plug
|
-- __init__.py
-- WavPlug.py
-- mp3_plug
|
-- __init__.py
-- Mp3Plug.py
....
-- etc_plug
|
-- __init__.py
-- EtcPlug.py

Every .py file contain a class definition whose name is identical to
to the file name, so in my main script I have to import each submodule
like that:

from plugins.wav_plug.WavPlug import WavPlug
from plugins.wav_plug.Mp3Plug import Mp3Plug

and so on. This is uncomfortable, since when a new plugin is added I
have to import it too. So my question is, is it possible to iterate
through the 'plugins' directory tree in order to automatically import
the submodules contained in each subdirectory?
I googled and found that the pkgutil could help, but it is not clear
how. Any hints?
Thanks in advance.

 

[Dave Angel]

Hi everyone,

in my project I have the following directory structure:

plugins
|
-- wav_plug
|
-- __init__.py
-- WavPlug.py
-- mp3_plug
|
-- __init__.py
-- Mp3Plug.py
...
-- etc_plug
|
-- __init__.py
-- EtcPlug.py

Every .py file contain a class definition whose name is identical to
to the file name, so in my main script I have to import each submodule
like that:

from plugins.wav_plug.WavPlug import WavPlug
from plugins.wav_plug.Mp3Plug import Mp3Plug

and so on. This is uncomfortable, since when a new plugin is added I
have to import it too. So my question is, is it possible to iterate
through the 'plugins' directory tree in order to automatically import
the submodules contained in each subdirectory?
I googled and found that the pkgutil could help, but it is not clear
how. Any hints?
Thanks in advance.

I think the key to the problem is the __import__() function, which takes
a string and returns a module object. So you make a list of the fully
qualified module names (filenames less the extension), and loop through
that list, Then for each one, you can extract items from the module.

It doesn't make sense to define the class names at your top-level,
though, since you'd not have any code to reference any new plugin if it
has a unique class name. So at some point, you're probably going to have
a list or map of such class objects.

 

[Jean-Michel Pichavant]

Hi everyone,

in my project I have the following directory structure:

plugins
|
-- wav_plug
|
-- __init__.py
-- WavPlug.py
-- mp3_plug
|
-- __init__.py
-- Mp3Plug.py
...
-- etc_plug
|
-- __init__.py
-- EtcPlug.py

Every .py file contain a class definition whose name is identical to
to the file name, so in my main script I have to import each submodule
like that:

from plugins.wav_plug.WavPlug import WavPlug
from plugins.wav_plug.Mp3Plug import Mp3Plug

and so on. This is uncomfortable, since when a new plugin is added I
have to import it too. So my question is, is it possible to iterate
through the 'plugins' directory tree in order to automatically import
the submodules contained in each subdirectory?
I googled and found that the pkgutil could help, but it is not clear
how. Any hints?
Thanks in advance.

Hi,

Try something like (*untested code*)

plugins = {}
classes = {}

for plugin, className in [('wav_plug', 'WavPlug'), ('mp3_plug', 'Mp3Plug')]:
plugins[plugin] = __import__(os.path.join('plugins', plugin, className))
classes[className] = getattr(plugins[plugin], className)

# raise a keyError if the plugin has not been imported
wav = classes['wav_plug']()


Make sure all subdirs have the __init__.py file, including the plugins
directory.

JM

 

[alex23]

plugins
    |
    -- wav_plug
          |
          -- __init__.py
          -- WavPlug.py
    -- mp3_plug
          |
          -- __init__.py
          -- Mp3Plug.py
...
    -- etc_plug
          |
          -- __init__.py
          -- EtcPlug.py

What do you gain by having each plugin as a package? Unless you're
storing other resources with each plugin, I'd move all your XXXPlug.py
files into plugins/ I'd also probably call the modules 'wavplug' and
the class 'WavPlug' to always make it clear to which you're
referring.

Every .py file contain a class definition whose name is identical to
to the file name, so in my main script I have to import each submodule
like that:

from plugins.wav_plug.WavPlug import WavPlug
from plugins.wav_plug.Mp3Plug import Mp3Plug

and so on. This is uncomfortable, since when a new plugin is added I
have to import it too. So my question is, is it possible to iterate
through the 'plugins' directory tree in order to automatically import
the submodules contained in each subdirectory?

It's not exactly automatic, but you could move all of those imports
into plugins/__init__.py, then just do a single

from plugins import *

in your main module.

 

[Shambhu Rajak]

-----Original Message-----
From: Massi [mailto:[email protected]]
Sent: 25/11/2011 6:30 PM
To: (e-mail address removed)
Subject: Automatic import of submodules

Hi everyone,

in my project I have the following directory structure:

plugins
|
-- wav_plug
|
-- __init__.py
-- WavPlug.py
-- mp3_plug
|
-- __init__.py
-- Mp3Plug.py
....
-- etc_plug
|
-- __init__.py
-- EtcPlug.py

Every .py file contain a class definition whose name is identical to
to the file name, so in my main script I have to import each submodule
like that:

from plugins.wav_plug.WavPlug import WavPlug
from plugins.wav_plug.Mp3Plug import Mp3Plug

and so on. This is uncomfortable, since when a new plugin is added I
have to import it too. So my question is, is it possible to iterate
through the 'plugins' directory tree in order to automatically import
the submodules contained in each subdirectory?
I googled and found that the pkgutil could help, but it is not clear
how. Any hints?
Thanks in advance.


=====
What you could do is :

Instead of importing method wise , import the module.

from plugins.wav_plug import WavPlug

you can then use like this then
WavPlug.WavPlug

This will avoid the-every time importing of each method

Regards,
Shambhu