Plugin system

[Reinhold Birkenfeld]

Hello,

another philosophical question:

How would you implement a plugin system (for a web-based application)?

Conditions are:
- One plugin can have one or many "parts" (or "subplugins", or whatever)
- Each subplugin has a type
- The rest of the plugin interface is dependent on the type

I have one possible solution in mind, but I want to wait for your
opinion before revealing it.

Reinhold

 

[Irmen de Jong]

How would you implement a plugin system (for a web-based application)?

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

--Irmen

 

[Reinhold Birkenfeld]

This is so general a question that I cannot think of anything concrete to answer.
Please ask again, with more detail about what you want/need/have in mind.

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

Reinhold

 

[Josiah Carlson]

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

...Have a place for plugins
app/plugins

...Load the plugins on startup
import traceback
import os
for fn in os.listdir('plugins'):
if fn[:1] != '_' and fn.split('.')[-1] in ('py', 'pyw'):
try:
globals()[fn] = \
__import__('plugins.%s'%fn, globals(), locals(), [fn])
except:
traceback.print_exc()

Etcetera. May not be the best, but it will get the job done.

- Josiah

 

[Peter Hansen]

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

And _this_ sounds like a combination of something that's more
a design issue (the structure) and a trivial implementation
detail (trivial in Python, anyway).

I'll try anyway, as a first test case: put all plugins in a
standard folder, with nothing else present in that folder.
They can be individual modules or packages as you will.
The code to load the plugin consists of an __import__...
and not much else.

Maybe this feedback will help you refine the question further...

-Peter

 

[Dan Perl]

And _this_ sounds like a combination of something that's more
a design issue (the structure) and a trivial implementation
detail (trivial in Python, anyway).

I'll try anyway, as a first test case: put all plugins in a
standard folder, with nothing else present in that folder.
They can be individual modules or packages as you will.
The code to load the plugin consists of an __import__...
and not much else.

I have done something similar, with a few differences. I am using a
convention that all the plugins are implemented as a class with a specific
name ("Handler" in my case), each plugin in its own module (so each Handler
class is scoped by its module). I import all the modules in the directory
dedicated to plugins and I do the following checks: the module contains a
class Handler, the module is not the one implementing the base class itself
(bit of a hack, I admit), and the Handler class is a subclass of the base
class (use isinstance). The checks allow to have also other modules in the
plugins directory that do not actually implement a plugin (for instance, I
have a mixin class in that same directory).

Because you have both plugins and sub-plugins you may want to use some
checks like mine, so you have both plugins and sub-plugins in the same
directory, but you select only the plugin classes. That's if I understand
your requirements correctly.

Dan

 

[Greg Copeland]

Okay, the question is too open: My focus is on how you would structure
the plugin code (directories, files) and how you would write code to
load these plugins.

...Have a place for plugins
app/plugins

...Load the plugins on startup
import traceback
import os
for fn in os.listdir('plugins'):
if fn[:1] != '_' and fn.split('.')[-1] in ('py', 'pyw'):
try:
globals()[fn] = \
__import__('plugins.%s'%fn, globals(), locals(), [fn])
except:
traceback.print_exc()

Etcetera. May not be the best, but it will get the job done.

- Josiah

Just FYI, dynamically loaded plugins will not be caught by tools like
py2exe. Another mechanism, which may be more py2exe friendly, is to make
a tool which dynamically creates a python file, which lists all of the
imports. That way, you import the dynamically created file, but once it's
created, you immediately leverage the file for use with py2exe, which can
now follow all of the imports.

Of course, if you never plan on being a py2exe user, you can happily
ignore my ramblings.

Cheers,

Greg

 

[Dan Perl]

I have done something similar, with a few differences. I am using a
convention that all the plugins are implemented as a class with a specific
name ("Handler" in my case), each plugin in its own module (so each
Handler class is scoped by its module). I import all the modules in the
directory dedicated to plugins and I do the following checks: the module
contains a class Handler, the module is not the one implementing the base
class itself (bit of a hack, I admit), and the Handler class is a subclass
of the base class (use isinstance). The checks allow to have also other
modules in the

I forgot what I'm doing in my own code. I looked at it again and found that
I'm not using isinstance(), instead I am doing a check for
"baseClass.Handler in hndlrCls.__bases__", which is much better, because it
doesn't need the creation of an instance. Here is the actual code (my
handlers are the equivalent of your plugins):
# Build a list of the modules that contain a valid Handler class.
hndlrList = []
for hndlrFile in glob.glob(
os.path.join(os.environ['zigzag_install_dir'], 'handlers',
'*.py')):
modName = os.path.split(os.path.splitext(hndlrFile)[0])[1]
if modName == 'baseClass' or \
modName == 'handlerTemplate':
continue
impMod = __import__('handlers.'+modName,
globals(),
locals(),
['Handler'])
if not vars(impMod).has_key('Handler'):
continue
hndlrCls = vars(impMod)['Handler']
if not baseClass.Handler in hndlrCls.__bases__:
continue
hndlrList.append(modName)

baseClass.Handler is the base class for all my handlers.