"Plugin" architecture - how to do?

[anglozaxxon]

I'm making a program that consists of a main engine + plugins. Both
are in Python. My question is, how do I go about importing arbitrary
code and have it be able to use the engine's functions, classes, etc?
First, here's how it's laid out on the filesystem:

-mainstarterscript.py
-<engine>
-__init__.py
-[whole bunch of files]
-<plugin>
-__init__.py
-main.py
-[whole bunch of files]

So you'd execute the whole thing with ./mainstarterscript.py, which
imports engine. Engine initializes, then executes plugin somehow.
Plugin needs to be able to call functions in engine, and vice versa.
Although various IPC methods would probably work, I think there's
gotta be an easier and more straightforward way to do it.

I use execfile to execute the plugin's __init__.py script. I insert
some globals from engine into it so that __init__ can use them.
__init__.py also imports plugin/main.py, which uses functions the
engine functions, and so must import __init__.py. The problem is,
when main imports __init__, the inserted globals aren't there, so it
dies. So I've tried to make some globals in __init__ that simply map
them to the engine's globals. Here's roughly how:
#plugin/__init__.py

engine_globals = {}

if __name__ == '__builtin__': #run via execfile
global engine_globals
engine_globals = globals() #I don't really want all the globals, but
in my real code I go through them one by one
else: #imported from one of the other scripts in plugin dir.
pass

import main #among other things
# end __init__.py

# plugins/main.py
import __init__

__init__.engine_globals.['some_function'](args)
#do other stuff
#end main.py

#engine/__init__.py
def some_function(args):
pass

sys.argv.append('plugins')
execfile('plugins/__init__.py')

When main.py imports plungin/__init__, it reinitializes engine_globals
to {}, of course. But I can't think of a way not to initialize it to
something, thus overwriting the original, because it needs to be in
the namespace.. Essentially, I want a global to not reset itself if
it's already set.

So how do I go about doing this? Am I on the right track? Should I
be __import__'ing plugins instead of execfile'ing them?

Thanks,
Nick

 

[Nate Finch]

I'm making a program that consists of a main engine + plugins. Both
are in Python. My question is, how do I go about importing arbitrary
code and have it be able to use the engine's functions, classes, etc?

For a true plugin architecture, you don't have the main engine calling
methods on the plugin. What you do is have an API on your engine with
methods the plugin can call and events it can hook into. The events
are how the engine communicates state to any plugins, without having
to know who they are or what they do. Your engine has a configuration
that tells it what plugins to load (which the plugins presumably
modified when they installed themselves) or otherwise has some
standard way that the engine can figure out what plugins need to be
loaded.

Now granted, I don't specifically know how to do this via python..
but, maybe what I've said will send you in the right direction.

-Nate

 

[anglozaxxon]

For a truepluginarchitecture, you don't have the main engine calling
methods on theplugin. What you do is have an API on your engine with
methods theplugincan call and events it can hook into. The events
are how the engine communicates state to any plugins, without having
to know who they are or what they do. Your engine has a configuration
that tells it what plugins to load (which the plugins presumably
modified when they installed themselves) or otherwise has some
standard way that the engine can figure out what plugins need to be
loaded.

Now granted, I don't specifically know how to do this via python..
but, maybe what I've said will send you in the right direction.

-Nate

Alright that's a good suggestion. But the problem I'm having is that
I don't know how to execute the plugin. I think I know how to do the
architecture, just that there needs to be some crazy circular
importing going on and I was wondering if there was a smarter way to
attack the problem.

Thanks,
Nick

 

[c james]

Take a look at Trac. This might give you some ideas.

http://trac.edgewall.org/wiki/TracDev/ComponentArchitecture

 

[anglozaxxon]

Take a look at Trac. This might give you some ideas.

http://trac.edgewall.org/wiki/TracDev/ComponentArchitecture

Thanks cJames, that's exactly what I'm looking for.