How to pass a reference to the current module

[James Stroud]

Hello All,

Say I have this code:


import AModule
import ModuleUser

ModuleUser.do_something_with(AModule, 'some_function_name')


But say the functions are defined in the same module that contains the
above code. I can imagine this hack:


import AModule
import ModuleUser

ModuleUser.do_something_with(AModule.__name__, 'some_name')


Where AModule.__name__ could be substituted for __name__ under the
proper circumstances. Then, I would have this code in ModuleUser:


import sys
def do_something_with(modname, funcname):
afunction = sys.modules[modname].funcname
[etc.]


Which is terribly ugly to me and makes for a pretty miserable API in my
opinion, requiring the programmer to type underscores.


Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

Thanks in advance for any suggestions.

James


--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[James Stroud]

Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

Is this the preferred way?


import sys
def do_something_with(funcname, amodule=None):
if amodule is None:
function = sys._getframe(1).f_locals[funcname]
else:
function = getattr(amodule, funcname)
[etc.]


James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[James Stroud]

Basically, what I am trying to acomplish is to be able to do this in
any arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

Is this the preferred way?


import sys
def do_something_with(funcname, amodule=None):
if amodule is None:
function = sys._getframe(1).f_locals[funcname]
else:
function = getattr(amodule, funcname)
[etc.]


James

_.replace('locals', 'globals')

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Carsten Haese]

Hello All,

Say I have this code:


import AModule
import ModuleUser

ModuleUser.do_something_with(AModule, 'some_function_name')


But say the functions are defined in the same module that contains the
above code. I can imagine this hack:


import AModule
import ModuleUser

ModuleUser.do_something_with(AModule.__name__, 'some_name')


Where AModule.__name__ could be substituted for __name__ under the
proper circumstances. Then, I would have this code in ModuleUser:


import sys
def do_something_with(modname, funcname):
afunction = sys.modules[modname].funcname
[etc.]


Which is terribly ugly to me and makes for a pretty miserable API in my
opinion, requiring the programmer to type underscores.


Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

Thanks in advance for any suggestions.

I can't be too sure because you may have abstracted away too much of
what you're actually trying to accomplish, but it seems to me that
instead of passing a reference to the current module and a function
name, you should just pass a reference to the function that you want
do_something_with to call.

If I missed the point of your question, please describe less abstractly
what you're trying to do.

 

[Carsten Haese]

Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

Is this the preferred way?


import sys
def do_something_with(funcname, amodule=None):
if amodule is None:
function = sys._getframe(1).f_locals[funcname]
else:
function = getattr(amodule, funcname)
[etc.]

This seems to confirm my suspicion that the do_something_with function
doesn't actually need a reference to the module, it only needs a
reference to the function to call.

Maybe your hurdle is how to obtain a reference to a function from the
current module when all you have is the name of the function in a
string. The answer to that would be "globals()[funcname]".

 

[James Stroud]

please describe less abstractly what you're trying to do.

You sound like my former thesis adviser.

OK. From an external source, such as configuration file, the user will
specify the *name* of a function somehow with the assumption that the
function will be coded somewhere (in the present or other modules) by
that name. This behavior is to be implemented in a very primitive
pythoncard-like application wherein the user can specify hooks via a
configuration file. The configuration file will describe the GUI. To go
one more step deeper into the rationale, the idea would be to replace
optparse type behavior with a gui wherein one can type input parameters
into fields rather than specify them at the command line with optional
flags.

Please take a look at my previous response to myself for one possible
solution I devised. I don't particularly like it much, however, because
it seems unnatural.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[James Stroud]

This seems to confirm my suspicion that the do_something_with function
doesn't actually need a reference to the module, it only needs a
reference to the function to call.

Maybe your hurdle is how to obtain a reference to a function from the
current module when all you have is the name of the function in a
string. The answer to that would be "globals()[funcname]".

This is like a bad case of phone-tag. Please see my response to your
previous post for why this does not seem feasible to me.

James


--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Paul Rubin]

You sound like my former thesis adviser.

I couldn't understand what you were trying to do either.

OK. From an external source, such as configuration file, the user will
specify the *name* of a function somehow with the assumption that the
function will be coded somewhere (in the present or other modules) by
that name.

Is something wrong with

import some_module
func = getattr(some_module, func_name)

?

 

[Paul Rubin]

namespace = module.__dict__
return namespace[name]

Am I missing something? It's likely that I just don't understand
the problem, but I don't understand these dictionary extractions
for what I thought would be an attribute lookup:

Python 2.4.4 (#1, Oct 23 2006, 13:58:00)
That just seems like a cleaner way to access stuff in a module,
using the published interface instead of an implementation detail.

 

[James Stroud]

Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

I second Carsten Haese's suggestion that instead of passing function
names, you pass function objects, in which case you don't need the module.
But perhaps you need some way of finding the function, given its name.

def get_function_from_name(name, module=None):
if module is None:
# use the current namespace
namespace = locals() # or globals() if you prefer
else:
namespace = module.__dict__
return namespace[name]

This assumes that get_function_from_name is defined in the same module
as the function named by name. However, I want this:


Module Behavior
============== ====================================================
UserDefined1 Imports FunctionUser
ThirdParty Contains User Functions (May be ==UserDefined1)
FunctionUser do_something_with() and/or get_function_from_name()

So the name-to-function mapping is done in FunctionUser but the function
is actually defined in UserDefined1 (or ThirdParty if ThirdParty is
different than UserDefined1).

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Steven D'Aprano]

Basically, what I am trying to acomplish is to be able to do this in any
arbitrary module or __main__:


funcname = determined_externally()
ModuleUser.do_something_with(AModule, funcname)


Ideally, it would be nice to leave out AModule if the functions were
designed in the same namespace in which do_something_with is called.

I second Carsten Haese's suggestion that instead of passing function
names, you pass function objects, in which case you don't need the module.
But perhaps you need some way of finding the function, given its name.

def get_function_from_name(name, module=None):
if module is None:
# use the current namespace
namespace = locals() # or globals() if you prefer
else:
namespace = module.__dict__
return namespace[name]

 

[Steven D'Aprano]

This seems to confirm my suspicion that the do_something_with function
doesn't actually need a reference to the module, it only needs a
reference to the function to call.

Maybe your hurdle is how to obtain a reference to a function from the
current module when all you have is the name of the function in a
string. The answer to that would be "globals()[funcname]".

This is like a bad case of phone-tag. Please see my response to your
previous post for why this does not seem feasible to me.

I've read your previous post, and I don't understand why you consider it
unfeasible. The user provides a function name in a config file, which
gives you funcname as a string. You already know which module it comes
from (you didn't specify how, but all your examples show that). Carsten is
simply showing you how to get the function from the function name without
the nasty hack using sys._getframe. Why is in unfeasible?

 

[Paul Rubin]

Module Behavior
============== ====================================================
UserDefined1 Imports FunctionUser
ThirdParty Contains User Functions (May be ==UserDefined1)
FunctionUser do_something_with() and/or get_function_from_name()

So the name-to-function mapping is done in FunctionUser but the
function is actually defined in UserDefined1 (or ThirdParty if
ThirdParty is different than UserDefined1).

I'm still completely confused. Does FunctionUser know what module the
user functions are supposed to come from? Could you give an example
of what you want the actual contents of those 3 modules to look like?
E.g.:

UserDefined1.py:
import FunctionUser

def foo(x):
print x+3

FunctionUser.py:
def do_something_with (module, funcname, *args, **kw):
func = getattr(module, funcname)
func (*args, **kw)

main.py:
import UserDefined1, FunctionUser

# the following should print 10
FunctionUser.do_something_with(UserDefined1, 'foo', 7)

I don't think the above is quite what you want, but is it somewhere
close?

 

[Carsten Haese]

You sound like my former thesis adviser.

Thanks. I guess.

OK. From an external source, such as configuration file, the user will
specify the *name* of a function somehow with the assumption that the
function will be coded somewhere (in the present or other modules) by
that name.

Does the configuration file indicate which module contains the function?

 

[James Stroud]

Thanks. I guess.

Yes, its a compliment in disguise--just check his CV.

Does the configuration file indicate which module contains the function?

It is possible. Please see my latest response to Paul Rubin.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[James Stroud]

I'm still completely confused. Does FunctionUser know what module the
user functions are supposed to come from? Could you give an example
of what you want the actual contents of those 3 modules to look like?
E.g.:

UserDefined1.py:
import FunctionUser

def foo(x):
print x+3

FunctionUser.py:
def do_something_with (module, funcname, *args, **kw):
func = getattr(module, funcname)
func (*args, **kw)

main.py:
import UserDefined1, FunctionUser

# the following should print 10
FunctionUser.do_something_with(UserDefined1, 'foo', 7)

I don't think the above is quite what you want, but is it somewhere
close?

Its very close. However, there is the possibiltiy that main.py and
UserDefined1.py are the same module. In such a case I'm guessing that I
need to resort to the gymnastics of frame inspection I mentioned
earlier. The problem is when this combination of possibilities exists:

1. main.py is named without the .py (e.g. `main`) as would be
the convention for "programs"), and is not a true python
module as a result--perhaps python has a mechanism for
importing such files?
2. foo is defined in `main`

Possibility two (even when it is a properly named module) sets the stage
for a circular import, but I believe in python this is not entirely
worrisome. However, the combination above makes it difficult to pass a
reference to the `main` namespace without some sort of introspection.
Ideally, I would prefer to not require the user to provide some mapping
as it detracts from the convenience of the API. For example:

from UserDefined1 import some_function
def foo(): [etc.]
def doit(): [etc.]
mapping = {
'foo' : foo,
'doit' : doit,
'some_function' : some_function
}

The idea would be that the above mapping would be specified in the
configuration file:

[foo]
param1 = float
param2 = 4

[option1]
__module__ = 'UserDefined1'
__function__ = 'doit'
param1 = str
param2 = 30.0

[doit]
param1 = float
param2 = float

[etc.]

James


--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Steven D'Aprano]

namespace = module.__dict__
return namespace[name]

Am I missing something? It's likely that I just don't understand
the problem, but I don't understand these dictionary extractions
for what I thought would be an attribute lookup:

Well, the *real* reason I used a __dict__ extraction was because I didn't
think of using getattr at the time, but as a post facto justification,
I'll point at the difficulty in using getattr for extracting a function
from the current module:
.... return x + 1
....

Maybe I've just got a blind spot, but I can't think of a way to use
getattr to get something from the current module. That would lead to
something like this:


def get_function_from_name(name, module=None):
if module is None:
return globals()[name] # or use locals()
else:
return getattr(name, module)

which is fine, I suppose, and I would have used it earlier if I had
thought of it *wink*

 

[Steven D'Aprano]

Its very close. However, there is the possibiltiy that main.py and
UserDefined1.py are the same module. In such a case I'm guessing that I
need to resort to the gymnastics of frame inspection I mentioned
earlier.

I suggest you're falling for the anti-pattern of "Big Design Up Front",
and are overly complicating your system "just in case it's useful". Why
not just _insist_ that main.py and UserDefined1.py must be different
modules? You're the application developer, you're allowed to do that.

The problem is when this combination of possibilities exists:

1. main.py is named without the .py (e.g. `main`) as would be
the convention for "programs"), and is not a true python
module as a result--perhaps python has a mechanism for
importing such files?

I don't believe it does.

2. foo is defined in `main`

Maybe you should just prohibit that? Do you really want to allow users to
call arbitrary code in your main application at arbitrary times and places?

Maybe you do. Okay, it's your gun and your foot.

But in any case, it shouldn't matter. main is the program running -- it
has to be, because you can't import it from another program -- so
globals()['foo'] will work.

Possibility two (even when it is a properly named module) sets the stage
for a circular import, but I believe in python this is not entirely
worrisome. However, the combination above makes it difficult to pass a
reference to the `main` namespace without some sort of introspection.

That's what globals() is for.

Ideally, I would prefer to not require the user to provide some mapping
as it detracts from the convenience of the API. For example:

from UserDefined1 import some_function
def foo(): [etc.]
def doit(): [etc.]
mapping = {
'foo' : foo,
'doit' : doit,
'some_function' : some_function
}

The idea would be that the above mapping would be specified in the
configuration file:

[foo]
param1 = float
param2 = 4

[option1]
__module__ = 'UserDefined1'
__function__ = 'doit'

I'm not sure why you need the quotation marks around the module and
function names. What else could UserDefined1 be, other than a string?

 

[Paul Rubin]

I'll point at the difficulty in using getattr for extracting a function
from the current module:

Hmm, it's a pain that there's no clean way to get at the current
module. PEP 3130 shows some icky and unreliable ways, e.g.

func = getattr(sys.modules[__name__], 'f')

PEP 3130's goal was to add a clean way to do this. Unfortunately it
was rejected.

 

[James Stroud]

Hmm, it's a pain that there's no clean way to get at the current
module. PEP 3130 shows some icky and unreliable ways, e.g.

func = getattr(sys.modules[__name__], 'f')

PEP 3130's goal was to add a clean way to do this. Unfortunately it
was rejected.

Yes, this is the essence of the problem. At least I'm not the only one
to have needed (or at least wanted) this.

James