funcs vs vars in global namespace

[David Rysdam]

Getting no answer yesterday, I've done some investigation and I
obviously don't understand how python namespaces work. Here's a test
program:

#!/usr/bin/python

b = 2

def sumWithGlobal(a):
return a + b

#this is easy
print 'Case 1: Normal function call:'
print sumWithGlobal(2)
print

#If you don't send a globals dict with eval, it uses the one from
#the calling program, so this is basically the same as Case 1
print 'Case 2: Function call from inside an eval using the same globals():'
print eval("sumWithGlobal(2)")
print

#Since I'm sending in a globals dict but haven't included a defintion
#for sumWithGlobal(), obviously this will fail (commented out to get
#past the error)
print 'Case 3: Attempt to replace just the global var b (fails for known
reason)'
#print eval("sumWithGlobal(2)", {'b':3})
print

#Here is define sumWithGlobals but not b and it still works. Why?
#Shouldn't I get an error that b is undefined, since it isn't in the
#globals dict of the eval?
print 'Case 4: Attempt to set just the function sumWithGlobal (succeeds
for unknown reason'
print eval("sumWithGlobal(2)", {'sumWithGlobal':sumWithGlobal})
print

#And finally I define both but I still get output as if b = 2
#Again, why? In the eval's global, b = 3, doesn't it?
print 'Case 5: Attempt to set both function and var. (var has wrong value)'
print eval("sumWithGlobal(2)", {'sumWithGlobal':sumWithGlobal, 'b':3})
print


If I add a "print globals()" to sumWithGlobal, I see {'b':2} in there in
cases 1, 2, 4 and 5. What am I doing wrong?

 

[Alex Martelli]

def sumWithGlobal(a):
return a + b ...
#Here is define sumWithGlobals but not b and it still works. Why?
#Shouldn't I get an error that b is undefined, since it isn't in the
#globals dict of the eval?
print 'Case 4: Attempt to set just the function sumWithGlobal (succeeds
for unknown reason'
print eval("sumWithGlobal(2)", {'sumWithGlobal':sumWithGlobal})
print

The function sumWithGlobal, internally, uses the global dictionary of
the module it was defined in, NOT that of the module it's being called
from, including the "kinda sorta pseudo" module you have within an eval.
This also explains:

If I add a "print globals()" to sumWithGlobal, I see {'b':2} in there in
cases 1, 2, 4 and 5. What am I doing wrong?

You're misunderstanding Python globals: they are per-module, not "across
all modules".


Alex

 

[David Rysdam]

The function sumWithGlobal, internally, uses the global dictionary of
the module it was defined in, NOT that of the module it's being called
from, including the "kinda sorta pseudo" module you have within an eval.
This also explains:




You're misunderstanding Python globals: they are per-module, not "across
all modules".


Alex

OK, I can kind of understand that. I'm not sure what the point of being
able to specify the globals for an eval() is then, but if it doesn't do
what I want it doesn't do what I want.

What I want to do is be able to run multiple scripts that use the same
global variable and function names but be able to have my master script
define those variables and functions programmatically. Can I do that
with the rexec sandbox stuff? Or is rexec more about keeping the
exec()'d code from doing things than specifying precisely what it can do?

 

[Alex Martelli]

OK, I can kind of understand that. I'm not sure what the point of being
able to specify the globals for an eval() is then, but if it doesn't do
what I want it doesn't do what I want.

The point is clearer when what you're passing to eval is an expression:

eval('a+b', dict(a=23, b=99)).

What I want to do is be able to run multiple scripts that use the same
global variable and function names but be able to have my master script
define those variables and functions programmatically. Can I do that
with the rexec sandbox stuff? Or is rexec more about keeping the
exec()'d code from doing things than specifying precisely what it can do?

rexec has been removed because it did not offer the security it
purported to offer (it was about keeping code from doing bad things, but
it reallly didn't), sigh.

You can do what you want for example by setting the values you want in
the module object of the function you're calling -- that module object's
dictionary is the function's global namespace. Say:

sub_module = __import__(which_one_this_time)
vars(sub_module).update(which_dict_this_time)
print sub_module.the_function(23)

There are other ways, but few are as direct as this one.


Alex

 

[David Rysdam]

...


rexec has been removed because it did not offer the security it
purported to offer (it was about keeping code from doing bad things, but
it reallly didn't), sigh.

You can do what you want for example by setting the values you want in
the module object of the function you're calling -- that module object's
dictionary is the function's global namespace. Say:

sub_module = __import__(which_one_this_time)
vars(sub_module).update(which_dict_this_time)
print sub_module.the_function(23)

There are other ways, but few are as direct as this one.


Alex

Oh wow, of course! I can set properties on the modules themselves. I'm
going to have to rethink what I'm doing and mess with that a bit, I'm
sure I'll have questions and problems later. Thanks!

 

[Alex Martelli]

...
Oh wow, of course! I can set properties on the modules themselves. I'm
going to have to rethink what I'm doing and mess with that a bit, I'm
sure I'll have questions and problems later. Thanks!

You're welcome! Yes, since your modules are not being used for other
purposes except running the scriptlets you're controlling, changing
those modules' global variables should be safe enough in your case.


Alex

 

[David Rysdam]

You're welcome! Yes, since your modules are not being used for other
purposes except running the scriptlets you're controlling, changing
those modules' global variables should be safe enough in your case.


Alex

OK, dumb question #1:

Why do this:

sub_module = __import__(which_module_this_time)
vars(sub_module).update(which_dict_this_time)

When I could just do this:

__import__(which_module_this_time, which_dict_this_time)

?

 

[Alex Martelli]

OK, dumb question #1:

Why do this:

sub_module = __import__(which_module_this_time)
vars(sub_module).update(which_dict_this_time)

When I could just do this:

__import__(which_module_this_time, which_dict_this_time)

?

You can do whatever you wish, but what makes you think these constructs
have similar effects? Quoting from Python's online docs on __import__,

"""
the standard implementation does not use its locals argument at all,
and uses its globals only to determine the package context of the
import statement
"""

In short, the standard implementation of __import__ does NOT alter in
any way the dict of the module it imports.


Alex

 

[David Rysdam]

You can do whatever you wish, but what makes you think these constructs
have similar effects? Quoting from Python's online docs on __import__,

"""
the standard implementation does not use its locals argument at all,
and uses its globals only to determine the package context of the
import statement
"""

In short, the standard implementation of __import__ does NOT alter in
any way the dict of the module it imports.


Alex

Ah, I see.

Your "which_dict_this_time" dictionary, how are you imagining that
working? I was just mapping function name strings to functions
({'logError':logError}), but (long story short) that isn't working how I
want. But shouldn't I be able to define the function right there in the
dictionary itself?

Perhaps this is getting too non-obvious, magical and unmaintainable, though.

 

[Jeff Shannon]

Your "which_dict_this_time" dictionary, how are you imagining that
working? I was just mapping function name strings to functions
({'logError':logError}), but (long story short) that isn't working how
I want. But shouldn't I be able to define the function right there in
the dictionary itself?

Perhaps this would be a bit clearer with more-meaningful names. (Of
course, that's presuming that I'm reading Alex's intent correctly... )

std_global_dict = { 'logError': logError, ... }
script_module = __import__('some_script_module')
vars(script_module).update(std_global_dict)

This will have the effect of injecting all of the specified names (in
std_global_dict) into the scriptlet's module, where they can be used as
global variables. (But note that logError(), and other functions put
into std_global_dict, will execute in the context in which they were
defined -- that is, logError is using its own module's globals, not
script_module's globals or std_global_dict.)

Jeff Shannon
Technician/Programmer
Credit International

 

[Bengt Richter]

Ah, I see.

Your "which_dict_this_time" dictionary, how are you imagining that
working? I was just mapping function name strings to functions
({'logError':logError}), but (long story short) that isn't working how I
want. But shouldn't I be able to define the function right there in the
dictionary itself?

Perhaps this is getting too non-obvious, magical and unmaintainable, though.

Maybe this will give you some ideas:

Here empty.py is just a single blank text line with a (windows in this case, which
for the heck of it I'll show in binary) EOL: '\r\n'

>>> import empty When you import, you automatically get some stuff in the module directory
>>> dir(empty) ['__builtins__', '__doc__', '__file__', '__name__']
>>> vars(empty).keys()

['__builtins__', '__name__', '__file__', '__doc__']

If you have source defining a function, you can exec the definition in a directory,
which will become what the function sees as its global directory. So,
... def foo(a):
... return a+b
... """ in empty.__dict__

It shows up:
['__builtins__', '__doc__', '__file__', '__name__', 'foo']

since b is retrieved from the global dict, we better put something there.
That happens to be the dict that stores module attributes, so we can:
223
.... as expected.

We can verify that foo's global dict is empty's dict: True

If you have definitions of functions in source files, you can execfile them
in a similar manner:

First I'll creat a source right here, though I could have done it in an editor ... def baz(x):
... return 2*b+x
... """)

Verify ----

def baz(x):
return 2*b+x
----

Use execfile do define it in empty:
It shows up: ['__builtins__', '__doc__', '__file__', '__name__', 'b', 'baz', 'foo']

Check b: 444

HTH. Note that exec-ing or execfile-ing untrusted source is not safe.

BTW, if in the same interpreter execution session you import empty from another module,
the updated contents should be visible. E.g.,
----
import empty
----

>>> import mimpbaz
>>> dir(mimpbaz) ['__builtins__', '__doc__', '__file__', '__name__', 'empty']
>>> mimpbaz.empty is empty True
>>> mimpbaz.empty.b 200
>>> mimpbaz.empty.baz(3056)

3456

recalling that baz did 3456

Note that the source of empty.py has not changed ----

---- '\n'

Of course, it doesn't have to start out empty. The point is
exec and execfile don't save a compiled .pyc for you to make
it faster next time, whereas import does (unless it's prevented
by lack of write privilege or such, I think).

Regards,
Bengt Richter

 

[Alex Martelli]

...
Your "which_dict_this_time" dictionary, how are you imagining that
working?

Just like any other dictionary -- maybe I don't get your question...?

I was just mapping function name strings to functions
({'logError':logError}), but (long story short) that isn't working how I
want. But shouldn't I be able to define the function right there in the
dictionary itself?

....because def is a statement and "in the dictionary itself" you can, of
course, only have expressions...? Is this a trick question...?

Perhaps this is getting too non-obvious, magical and unmaintainable, though.

Your questions are definitely anything but obvious to me...


Alex

 

[David Rysdam]

Maybe this will give you some ideas:

Aha, yes indeed this gives me ideas. Very excellent tutorial, I've
saved it for when I manage to confuse myself again later. I think I
know just how to do what I want to do now, though I'm beginning to
question the wisdom of providing the feature in question at all. But if
I'm going to, it should be right, so I still thank you.

 

[David Rysdam]

Aha, yes indeed this gives me ideas. Very excellent tutorial, I've
saved it for when I manage to confuse myself again later. I think I
know just how to do what I want to do now, though I'm beginning to
question the wisdom of providing the feature in question at all. But if
I'm going to, it should be right, so I still thank you.

I knew I was doing this the wrong way and I've just realized what the
Right Way is: I should just use objects instead of trying to manipulate
dictionaries. When I make a new version of the "API", I just create a
new class to inherit from the previous version.

I'm a bit concerned that it took me so long to realize this...