python exec behaves inconsistent with respect to module imports

[carl.dhalluin]

Hello

I am completely puzzled why the following exec code does not work:

mycode = "import math\ndef f(y):\n print math.floor(y)\nf(3.14)"
def execute():
exec mycode
execute()


I get the error:

root@devmachine1:/opt/qbase# python error1.py
Traceback (most recent call last):
File "error1.py", line 5, in ?
execute()
File "error1.py", line 4, in execute
exec mycode
File "<string>", line 4, in ?
File "<string>", line 3, in f
NameError: global name 'math' is not defined


Note that the following code _does_ work:

mycode = "import math\ndef f(y):\n print math.floor(y)\nf(3.14)"
exec mycode


I have tested this in python 2.3 and 2.4.


Regards
Carl

 

[Peter Otten]

Am Wed, 05 Sep 2007 13:12:24 +0000 schrieb (e-mail address removed):

I am completely puzzled why the following exec code does not work:

mycode = "import math\ndef f(y):\n print math.floor(y)\nf(3.14)"
def execute():
exec mycode
execute()


I get the error:

root@devmachine1:/opt/qbase# python error1.py
Traceback (most recent call last):
File "error1.py", line 5, in ?
execute()
File "error1.py", line 4, in execute
exec mycode
File "<string>", line 4, in ?
File "<string>", line 3, in f
NameError: global name 'math' is not defined

Note that the following code _does_ work:

mycode = "import math\ndef f(y):\n print math.floor(y)\nf(3.14)"
exec mycode

I have tested this in python 2.3 and 2.4.

exec breaks nested namespaces here.

Essentially

exec "import math"

puts the "math" name into the local namespace, but

"def f(): math.floor"

looks up "math" in the global namespace. On the module level global and
local namespace are identical
True

but inside a function they are distinct
False

A workaround is to declare "math" as global:
.... global math
.... import math
.... def f(y): print math.floor(y)
.... f(3.14)
.... """.... exec s
.... 3.0

or pass it explicitly:

[new interpreter session].... import math
.... def f(y, math=math): print math.floor(y)
.... f(3.14)
.... """.... exec s
.... 3.0

Peter

 

[carl.dhalluin]

Hi

Thank you for the explanation

It seems that python behaves different for variables created in the
toplevel namespace, versus deeper namespaces.

CODE:

def g():
a=2
print "a in LOC:",locals().has_key('a')
print "a in GLO:",globals().has_key('a')
def f():
print "a in LOC:",locals().has_key('a')
print "a in GLO:",globals().has_key('a')
print a
f()
g()

b=3
print "b in LOC:",locals().has_key('b')
print "b in GLO:",globals().has_key('b')
def h():
print "b in LOC:",locals().has_key('b')
print "b in GLO:",globals().has_key('b')
print b
h()


RESULT:

a in LOC: True
a in GLO: False
a in LOC: True
a in GLO: False
2
b in LOC: True
b in GLO: True
b in LOC: False
b in GLO: True
3


DISCUSSION:

locals() are passed through from a function to a nested function
locals() are not passed if you go from top-level to a nested function.

globals() are visible from all levels
globals() are auto-populated at top level

So now get back to my exec code in the previous post.
The exec applies a mixture of both rules
1. The exec looks at its namespace to apply the rule for globals().
Since I'm not at top-level, variables are not auto-populated in the
globals(), unless I add the keyword 'global' like you suggested.
2. However, the exec applies the other rule for locals(). It does NOT
copy the locals() from one level to the nested one, although it
should.

To me this seems a bug in python exec:
- exec should look at its context to find out if the exec is called
top-level or deeper
- if top-level: auto-populate globals, and do not allow inheritance of
locals to nested levels
- if non top-level: dont populate globals, but allow inheritance of
locals to nested levels.

What do you think?

 

[Peter Otten]

Am Wed, 05 Sep 2007 14:45:11 +0000 schrieb (e-mail address removed):

So now get back to my exec code in the previous post.
The exec applies a mixture of both rules
1. The exec looks at its namespace to apply the rule for globals().
Since I'm not at top-level, variables are not auto-populated in the
globals(), unless I add the keyword 'global' like you suggested.
2. However, the exec applies the other rule for locals(). It does NOT
copy the locals() from one level to the nested one, although it
should.

To me this seems a bug in python exec:
- exec should look at its context to find out if the exec is called
top-level or deeper
- if top-level: auto-populate globals, and do not allow inheritance of
locals to nested levels
- if non top-level: dont populate globals, but allow inheritance of
locals to nested levels.

I like your systematic approach.

What do you think?

Too much work

Seriously, the local namespace in a normal python function is not a
dictionary, and the decision on how to access a variable is made at
compile time. One way to achieve consistency would be to always use
dictionaries and have the lookup work its way up through the enclosing
namespaces. Of course that would kill performance...

Peter

 

[Gabriel Genellina]

En Wed, 05 Sep 2007 11:45:11 -0300, (e-mail address removed)

Thank you for the explanation

It seems that python behaves different for variables created in the
toplevel namespace, versus deeper namespaces.

Yes. Older Python versions had only two scopes: local and global. A name
was either local or global. Now, there are nested scopes too.
If a name is bound inside a function (by example, an identifier at the
left hand side of an assignment) it is local to that function (unless the
global statement is used). The check is made at compile time.
If a name is bound at the module level, it is a global variable.
Names used in a function but defined on an enclosing function are free
variables.

CODE:

def g():
a=2
print "a in LOC:",locals().has_key('a')
print "a in GLO:",globals().has_key('a')
def f():
print "a in LOC:",locals().has_key('a')
print "a in GLO:",globals().has_key('a')
print a
f()
g()

b=3
print "b in LOC:",locals().has_key('b')
print "b in GLO:",globals().has_key('b')
def h():
print "b in LOC:",locals().has_key('b')
print "b in GLO:",globals().has_key('b')
print b
h()


DISCUSSION:

locals() are passed through from a function to a nested function
locals() are not passed if you go from top-level to a nested function.

No.
A function (more generally: a code block) can refer to local variables,
free variables, or global variables.
Local variables are NOT stored in a dictionary, but on a fixed size array,
and indexed by ordinal instead of name. The compiler knows exactly which
variables are local at compile time, just looking at the code.
Free variables are accessed thru "cell" objects, which keep a reference to
the enclosing scope. The compiler knows about free variables too (they are
local to some enclosing scope). Try adding these lines at the end of g
function above:

f()
a=-1
f()

Global variables are those in the module where the function is defined
(plus some built-in names) - they're searched by name at runtime.
Local variables are, uhm, local and not "passed" anywhere.

locals() is a dictionary built on-demand. It includes local variables
*AND* free variables too.
In function f above, there are no local variables, and `a` is a free
variable.
In function g, `a` and `f` are local variables.
In function h, `b` is a local variable.

globals() are visible from all levels

Yes, unless an inner scope shadows the name. Putting b=5 at the first line
on function h above hides the global `b`

globals() are auto-populated at top level

Yes; global variables are those defined at the module level.

So now get back to my exec code in the previous post. [...] To me this
seems a bug in python exec:

Now that you know a bit more about locals and globals, try to understand
what exec does in your example.
Note that the exec statement has two optional arguments that are used as
the global and local namespaces: <http://docs.python.org/ref/exec.html>