One function calling another defined in the same file being exec'd

[Mitchell L Model]

[Python 3.1]

I thought I thoroughly understood eval, exec, globals, and locals, but I
encountered something bewildering today. I have some short files I
want to
exec. (Users of my application write them, and the application gives
them a
command that opens a file dialog box and execs the chosen file. Users
are
expected to be able to write simple Python scripts, including function
definitions. Neither security nor errors are relevant for the purposes
of this
discussion, though I do deal with them in my actual code.)

Here is a short piece of code to exec a file and report its result.
(The file
being exec'd must assign 'result'.)

def dofile(filename):
ldict = {'result': None}
with open(filename) as file:
exec(file.read(), globals(), ldict)
print('Result for {}: {}'.format(filename, ldict['result']))

First I call dofile() on a file containing the following:

################################
def fn(arg):
return sum(range(arg))

result = fn(5)
################################

The results are as expected.

Next I call dofile() on a slightly more complex file, in which one
function
calls another function defined earlier in the same file.

################################
def fn1(val):
return sum(range(val))

def fn2(arg):
return fn1(arg)

result = fn2(5)
################################

This produces a surprise:

NameError: global name 'fn1' is not defined

[1] How is it that fn2 can be called from the top-level of the script
but fn1
cannot be called from fn2?

[2] Is this correct behavior or is there something wrong with Python
here?

[3] How should I write a file to be exec'd that defines several
functions that
call each other, as in the trivial fn1-fn2 example above?

 

[anon]

Rather than exec the files, why not import them?

I can get both your examples to work using the 'imp' module.
http://docs.python.org/3.1/library/imp.html#module-imp

I used python 2.6.4. Note that 3.1 also has 'importlib' module.


import imp

# the name of the python file written by a user
name = 'test1'

fp, pathname, description = imp.find_module(name)
test1 = imp.load_module(name, fp, pathname, description)

print test1.result

# remember to close file (see docs)
fp.close()

 

[Steven D'Aprano]

Next I call dofile() on a slightly more complex file, in which one
function calls another function defined earlier in the same file.

################################
def fn1(val):
return sum(range(val))

def fn2(arg):
return fn1(arg)

result = fn2(5)
################################

This produces a surprise:

NameError: global name 'fn1' is not defined

[1] How is it that fn2 can be called from the top-level of the script
but fn1 cannot be called from fn2?

This might help you to see what's going on. Define your own cut-down
version of the global namespace, and a local namespace, and a string to
execute:


myglobals = {'__builtins__': None, 'globals': globals, 'locals': locals,
'print': print}
mylocals = {'result': None}
s = """def f():
print("Globals inside f:", globals())
print("Locals inside f:", locals())

print("Globals at the top level:", globals())
print("Locals at the top level:", locals())
f()
"""

exec(s, myglobals, mylocals)



And this is what you should see:


Globals at the top level: {'__builtins__': None, 'print': <built-in
function print>, 'globals': <built-in function globals>, 'locals': <built-
in function locals>}
Locals at the top level: {'result': None, 'f': <function f at 0xb7ddeeac>}
Globals inside f: {'__builtins__': None, 'print': <built-in function
print>, 'globals': <built-in function globals>, 'locals': <built-in
function locals>}
Locals inside f: {}


Does that clarify what's going on?

[2] Is this correct behavior or is there something wrong with Python
here?

This certainly surprised me too. I don't know if it is correct or not,
but it goes back to at least Python 2.5.

[3] How should I write a file to be exec'd that defines several
functions that call each other, as in the trivial fn1-fn2 example above?

My preference would be to say, don't use exec, just import the module.
Put responsibility on the user to ensure that they set a global "result",
and then just do this:

mod = __import__('user_supplied_file_name')
result = mod.result


But if that's unworkable for you, then try simulating the namespace setup
at the top level of a module. The thing to remember is that in the top
level of a module:
True

so let's simulate that:


myglobals = {'result': None} # You probably also want __builtins__
s = """def f():
return g() + 1

def g():
return 2

result = f()
"""
exec(s, myglobals, myglobals)
myglobals['result']


This works for me.