trouble with nested closures: one of my variables is missing...

[Cameron Simpson]

I'm having some trouble with closures when defining a decorator.

TL;DR:
I have a function that makes a decorator, and only some of the names
from an outer scope appear in the inner closure's locals().
And I do not understand why at all.

Let me explain...

Environment: python 2.7.3 on MacOSX Mountain Lion from MacPorts.

Background: I have a decorator called "file_property" that watches a
file for changes and reloads the file at need. Otherwise it returns a
cached value. It has a lock and some sanity checks and works quite well.

Example method:

class myclass(object):
@file_property
def rules(self):
with open(self._rules_path) as rfp:
R = parse(rfp)
return R
C = myclass()

and using C.rules fetches parses the rule file as needed and caches the
value until the file next changes.

I naively wrote file_property with a bunch of default parameters:

def file_property(func, attr_name=None, unset_object=None, poll_rate=1):

and because I never overrode these failed to realise they're useless. Fine.

So, I'm rewriting file_property like this:

def file_property(func):
return make_file_property()(func)

i.e. it makes a "vanilla" file_property using the default internals.
Now I have make_file_property with the default parameters:

def make_file_property(attr_name=None, unset_object=None, poll_rate=1):

which I might use as:

class myclass(object):
@make_file_property(poll_rate=3)
def rules(self):
with open(self._rules_path) as rfp:
R = parse(rfp)
return R

The inner function is the same, but it won't reload the file more often
that once every 3 seconds.

However, I can't make my make_file_property function work. I've stripped
the code down and it does this:

[hg/css-mailfiler]fleet*1> python foo.py
make_file_property(attr_name=None, unset_object=None, poll_rate=1): locals()={'attr_name': None, 'poll_rate': 1, 'unset_object': None}
made_file_property(func=<function f at 0x10408b0c8>): locals()={'func': <function f at 0x10408b0c8>, 'unset_object': None}
Traceback (most recent call last):
File "foo.py", line 21, in <module>
def f(self, foo=1):
File "foo.py", line 4, in file_property
return make_file_property()(func)
File "foo.py", line 10, in made_file_property
if attr_name is None:
UnboundLocalError: local variable 'attr_name' referenced before assignment

Observe above that 'unset_object' is in locals(), but not 'attr_name'.
This surprises me.

The stripped back code (missing the internals of the file property
watcher) looks like this:

import sys

def file_property(func):
return make_file_property()(func)

def make_file_property(attr_name=None, unset_object=None, poll_rate=1):
print >>sys.stderr, "make_file_property(attr_name=%r, unset_object=%r, poll_rate=%r): locals()=%r" % (attr_name, unset_object, poll_rate,locals())
def made_file_property(func):
print >>sys.stderr, "made_file_property(func=%r): locals()=%r" % (func, locals())
if attr_name is None:
attr_name = '_' + func.__name__
lock_name = attr_name + '_lock'
def getprop(self):
with getattr(self, lock_name):
# innards removed here
pass
return getattr(self, attr_name, unset_object)
return property(getprop)
return made_file_property

@file_property
def f(self, foo=1):
print "foo=%r" % (foo,)

@make_file_property(attr_name="_blah")
def f2(self, foo=2):
print "foo=%r" % (foo,)

Can someone explain what I'm doing wrong, or tell me this is a python
bug?
--
Cameron Simpson <[email protected]>

Bolts get me through times of no courage better than courage gets me
through times of no bolts!
- Eric Hirst <[email protected]>