Logging global assignments

[Jacek Generowicz]

I am dealing with an application which reads in configurations by
calling (a wrapper around) execfile. Any configuration file may itself
execfile other configuration files in the same manner.

I would like to create a log containing all global assignments made in
these files. Comparing the globals dictionaries before and after is
not quite enough, as

a) this misses multiple assignments to the same name, within a single
file,

b) doesn't allow me to pinpoint the line at which the assignment is
made.

Inspecting the implementation of execfile suggests to me that the
assignments are performed by a hard-wired call to PyDict_SetItem, in
the opcode implementations, so it looks like ideas based on giving
execfile a globals dictionary with an instrumented __setitem__ are
probably doomed to failure.

Is there any way of injecting some of my code into the process of
global name binding, or some other means of "capturing" global
assignments ?

 

[Larry Bates]

I am dealing with an application which reads in configurations by
calling (a wrapper around) execfile. Any configuration file may itself
execfile other configuration files in the same manner.

I would like to create a log containing all global assignments made in
these files. Comparing the globals dictionaries before and after is
not quite enough, as

a) this misses multiple assignments to the same name, within a single
file,

b) doesn't allow me to pinpoint the line at which the assignment is
made.

Inspecting the implementation of execfile suggests to me that the
assignments are performed by a hard-wired call to PyDict_SetItem, in
the opcode implementations, so it looks like ideas based on giving
execfile a globals dictionary with an instrumented __setitem__ are
probably doomed to failure.

Is there any way of injecting some of my code into the process of
global name binding, or some other means of "capturing" global
assignments ?

Suggestion:
Use ConfigParser to set your globals instead and you can track
your assignments over each file, section and option in the file.

Larry Bates

 

[Jacek Generowicz]

Suggestion:
Use ConfigParser to set your globals instead and you can track
your assignments over each file, section and option in the file.

I would dearly love to go down this sort of route but, unfortunately,
they are not _my_ configuration files, and it is not _my_
configuration mechanism ... it's just the one I have to work with. I
do not have the freedom to change it; I merely have to try to make
some sense out of it :-(

 

[Peter Otten]

Inspecting the implementation of execfile suggests to me that the
assignments are performed by a hard-wired call to PyDict_SetItem, in
the opcode implementations, so it looks like ideas based on giving
execfile a globals dictionary with an instrumented __setitem__ are
probably doomed to failure.

For Python 2.4 here is evidence against that assumption:

$ cat config.py
a = 42
b = "so what"

.... def __setitem__(self, k, v):
.... print k, "->", v
.... dict.__setitem__(self, k, v)
....a -> 42
b -> so what

We are all doomed to succeed, then...

Peter

 

[brianmce]

In 2.4 at least, it looks like execfile can take an arbirary mapping
object for globals, so something like this may work:

class LoggedDict(dict):
def __setitem__(self, item, val):
dict.__setitem__(self, item, val)
# Do whatever logging is needed here
print "Assignment made: %s = %r" % (item, val)

# When loading your config file use:
logged_globals = LoggedDict()
execfile(my_config_file, logged_globals)

Note that this covers all name binding (ie. function and class
definitions etc), not just assignments.

This might not work with earlier versions of python, since I think
these may restrict the globals argument to a real dictionary.