run time code generation in python

[Carlo v. Dango]

Hello there. I have found a need to runtime generate a method and instert
it into an object instance. The code is a simple forwarding mechanism like

def foo(self, *args, **kwargs):
self.i.foo(*args, **kwargs)

method.. however, it is only at runtime that I know the name "foo" so I
cannot gerenate such a method any sooner. I have tried the compile(str)
method but I haven't succeeded. I've tried using the __getattribute__ but
there is a problem in that I do not know if __getattribute__ was called due
to a method call taking place, or due to someone who just wants a reference
to a method.. (and for other reasons as well I need to runtime generate
the above method.

hope some one has some good pointers to run time code generation in
python..

many thanks

Carlo Van Dango

 

[Duncan Booth]

Hello there. I have found a need to runtime generate a method and
instert it into an object instance. The code is a simple forwarding
mechanism like

def foo(self, *args, **kwargs):
self.i.foo(*args, **kwargs)

method.. however, it is only at runtime that I know the name "foo" so
I cannot gerenate such a method any sooner.

I don't see why you need to generate any code at runtime.
This does the same as your code, only now the name of the function to which
you are forwarding the call is stored in a variable:

name = 'foo'
def foo(self, *args, **kwargs):
getattr(self.i, name)(*args, **kwargs)

You should be aware that if you store a function in an instance at runtime,
the magic binding to 'self' won't happen. You either need to store the
function in the class, or leave out the self parameter if you want to
insert it into an instance. Something like this may be what you need:

def forwarder(obj, name):
def fwd(*args, **kwargs):
getattr(obj.i, name)(*args, **kwargs)
return fwd

something.foo = forwarder(something, 'foo')
something.foo(42) # Calls something.i.foo(42)

Alternatively:

def forwarder(obj, name):
def fwd(*args, **kwargs):
getattr(obj, name)(*args, **kwargs)
return fwd

something.foo = forwarder(something.i, 'foo')
something.foo(42) # Calls something.i.foo(42)

(I'm assuming you want a bit more in the function definition, otherwise of
course, 'something.foo=something.i.foo' is an even easier solution).

All code samples above are untested and therefore almost certainly wrong.

 

[Carlo v. Dango]

I don't see why you need to generate any code at runtime.
This does the same as your code, only now the name of the function to
which you are forwarding the call is stored in a variable:

name = 'foo'
def foo(self, *args, **kwargs):
getattr(self.i, name)(*args, **kwargs)

because there potentially could be many forwarder methods applied the
instance (forwarding to different methods.. but ofcourse are named
differently themselves)... hence it would not know where to find its
name.. hmm.. maybe if the method could somehow find out its own name, it
would know what to call on the i attribute. Is that possible?

You should be aware that if you store a function in an instance at
runtime, the magic binding to 'self' won't happen. You either need to

aahh.. ok so i can only store functions and not methods! nice point! hmm
but in this particular case I'm taking over the handling of self anyway
so it will not hurt me I think.. except that I have to get hold of the
attribute i on the instance. So that's why you use getattr() rather than
self.i

store the function in the class, or leave out the self parameter if
you want to insert it into an instance. Something like this may be
what you need:

well I need to insert it in the instance.. but yes I thought about
inserting the function in the class.. I thought I may be able to insert
the function during the __init__ execution..

(I'm assuming you want a bit more in the function definition,
otherwise of course, 'something.foo=something.i.foo' is an even easier
solution).

well, outsiders may not know that there is a forwarding taking place..
and I need only this since I have wrapper methods on top of each method
execution.. or atleast Ive made that using metaclasses.. but this is not
enough as I've found out I need to operate purely on an instance level..

All code samples above are untested and therefore almost certainly
wrong.

no problem. ;-)


-Carlo

 

[Duncan Booth]

because there potentially could be many forwarder methods applied the
instance (forwarding to different methods.. but ofcourse are named
differently themselves)... hence it would not know where to find its
name.. hmm.. maybe if the method could somehow find out its own name, it
would know what to call on the i attribute. Is that possible?

That's why my other examples all generated the wrapper function inside
another function. That way you use the nested scope variables to store
useful things like the attribute name.

 

[Terry Reedy]

Hello there. I have found a need to runtime generate a method and instert
it into an object instance. The code is a simple forwarding mechanism like

def foo(self, *args, **kwargs):
self.i.foo(*args, **kwargs)

method.. however, it is only at runtime that I know the name "foo" so I
cannot gerenate such a method any sooner. I have tried the compile(str)
method but I haven't succeeded.

compile() gives you a code object;
you probably want 'exec string' to get a function in the current
namespace
g defined

or follow other responder's advice...

Terry J. Reedy

 

[Bengt Richter]

I don't see why you need to generate any code at runtime.
This does the same as your code, only now the name of the function to which
you are forwarding the call is stored in a variable:

name = 'foo'
def foo(self, *args, **kwargs):
getattr(self.i, name)(*args, **kwargs)

You should be aware that if you store a function in an instance at runtime,
the magic binding to 'self' won't happen. You either need to store the
function in the class, or leave out the self parameter if you want to
insert it into an instance. Something like this may be what you need:

I don't quite understand the background motivation, but you can store
bound methods as attributes of arbitrary instances. They can happen to
be bound to the same instance or not (see switcheroos below):
Note that they can shadow methods of the same name, do deleting reveals
the class-defined methods.
... def __init__(self, name): self.name=name
... def foometh(self, other):
... print 'foometh bound to self named %r' %self.name
... print 'name of other instance: %r'% other.name
... foometh bound to self named 'alice'
name of other instance: 'bob' foometh bound to self named 'bob'
name of other instance: 'alice'
You can get to the unbound method via the instance's class, and re bind it
"My name is 'Bob'"

(I left the class out of those __get__ calls. Not sure when that will cause a
problem).

def forwarder(obj, name):
def fwd(*args, **kwargs):
getattr(obj.i, name)(*args, **kwargs)
return fwd

something.foo = forwarder(something, 'foo')
something.foo(42) # Calls something.i.foo(42)

Alternatively:

def forwarder(obj, name):
def fwd(*args, **kwargs):
getattr(obj, name)(*args, **kwargs)
return fwd

something.foo = forwarder(something.i, 'foo')
something.foo(42) # Calls something.i.foo(42)

(I'm assuming you want a bit more in the function definition, otherwise of
course, 'something.foo=something.i.foo' is an even easier solution).

I was wondering about that too. The larger picture is still unclear to me ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

I don't quite understand the background motivation, but you can store
bound methods as attributes of arbitrary instances. They can happen to
be bound to the same instance or not (see switcheroos below):
Note that they can shadow methods of the same name, do deleting reveals
the class-defined methods.

[... pasting-problem, sorry :-/ ]
... def __init__(self, name): self.name = name
... def speak(self): return 'My name is %r'%self.name
...

You can get to the unbound method via the instance's class, and re bind it

"My name is 'Bob'"

(I left the class out of those __get__ calls. Not sure when that will cause a
problem).

[...]

Regards,
Bengt Richter