Creating anonymous functions using eval

[Julian Smith]

I've been playing with a function that creates an anonymous function by
compiling a string parameter, and it seems to work pretty well:

def fn( text):
exec 'def foo' + text.strip()
return foo

This can be used like:

def foo( x):
print x( 2, 5)

foo( fn( '''
( x, y):
print 'x^2 + y^2 = ', x*x + y*y
return y
'''))

- which outputs:

x^2 + y^2 = 29
5


You can also mimic conventional function definitions:

f = fn( '''
( x, y):
print 'x, y=', x, y
print 1.0*x/y
return y
''')

print 'calling f'
f( 5, 6)

This outputs:

calling f
x, y= 5 6
0.833333333333


You do things like allow/require an initial `def' for clarity, check that all
the lines of text are indented so that nothing is executed when the anonymous
function is created, etc etc.

Obvious disadvantages are that the function text may have to escape quotes,
and will not be showed with syntax colouring in editors.

Apart from that, it seems quite a useful way of overcoming the limitations of
python's lambda.

But... I haven't seen this technique mentioned anywhere, so does anyone know
of any problems that I should be wary of?

- Julian

 

[Devan L]

How is this different from a nested function?

 

[Steven D'Aprano]

How is this different from a nested function?

Well, "this" is a newsgroup posting written by you. Nested functions in
Python are callable objects that exist as attributes of other callable
objects, so the two are very different.

Alternatively, if you expect a sensible answer, how about asking a
sensible question? You can start by telling us just what it is that you
are comparing to nested functions.

 

[Devan L]

Well, the string that gets passed is more or less a function
definition, which is then called with exec. I don't see why you'd need
to write a string out with the function definition and then call it.
You could just write the function.

As for the nested functions, I had been presuming that it was intended
to use as a better function than lambda within a function. Sorry for
the confusion.

 

[Robert Kern]

Well, the string that gets passed is more or less a function
definition, which is then called with exec. I don't see why you'd need
to write a string out with the function definition and then call it.
You could just write the function.

As for the nested functions, I had been presuming that it was intended
to use as a better function than lambda within a function. Sorry for
the confusion.

You missed Steven's point which is to quote the message to which you are
replying. Not everyone is reading this list in a conveniently threaded
form, so you need to provide some context for them to be able to follow
along.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Devan L]

You missed Steven's point which is to quote the message to which you are

replying. Not everyone is reading this list in a conveniently threaded
form, so you need to provide some context for them to be able to follow
along.

Ah, sorry, I didn't quite get what he was referring to.

 

[Joseph Garvin]

Not everyone is reading this list in a conveniently threaded
form

Why not? Just about every modern newsgroup reader and e-mail app has a
threaded view option.

 

[Dave Benjamin]

Why not? Just about every modern newsgroup reader and e-mail app has a
threaded view option.

My newsreader supports threading, but the first message I see in this
thread is from Devan L. I didn't notice the "Re:" and assumed his post
was half in the subject line and half in the body.

Sometimes I only download the 100 most recent posts, so I don't see the
original (or referred-to) post. Other times the threading doesn't work
properly, splitting conversations or using the wrong level of
indentation. Also, this is both a newsgroup and a mailing list, and that
has adverse effects on threading as well.

So, please quote. =)

Dave

 

[Robert Kern]

Why not? Just about every modern newsgroup reader and e-mail app has a
threaded view option.

Good point. Allow me to modify my statement: not all newsreaders/email
apps thread python-list/c.l.py conversations properly. The gateway often
messes things up.

It's also a royal pain in the butt to have to go read another message
just to dereference anaphora, threading or no.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Steven D'Aprano]

Why not? Just about every modern newsgroup reader and e-mail app has a
threaded view option.

Technology as a substitute for manners is it?

I have a modern newsgroup reader. I don't like threaded views, but even
if I did, that's not the point. News servers sometimes drop posts, or the
posts expire. Sometimes news readers lose posts (that just happened to me
yesterday). Sometimes threading breaks. Quoting enough of the previous
post to establish context is the only sensible behaviour in the face of
all these potential problems.

But even that is not the point.

It is rude for people to assume that their post is so vitally important to
me that I'll drop what I'm doing to hunt back through past posts searching
for context. Even if that search is "back one post in the thread", that's
still one post too many.

In the face of that breach of manners, people may choose to respond in
many ways. Some might choose to reward the rudeness by searching previous
posts, then responding with an answer to the question. Some might choose
to just ignore the post, which has the disadvantage of leaving the
original poster no wiser and likely to repeat his behaviour. Some might
flame them, which is usually counterproductive. And some might drop some
fairly heavy hints, but that assumes the poster is capable of getting a
clue.

 

[Julian Smith]

I've been playing with a function that creates an anonymous function by
compiling a string parameter, and it seems to work pretty well:

def fn( text):
exec 'def foo' + text.strip()
return foo

This can be used like:

def foo( x):
print x( 2, 5)

foo( fn( '''
( x, y):
print 'x^2 + y^2 = ', x*x + y*y
return y
'''))

- which outputs:

x^2 + y^2 = 29
5

]

How is this different from a nested function?

It's different because it is defined in an expression, not by a
statement. So you can create a new function inside a function call, for
example. Like I said in the original post, it's a way of overcoming the
limitations of python's lambda.

The simple version I posted fails to lookup things in the caller's context,
which one can easily fix by passing locals() as an extra parameter. Hopefully
it'll be possible to grab the caller's locals() automatically using traceback
or similar.

Here's a version I've been using for a while which seems to work pretty
well:

def fn( text, globals_={}, locals_={}):
'''
Returns an anonymous function created by calling exec on <text>.
<text> should be a function definition, ommiting the initial
`def <fnname>' (actually, leading `def' can be retained, for
clarity). Any leading/trailing white space is removed using
str.strip(). Leading white space on all lines will be handled
automatically by exec, so you can use an indented python
triple-quoted string.

In addition, newlines and backslashes are re-escaped inside
single/double-quoted strings. this enables nested use of fn().

example usage:

fn(
"""
def ( target, state):
if target != 'foo': return None
return [], None, 'touch foo'
""", globals(), locals())

Would be nice to grab our caller's globals() and locals() if
globals_/locals_ are None, using some sort of introspection. For
now, you'll have to pass globals() and locals() explicitly.
'''
text = text.strip()
if text.startswith( 'def'): text = text[ 3:].strip()
if not text.startswith( '('):
raise Exception( 'fn string must start with `(\'')

def _escape_quotes( text):
'''
escape newlines and backslashes that are inside
single/double-quoted strings. should probably do something about
backslashes inside triple-quoted strings, but we don't bother
for now.
'''
quote = None
ret = ''
for c in text:
if quote:
if c=='\n':
ret += '\\n'
elif c=='\\':
ret += '\\\\'
else:
if c==quote:
quote = None
ret += c
else:
if c=='\'' or c=='"':
quote = c
ret += c
return ret


text = _escape_quotes( text)

#print 'execing:', text

# the exec will put the fn in the locals_ dict. we return it after
# removing it from this dict.
exec 'def _yabs_fn_temp' + text in globals_, locals_
ret = locals_['_yabs_fn_temp']
del locals_['_yabs_fn_temp']
return ret


- Julian