How do I dynamically create functions without lambda?

R

Russell

I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)
 
R

Robert Kern

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)

It doesn't exist, yet. Python 3000 isn't even in planning stages, yet. There are
just some loose ideas floating around about what will (and won't!) be in it.

You can't write Python 3000 compliant code right now because there is nothing to
comply with.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
P

Paul Rubin

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated.

Nobody knows yet what Python 3000 will change, so relax.
 
K

Kay Schluehr

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)

If you want to code partial application without lambda I recommend
using the code presented in the accepted PEP 309 that will be
implemented in the functional module in Python 2.5.

http://www.python.org/peps/pep-0309.html

Kay
 
J

James Stroud

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)

Is this what you mean?

# making functions partially bound with foo, dynamically

def make_newfunc(foo):
def _newfunc(x, f=foo):
print x, foo
return _newfunc

foo = 42

newfunc = make_newfunc(foo)

newfunc()
 
J

James Stroud

Kay said:
If you want to code partial application without lambda I recommend
using the code presented in the accepted PEP 309 that will be
implemented in the functional module in Python 2.5.

http://www.python.org/peps/pep-0309.html

Kay
For anyone who got my last post: sorry, typos....


def f(foo, x):
print foo, x

def make_newfunc(foo):
def _newfunc(x, foo=foo):
f(foo, x)
return _newfunc

foo = 42 # or "dynamically generated"

newfunc = make_newfunc(foo)

newfunc(14) # output will be "42 14"

newfunc2 = make_newfunc(69)

newfunc(21) # output will be "69 21"


If this doesn't fit your needs, then please elucidate.
 
G

Gary Herron

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.
That's nonsense! A lambda function *IS* a nested function, and has *NO*
extra capabilities over a nested function. If you can do something with
a lambda, you can also do it with a nested function defined at the same
point. However, a nested function *does* give you several extra
capabilities: (1) it has a name rather than being anonymous like the
lambda, and (2) the body can use statements rather just one expression.

So use a nested function. You'll get the same capabilities, plus a
name, plus more expressive power in the body. You won't lose anything.
and your code will survive any eventual removal of the lambda functionality.
 
S

Steven Bethard

Russell said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

I don't understand this argument here. The code above is almost exactly
equivalent to:

def newfunc(x):
return f(foo, x)
someobject.newfunc = newfunc

Observe:
.... pass
.... 1 0 LOAD_GLOBAL 0 (f)
3 LOAD_GLOBAL 1 (foo)
6 LOAD_FAST 0 (x)
9 CALL_FUNCTION 2
12 RETURN_VALUE.... return f(foo, x)
.... 2 0 LOAD_GLOBAL 0 (f)
3 LOAD_GLOBAL 1 (foo)
6 LOAD_FAST 0 (x)
9 CALL_FUNCTION 2
12 RETURN_VALUE

Note that both the lambda and the function have exactly the same
byte-code. The only real difference is that if you use a def-statement
instead of a lambda, your function will get a real name, "newfunc",
instead of <lambda>.

STeVe
 
S

Steven D'Aprano

If you want to code partial application without lambda I recommend
using the code presented in the accepted PEP 309 that will be
implemented in the functional module in Python 2.5.

http://www.python.org/peps/pep-0309.html


Fascinating. A couple of thoughts:

- I really wish that people would make up their minds about what currying
is, and how it differs from partials and closures. If the people who
do know can't agree, how do they expect the rest of us to understand?

- What, if anything, is the difference between a closure and an iterator?
Is an iterator just a closure wrapped up with an API?

- The first sample code seems needlessly confusing to me. It gives:

class partial(object):
def __init__(*args, **kw):
self = args[0]
self.fn, self.args, self.kw = (args[1], args[2:], kw)
def __call__(self, *args, **kw):
if kw and self.kw:
d = self.kw.copy()
d.update(kw)
else:
d = kw or self.kw
return self.fn(*(self.args + args), **d)

It seems to me to needlessly break the convention that the first argument
is self, to no benefit and considerable reduction in clarity. After some
experimentation, I worked out what it was doing, and realised that it
would work just as well but much less obscurely if the __init__ function
was written as:

def __init__(self, fn, *args, **kw):
self.fn, self.args, self.kw = fn, args, kw


- It seems a shame to me that having created a partial _function_ using
that technique, type(partial(...)) returns <class partial>. It would be
nicer if the type made it more obvious that the instance was callable.
Something like <callable class partial> perhaps? Is there any way for a
class to customise the type representation?
 
S

Scott David Daniels

Steven said:
Fascinating. A couple of thoughts:

- I really wish that people would make up their minds about what currying
is, and how it differs from partials and closures. If the people who
do know can't agree, how do they expect the rest of us to understand?

Me too, but I disagree with the people who won the argument on the name.
Curry derives its name from the mathematician who formulated the
Curry-Howard isomorphism that says a function of multiple arguments
can be expressed as a function taking a single argument which returns a
function which takes a single argument which... takes a single argument
and returns the result of the original multi-argument function. This
isomorphism means that you need only define functions which take single
arguments. In some functional languages, functions are called adjacent
to their arguments, and a function which looks like it takes multiple
arguments is actually taking a single argument of a tuple.

The Curry recipe I put in the Python Cookbook did the analogous thing
for the Python language, since there is no way for a "Curry" argument
to know when to call the underlying function, rather than simply
accumulate arguments.
- What, if anything, is the difference between a closure and an iterator?
Is an iterator just a closure wrapped up with an API?

A closure is a function and the environment it was created in.
So, for example, the result of calling:

def multiplier(n):
def result(m):
return n * m

multiplier(2) returns a closure (the function called result and its
environment where n is set to 2).

An iterator is a different beasty.
- The first sample code seems needlessly confusing to me. It gives:

class partial(object):
def __init__(*args, **kw):
self = args[0]
self.fn, self.args, self.kw = (args[1], args[2:], kw)
def __call__(self, *args, **kw):
if kw and self.kw:
d = self.kw.copy()
d.update(kw)
else:
d = kw or self.kw
return self.fn(*(self.args + args), **d)

It seems to me to needlessly break the convention that the first argument
is self.
Actually, it does, but only because the second doesn't break it as well.
> to no benefit and considerable reduction in clarity. After some
experimentation, I worked out what it was doing, and realised that it
would work just as well but much less obscurely if the __init__ function
was written as:

def __init__(self, fn, *args, **kw):
self.fn, self.args, self.kw = fn, args, kw

It would pretty much work like this, _but_ you could not used named args
of self or fn when using one of these things.

A better definition:
class partial(object):
def __init__(*args, **kw):
self = args[0]
self.fn, self.args, self.kw = (args[1], args[2:], kw)
def __call__(*args, **kw):
self = args[0]
if kw and self.kw:
d = self.kw.copy()
d.update(kw)
else:
d = kw or self.kw
return self.fn(*(self.args + args[1:]), **d)

Now you can, for example:

def function(self, other, fn):
return fn([self, other])

defaults = partial(function, self=object(), fn=repr)
print defaults(other=3), defaults(self='self', other=3)

The fancy-schmancy stuff is to keep from hiding some names. I suspect
both args and kw have this problem as well. It might have been
reasonable to call them __self, __function, __args, and __kwargs
and do it your way.
- It seems a shame to me that having created a partial _function_ using
that technique, type(partial(...)) returns <class partial>. It would be
nicer if the type made it more obvious that the instance was callable.
But callable(defaults) returns True. You will eventually learn that
partial is callable, just as you know "type"s are callable. Seeing
that it is class partial, you can (if you are nosey), see what the
arguments already provided are (so you could get a nicer print).
For example, adding a method to partial:

def __repr__(self):
return 'partial(%r, *%r, **%r)' % (self.fn, self.args, self.kw)
 
K

Kay Schluehr

Steven said:
Fascinating. A couple of thoughts:

- I really wish that people would make up their minds about what currying
is, and how it differs from partials and closures. If the people who
do know can't agree, how do they expect the rest of us to understand?

If they can't agree they probably don't know ;)

Concepts like "closure" and "currying" are well defined in lambda
calculus and hence in functional programming.
- What, if anything, is the difference between a closure and an iterator?
Is an iterator just a closure wrapped up with an API?

I would expect a closure to be a function with free/unbound variables
that are bound by an enclosing context. In FP slang a function without
any free variables is called a "combinator". An iterator is a function
related to an abstract data type equipped with a partial order. The
iterator abstracts from the particular ordering scheme and makes the
ADT look like a list. In Python we have a nice correspondence between
generators as implicit defined sequences of yielded values, iterators
that enable uniform access to the values produced by generators and
lists that make the sequence explicit ( if finite ).

Fun with combinators
======================
There are quite interesting relationships deep down in the theory of
computation. For instance understanding recursion in lambda calculus is
not that easy because you cannot simply reintroduce an anonymous
function by the name into its own body. The idea is to create a
fixpoint of a higher order curried function instead. See for example
this curried extension of a factorial:

F = lambda h: lambda n : n<2 and 1 or n*h(n-1)

Now imagine you have a fixpoint fix(F) of F which is again a function
and pass it into F then you will get:

fix(F) = F(fix(F)) = lambda n : n<2 and 1 or n*fix(F)(n-1)

That is fix(F) is actually the searched factorial ! There is a generic
way to create fixpoints of higher order curried functions such as F.
The most popular one is the so called Y-combinator ( not to confuse
with Paul Grahams company ;).

In Python you can write:

Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
arg: f(f)(arg)))

This serves the purpose. Try Y(F) and see.

Kay
 
S

Steven D'Aprano

On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:

[snip lambda calculus stuff]
In Python you can write:

Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
arg: f(f)(arg)))

This serves the purpose. Try Y(F) and see.


Is any of this stuff maintainable in the real world of IT, where
most programmers don't have computer science degrees? You come along six
months after the project was finished to maintain this code and discover
that the whiz-kid lambda calculus guy never commented anything because
that would detract from the elegance of his one liners; what happens next?
 
K

Kay Schluehr

Steven said:
On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:

[snip lambda calculus stuff]
In Python you can write:

Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
arg: f(f)(arg)))

This serves the purpose. Try Y(F) and see.


Is any of this stuff maintainable in the real world of IT, where
most programmers don't have computer science degrees?

Probably not. But the good thing about Y is that it is in closed form
and the expression is not infinitely long. By the way I have no less a
hard time to read C code with advanced data-structures and many type
casts which is much more likely to happen in the "real world".
From what I've seen the only *practical* purpose fixpoint combinators
serve is a kind of recursion "overloading". Without modifying F one can
memoize values by adapting the fixpoint combinator. We already know
something similar from the Go4 "command pattern" but there is no
self-referential entanglement and it is less general.
You come along six
months after the project was finished to maintain this code and discover
that the whiz-kid lambda calculus guy never commented anything because
that would detract from the elegance of his one liners; what happens next?

Ask a Scheme guy at LtU. Accuse the original author in Den Haag and if
caught put him to shame by making photos at a dentist. Google for
"lambda calculus Y". Ask someone here at comp.lang.python. Ask your
boss for more time ;)

Kay
 
A

Antoon Pardon

Op 2006-01-27 said:
I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newfunc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

I don't see how dynamic is supposed to contradict with
nested functions. I assume from your tekst that foo
has some parameter-like charateristics. So the following
should work.

def produce(foo):

def func(x):
return f(foo, x)

return func


someobject.newfunc = produce(foo)


This kind of code can be generalized. Look for partial
application or curry.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)

The new and shiny, pythonic way depends on what you
really want, your question was a bit vague to answer
that.
 
A

Alan Morgan

On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:

[snip lambda calculus stuff]
In Python you can write:

Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
arg: f(f)(arg)))

This serves the purpose. Try Y(F) and see.


Is any of this stuff maintainable in the real world of IT, where
most programmers don't have computer science degrees? You come along six
months after the project was finished to maintain this code and discover
that the whiz-kid lambda calculus guy never commented anything because
that would detract from the elegance of his one liners; what happens next?

Excessive cleverness can lead to unmaintainable code. So can excessive stupidity.
Since there are a lot more stupid people than clever people out there I think
the more likely scenario is having to maintain unmaintainable code written by a
complete idiot whose programming knowledge comes solely from books whose titles
end with "In 7 Days".

Oh, and I'd hope that code reviews, etc. would have kept lambda-boy from getting
too far down this path of self invocation.

Alan
 
A

Alex Martelli

Alan Morgan said:
Excessive cleverness can lead to unmaintainable code. So can excessive
stupidity.

+1 QOTW.
Since there are a lot more stupid people than clever people out there I
think the more likely scenario is having to maintain unmaintainable code
written by a complete idiot whose programming knowledge comes solely from
books whose titles end with "In 7 Days".

Disagree -- far more people THINK they're clever, than really ARE
clever. According to a recent article in the Financial Times, over 40%
of a typical financial firm's employees firmly believe they are among
the 5% best employees of the firm -- and the situation, believe me, is
no different in programming.


Alex
 
S

Steve Holden

Alex said:
+1 QOTW.
++



Disagree -- far more people THINK they're clever, than really ARE
clever. According to a recent article in the Financial Times, over 40%
of a typical financial firm's employees firmly believe they are among
the 5% best employees of the firm -- and the situation, believe me, is
no different in programming.

Just as 85% of drivers believe their skill level is "above average".

regards
Steve
 
A

Alan Morgan


import blush
Disagree -- far more people THINK they're clever, than really ARE
clever.

No doubt about it, but I don't see how it contradicts my statement.
According to a recent article in the Financial Times, over 40%
of a typical financial firm's employees firmly believe they are among
the 5% best employees of the firm -- and the situation, believe me, is
no different in programming.

I wonder if python, which has a low barrier to entry due to its simple syntax
and general readability, might not have a worse time of this than other languages.

Alan
 
A

Alex Martelli

Benji York said:

Tx. Maybe it's a genetically adaptive trait: the FT points out that
while there is little correlation between an employee's opinion of their
skills, and the employee's actual performance, there IS positive
correlation between said opinion and the employee's career advancement.
One way to explain the correlation is: if you truly (albeit perhaps
mistakenly) believe you're great, you project that and are more likely
to get promotions or raises, than is somebody else, perhaps objectively
better, who's torn by self-doubt and projects THAT.

My working hypothesis would be that this effect would be far stronger in
fields where it's hard to get hard quantitative measures of performance,
so the managers (or clients, etc) rely to a large extent on subjective
judgment which can be influenced by such "projections"...


Alex
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,283
Messages
2,571,405
Members
48,100
Latest member
Calfin5299

Latest Threads

Top