Ideas for Python 3

[David MacQuigg]

I am starting a new thread so we can avoid some of the non-productive
argument following my earlier post "What is good about Prothon". At
Mr. Hahn's request, I will avoid using the name "Prothon" in the
subject of any post to this newsgroup. Please ignore the old thread.

I've also updated my webpage http://ece.arizona.edu/~edatools/Python
Anyone with some good ideas for "Python 3" is welcome to contribute.
I hope GvR won't sue me for calling it "Python 3" :>)

Here is the original proposal:

< snip criticism of Prothon syntax >

... There are a number of
aspects to this simplification, but for me the unification of methods
and functions is the biggest benefit.

All methods look like functions (which students already understand).
Prototypes (classes) look like modules. This will make teaching OOP
much simpler, especially for the students and professional engineers
(non-CIS) that I am most concerned about. I teach electronic design
tools, not programming. Current plans are to include some basic
Python, no OOP. If I could add OOP with another 4 hours, I would do
it.

I've written a proposal for a prototype syntax that I believe captures
the essense of what is good in Prothon, while not departing too
radically from Python. see PrototypeSyntax.htm at
http://ece.arizona.edu/~edatools/Python/ I would like to get
feedback from Python experts on the potential problems with this
syntax. The major question is automatic translation of existing Python
programs to the new syntax. I'm one of those users who would not give
up the existing libraries in Python, no matter how good some
alternative language may be.

I would also like to get feedback from users on what they like or
dislike about this proposal. I will summarize this feedback in the
"Pros and Cons" section of the proposal.

Below are some excerpts from the syntax proposal. Please see the link
above for better formatting.

-- Dave

Proposed Prototype Syntax
=========================
< snip >

Example of Simplified Classes ( Prototypes )
============================================

Animal --> Mammal --> Feline --> Cat
------- ------- ------- -------
numAnimals numMammals numFelines numCats
home genus
__init__() __init__() __init__() __init__()
.sound .sound
.name
show() show() show() show()
talk() talk()


proto Animal(object): # Inherit from the primitive object.
numAnimals = 0
home = "Earth"
....
<see the "OOP Chapter" at http://ece.arizona.edu/~edatools/Python/
for the complete example.>
....
proto Cat(Feline):
numCats = 0
__init__ n = "unknown", s = "Meow" ):
Feline.__init__()
Cat.numCats += 1
.name = n # Set instance variables.
.sound = s
show ): # Define a "static method".
Feline.show()
print " Cats:", Cat.numCats
talk ):
print "My name is ...", .name
print "I am a %s from %s" % (.genus, .home)
Mammal.talk() # Call an unbound function.
print __self__ ### Diagnostic check.

cat1 = Cat() # Create instance.
with cat1: # Modify instance variables.
home = "Tucson"
genus = "feline"
name = "Garfield"
My name is ... Garfield
I am a feline from Tucson
Mammal sound: Meow

Changes from Current Python
===========================
-- Keyword class is changed to proto.
-- All methods have the same form, identical to a simple function.
-- Function header lines are re-arranged. def --> : Parentheses are
optional.
-- self is replaced with a leading dot, and eliminated from the arg
list.
-- Current instance is available if ever needed via __self__
-- Instances can have their attributes modified in a with block.

Benefits of Proposed Syntax
===========================
-- Unification of all function forms ( bound, unbound, static, class,
lambda ). All will have the same form as a normal function
definition. This will make it easier to teach OOP. Students will
already understand functions and modules. OOP is a small step up. A
prototype will look just like a module ( except for the instance
variables ). See Parallels between Prototypes and Modules below.

-- Using an explicit __self__ variable avoids the magic first
argument, and makes it easier to explain instance variables. See the
sections below comparing a brief explanation of instance variables in
Python vs the simplified form. A full presentation of OOP, like pages
295-390 in Learning Python, 2nd ed. will likely be 1/2 the number of
pages. Not only is the basic presentation simpler, but we can
eliminate a lot of discussion of lambda functions, static methods,
etc.

-- All attributes of a prototype ( both data and functions ) will be
in a neat column, making it easier to find a particular attribute when
visually scanning a program. Understanding the structure of a program
will be almost as quick as seeing a UML diagram.

-- Lambda keyword will be gone. An anonymous function using normal
function syntax can be extremely compact. ( :x,y:x+y )

-- Method definitions will be less cluttered and less typing with
__self__ as a hidden variable.

-- Changing numerous attributes of an instance will be more
convenient. ( need use case )

-- Migration of Python programs to Python 3 will be automatic and
reliable. ???

Pros and Cons of the Proposed Syntax
====================================
Classlessness

Con: The proposed syntax is not purely classless. This is important
because ... ???

Unification of Methods and Functions

Pro1: Less to learn.

Con1: Experts don't care. Beginners don't need advanced method
syntax.

Pro2: Replace lambdas with standard function syntax.

Con2: ???

Explicit __self__

Pro1: Allows the unification of methods and functions.

Con1: ???

Pro2: Explanation of instance variables is simpler.

Con2: Using __self__ instead of a special first argument is less
explicit.

Pro3: Less typing and less clutter in method definitions.

Con3: Can use "s" or "_" instead of "self" to minimize typing and
clutter.

"Assignment" Syntax for Function Definitions

Pro1: See all the variables at a glance in one column.

Con1: ???

Pro2: Emphasize the similarity between data and functions as
attributes of an object.

Con2: ???

Symbol instead of def Keyword

Pro: Allows lambda functions to be included in the unification.

Con: Symbols are never as clear as keywords.

With Block

Pro: Saves typing the object name on each line.

Con: Making it too easy to modify prototypes after they have been
created will lead to more undisciplined programming.

Issues relevant to teaching OOP
===============================
Parallels between Prototypes and Modules
----------------------------------------
Ninety percent of what students need to know about prototypes is
already understood from their study of functions and modules. Even
some tricky issues are best explained by comparing to a parallel
situation with modules.
< snip >

Explanation of Instance Variables in Python
-------------------------------------------
""" Some of the variables inside the functions in a class have a
self. prefix. This is to distinguish local variables in the function
from "instance variables". These instance variables will be found
when the function is called, by searching the instance which called
the function. The way this works is that calling the function from an
instance causes that instance to be passed as the first argument to
the function call. So if you call cat1.talk(), that is equivalent to
Cat.talk(cat1) If you call cat1.set_vars( "Garfield", "Meow"), that is
equivalent to Cat.set_vars(cat1, "Garfield", "Meow")

The "current instance" argument is auto-magically inserted as the
first argument, ahead of any other arguments that you may provide in
calling a method that is "bound" to an instance. Note: The
distinction between instances and classes is important here. If you
call a function from a class, that function is not bound to any
instance, and you have to supply the instance explicitly in the first
argument ( Cat.talk(cat1) )

The variable name self is just a convention. As long as you put the
same name in the first argument as in the body of the definition, it
can be self or s or even _ The single underscore is handy if you
want to maximally suppress clutter. """

Explanation of Simplified Instance Variables
--------------------------------------------
""" Some of the variables inside the functions in a prototype have a
leading dot. This is to distinguish local variables in the function
from "instance variables". When a function is called from an instance
( cat1.talk() ) a special global variable __self__ is automatically
assigned to that instance ( __self__ = cat1 ) Then when the function
needs an instance variable ( .sound ) it uses __self__ just as if you
had typed it in front of the dot ( __self__.sound ) The leading dot
is just an abbreviation to avoid typing __self__ everywhere. """

========== END ==============

 

[David MacQuigg]

Mike, thanks for a very thorough and thoughtful review of this
proposal.

David MacQuigg wrote:
...

I think it might be more proper to say that you've made all functions
methods with an implicitly defined target, but I suppose the statement
is still technically true.

A function with no instance variables is identical to a function
defined outside a class. In that case, the only way to tell if it is
a method or a function is to look at the surrounding code. "Method"
is one of those mystery words that tend to put off non-CIS students.
I use it when it is necessary to distinguish a method from a function,
but otherwise I prefer the term function. It emphasizes the
similarity to what the students already know.

This is nice. Not "I'm going to rush out to adopt a language because of
it" nice, but nice enough. I'm curious about one thing:

proto x( object ):
flog x, y ):
.x = x
a = x()
b = x()
a.flog = b.flog
a.flog()
print b.x

In other words, how do I hold a reference to a bound method/function if
there are no such things and only the "last access" determines what the
implicit target is? Just to be clear, I'm assuming you're going to have
storage *somewhere* so that:

a = module.do
a()

works.

Bound and unbound functions work just like in Python. This is where I
differ with Prothon, on the need for special binding syntax.

This is a wash IMO, with the explicit "self" having a slight edge on
"Explicit is better than Implicit" grounds. You now have to explain
where the magic __self__ comes from instead of how self is bound when
you access the instance's method. They're both magic, the Python stuff
is just explicitly visible. Still, since you're coding it deep into
this new language, it'll be first nature to the Whateverthon programmer.

On a personal note, the moment where I "got" the concept of methods
(Python was my first OO language) was seeing "self" in the argument list
of a function and realising that it's just a parameter curried into the
function by doing x.method lookup. That is, it just looked like any
other function, the parameter was just a parameter, nothing special,
nothing requiring any extra knowledge save how it got bound (and that's
pretty darn simple). Coming from a structures+functions background it
made complete sense.

I assume no background other than what the students will know from
studying Python up to the point of introducing OOP. At this point,
they have a good understanding of functions and global variables.

I've seen a lot of discussion on the "explicitness" of self in Python,
and I have to conclude that most of it is missing the real problem,
which is complexity from the fact that some functions have a special
first argument and others don't. It is hard to compare alternatives
by focusing our microscope on something as small as setting a global
variable vs inserting a special first argument.

What I would do in comparing complexity is look at the length of basic
but complete "textbook explanations" of the alternatives. I've copied
at the end of this post the explanation of instance variables from my
OOP chapter at http://ece.arizona.edu/~edatools/Python/ I've also
made my best effort to write an equivalent explanation of Python's
instance variables. Comments are welcome. Also, if anyone can write
a better explanation for Python's syntax, please post it.

Can't say I find it particularly compelling as an argument, not if
introducing punctuation-itis is the cost, anyway. Most people I know
use syntax colouring editors, after all.

Do we want to assume syntax coloring is the norm? This will make a
difference in the readability of ): I use IDLE for my Python
editor, and I love it, so maybe I'm just taking syntax coloring for
granted.

For the function def syntax, we should consider alternative forms,
depending on how much clarity or compactness we want. Any of these
would be acceptable:

func1 = function( x, y ):
func1 = func( x, y ):
func1 = def( x, y ):
func1 = x, y ):
func1 = : x, y :
:x,y:

The last form would be used where we now have lambda x,y:

It seems like the choice of symbols and keywords here is a matter of
personal preference. The only objective criteria I have is that the
short form should be as short as possible and reasonably close to the
normal form. Verbosity is one of the reasons I don't use lambda.

That particular example almost screams "don't do this", doesn't it?
x,y): x+y I can see as an improvement, but yawn, really. Making
function definitions expressions rather than statements would have the
same effect. By the way, how do you know when your lambda is finished?
I gather the ()s are required if using as an expression?

It took me a long time to realize that lamdas have only one advantage
over named functions - they can be crammed into a tight space. Here
is an example:

L = [(lambda x: x**2), (lambda x:x**3), (lambda x:x**4), (lambda
x:x**5)]

If the purpose is to save space, wouldn't this be better as:

L = [:x:x**2, :x:x**3, :x:x**4 :x:x**5]

I'm assuming the parens are optional. Is there a parsing problem I'm
not seeing? I would add parens simply because I like the appearance
of func1 = x, y ):

I would also be happy with just deprecating lambdas entirely.

I personally prefer explicit to implicit, but I know there's lots of
people who are big into saving a few keystrokes.

See discussion above on explicitness. I'm not seeing any advantage in
the explicitness of self.something over .something -- *provided* that
the leading dot is not used for any other purpose than an abbreviation
for __self__.

Keystrokes are not a big issue for me either, but in this case, where
we have such frequent use of the syntax, I can see where it would be
significant.

-- Changing numerous attributes of an instance will be more
convenient. ( need use case )

That's nice, but honestly, if you're doing a lot of this in cases
trivial enough to warrant the addition you should likely be refactoring
with a domain-modelling system anyway. Still, if you modify the with to
work something like this:

with x:
.this = 32
.that = 43
temp = 'this'*repeat
.something = temp[:55]

i.e. to just alter the implicit target of the block, not force all
variables to be assigned to the object, it seems a nice enough feature.

I'm not sure I understand your use of the leading dots above. Do I
assume that .something gets attached to x, and temp is discarded when
the with block is finished? This will conflict with the use of
leading dots as an abbreviation for __self__. Why do we care about
temp variables here? If it really matters, wouldn't it be easier to
just del x.temp when we are done?

Fine, but no need to redefine the spelling for that save to make the
definition itself an expression that returns the function as a value and
allows one to drop the name. i.e. a = def ( y,z ): y+z would work just
as well if you could assign the result to a variable and figured out how
you wanted to handle the indentation-continuation thing to know when the
function ended.

The tradeoff is compactness vs preference for a keyword over a symbol.
I don't see any objective criteria, except that the lambda syntax
should be similar to the normal sytnax.

Is hidden (implicit) magic that requires the user to learn rules as to
what the target is when treating functions/methods as first-class
objects. Not a big deal, really.

Um, can't say I see this as a huge pedagogical win. A function either
takes an argument self and can set attributes of the object, or a
function has access to a magical "global" __self__ on which it can set
attributes. I'll agree that it's nice having the same concept for
module and class variables, but seeing that as a huge win assumes, I
think, that those being taught are coming from a "globals and functions"
background rather than a structures and functions background. One type
is accustomed to altering their execution environment, the other to
altering solely those things which are passed into the function as
parameters.

I am assuming no background at all other than Python up to the point
where we introduce OOP. At that point, students will understand both
global variables and functions. I measure simplicity by how much text
it takes to provide a basic explanation. See the samples at the end of
this post.

That's a counter, not a con. Similarly "Explicit is better than
Implicit" is only a counter, not a con. A con would be: "presence of
variable of implicit origin" or "too much punctuation". Don't think
either is a huge concern.

Doesn't seem a particularly strong pro. IOW seems pretty minimal in
benefit. As for a con, the eye, particularly in a syntax-colouring
editor picks out keywords very well, while punctuation tends to blur
into other punctuation.

I see the pro, seems approx. the same to me.

As specified, makes it only useful for trivial assignments. If you're
going to all the trouble of introducing .x notation to save keystrokes,
why not simply have with alter __self__ for the block so you can still
distinguish between temporary and instance variables?

I'm using __self__ exclusively for the bind object in a method call.

My biggest concern is not wanting to make leading dots the norm on
every variable assignment in a prototype definition. If we are going
to highlight the instance variables, and say to students "This is the
key difference between what you already know (modules) and what you
are going to learn next (prototypes), then I don't want every other
variable in the prototype definition to look just like the instance
variables.

I've included the with blocks in my proposal to please the Prothon
folks, but unless someone can come up with a use case, they are not
worth the confusion they are causing.

In the final analysis, this really seems like about 3 separate proposals:

* I like the .x notation's universal applicability, it does seem
simple and elegant from a certain point of view
o I don't like the implicit __self__, but that's an integral
part of the proposal, so a wash

If we can come up with an alternative that doesn't require multiple
function forms, I would like to consider it.

o I'd want clarification of how to store a reference to
another object's (bound) method (which is *extremely* common
in Python code for storing, e.g. callbacks)

bf = cat1.func # where cat1 is an instance not a prototype.

* I really dislike the ): function definition notation,
"Readability Counts". Why clutter the proposal with that?

It looks good to me, but I'm probably not the best judge of
aesthetics. I'll collect some other opinions on this. If enough
people prefer def ( ): ( or def : for the lambda form), I'll change
the proposal.

Does it make a difference in your preference that the parens are
optional? I would use them to enhnace readability on normal
functions, but leave them out on lambdas.

* I'm neutral on the with: stuff, I'd much prefer a real block
mechanism similar to Ruby with (if we're using implicit targets),
the ability to specify the .x target for the block

I've never understood the advantage of Ruby code blocks over Python
functions, but that is a separate discussion.

So, the .x notation seems like it would be nice enough, but nothing else
really makes me jump up and down for it...

That said, I'd probably be willing to use a language that was running on
the PythonVM with a parser/compiler that supported the syntax. I'd be
totally uninterested in automated translation of Python code to the new
form. That's the kind of thing that can be handled by running on the
same VM just as easily as anything else and you then avoid lots of
migration headaches.

I don't understand. If you need to use modules written in Python 2,
you would need at least some kind of wrapper to make the calls look
like Python 3. It seems like any changes that are not "backward
compatible" with Python 2 will need to be at least "migratable" from
earlier versions, using some automatic translator. That is the major
constraint I have assumed in thinking about new syntax. Is this not a
vital requirement?

So, just as a marketing data-point; I'm not convinced that this is
markedly superior, but I'd be willing to try a language that differed
from Python in just the .x aspects to see whether it was worthwhile.

Thanks again for your time and effort.

-- Dave

Explanation of Instance Variables in Python
===========================================
""" Some of the variables inside the functions in a class have a
self. prefix. This is to distinguish local variables in the function
from "instance variables". These instance variables will be found
when the function is called, by searching the instance which called
the function. The way this works is that calling the function from an
instance causes that instance to be passed as the first argument to
the function call. So if you call cat1.talk(), that is equivalent to
Cat.talk(cat1) If you call cat1.set_vars( "Garfield", "Meow"), that is
equivalent to Cat.set_vars(cat1, "Garfield", "Meow")

The "current instance" argument is auto-magically inserted as the
first argument, ahead of any other arguments that you may provide in
calling a method that is "bound" to an instance. Note: The
distinction between instances and classes is important here. If you
call a function from a class, that function is not bound to any
instance, and you have to supply the instance explicitly in the first
argument ( Cat.talk(cat1) )

The variable name self is just a convention. As long as you put the
same name in the first argument as in the body of the definition, it
can be self or s or even _ The single underscore is handy if you
want to maximally suppress clutter. """

Explanation of Simplified Instance Variables
============================================
""" Some of the variables inside the functions in a prototype have a
leading dot. This is to distinguish local variables in the function
from "instance variables". When a function is called from an instance
( cat1.talk() ) a special global variable __self__ is automatically
assigned to that instance ( __self__ = cat1 ) Then when the function
needs an instance variable ( .sound ) it uses __self__ just as if you
had typed it in front of the dot ( __self__.sound ) The leading dot
is just an abbreviation to avoid typing __self__ everywhere. """

 

[Michael Walter]

> [...]
It took me a long time to realize that lamdas have only one advantage
over named functions - they can be crammed into a tight space. Here
is an example:

L = [(lambda x: x**2), (lambda x:x**3), (lambda x:x**4), (lambda
x:x**5)]

If the purpose is to save space, wouldn't this be better as:

L = [:x:x**2, :x:x**3, :x:x**4 :x:x**5]

L = map(lambda x: lambda y: x**y,range(2,6))

Almost. Or write mapc ("map currying") and have:

L = mapc(pow,range(2,6))

Shorter than your example, less mistake-prone, no obvious lambda at all

Cheers,
Michael

 

[Ville Vainio]

Michael> Almost. Or write mapc ("map currying") and have:

Note - the name mapc is very misleading, because it looks too much
like Lisp mapc (mapcar, equicalent to normal python map).

 

[Greg Ewing]

Explanation of Simplified Instance Variables
--------------------------------------------
""" Some of the variables inside the functions in a prototype have a
leading dot. This is to distinguish local variables in the function
from "instance variables". When a function is called from an instance
( cat1.talk() ) a special global variable __self__ is automatically
assigned to that instance ( __self__ = cat1 ) Then when the function
needs an instance variable ( .sound ) it uses __self__ just as if you
had typed it in front of the dot ( __self__.sound ) The leading dot
is just an abbreviation to avoid typing __self__ everywhere. """

Explanation of Python Instance Variables, Omitting Unnecessary Words
--------------------------------------------------------------------

When a function is called from an instance (e.g. cat1.talk()),
the instance is passed in as an extra parameter at the beginning
of the parameter list, conventionally named 'self'. This allows
you to refer to attributes of the instance as 'self.attrname'
(e.g. self.sound).

========== END ==============

(I've left out mention of the difference between classes and
instances, as you did in your explanation even though it's just
as important in your scheme. Including it still wouldn't make
my explanation significantly longer than yours.)

 

[David MacQuigg]

Explanation of Python Instance Variables, Omitting Unnecessary Words
--------------------------------------------------------------------

When a function is called from an instance (e.g. cat1.talk()),
the instance is passed in as an extra parameter at the beginning
of the parameter list, conventionally named 'self'. This allows
you to refer to attributes of the instance as 'self.attrname'
(e.g. self.sound).

========== END ==============

(I've left out mention of the difference between classes and
instances, as you did in your explanation even though it's just
as important in your scheme. Including it still wouldn't make
my explanation significantly longer than yours.)

This seems much too brief for my taste. I think some of the
"unnecessary words" will be very helpful for students. I need other
opinions on this.

The distinction between calling from classes and calling from
instances is not necessary with the new syntax, because we don't have
the problem of needing a different calling sequence. i.e. the student
doesn't have to remember cat1.talk() in one case and Mammal.talk(cat1)
in another.

With the new syntax, the distinction becomes important when we get to
bound and unbound functions (same as in Python), but that is later in
the chapter, not part of the introductory explanation.

I've started a new thread "Explanation of Instance Variables in
Python" so we can explore this topic more fully.

Thanks for your help.

-- Dave

 

[Michael Walter]

Michael> Almost. Or write mapc ("map currying") and have:

Note - the name mapc is very misleading, because it looks too much
like Lisp mapc (mapcar, equicalent to normal python map).

True.

L = map(curry(pow),range(2,6))

might be more intelligent, anyway, what do you think?

Cheers,
Michael

 

[David MacQuigg]

[...]
It took me a long time to realize that lamdas have only one advantage
over named functions - they can be crammed into a tight space. Here
is an example:

L = [(lambda x: x**2), (lambda x:x**3), (lambda x:x**4), (lambda
x:x**5)]

If the purpose is to save space, wouldn't this be better as:

L = [:x:x**2, :x:x**3, :x:x**4 :x:x**5]

L = map(lambda x: lambda y: x**y,range(2,6))

Almost. Or write mapc ("map currying") and have:

L = mapc(pow,range(2,6))

Shorter than your example, less mistake-prone, no obvious lambda at all

The problem with this suggestion ( and many other similar uses of
existing functions ) is that it relies on a regular mathematical
sequence. Think of a more general case, something like:

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]

I would like to get some feedback on the more general questions:

Q1) Is it worth having a "lambda" syntax like this, or should we just
deprecate lambda functions entirely and use:

def f1(x): return x**2
def f2(x): return x+4
def f3(x): return x/5
def f4(x): return 2-x
def f5(x): return x*7
L = [ f1, f2, f3, f4, f5 ]

Q2) Will it help new users to have the "lambda" syntax be as close as
possible to a normal function definition? i.e.

f x): return x**2 # a simple function
:x:x**2 # equivalent lambda expression

-- or --

f = def(x): return x**2
def x:x**2

I am especially interested in feedback from users who have recently
learned Python. I suspect that many experienced users will have long
forgotten any difficulties they had while learning.

-- Dave

 

[David MacQuigg]

I'm starting a new sub-thread. Warning: Don't venture into this one if
you can't stretch your imagination a little. What I'm looking for
here is ideas for the ideal Python-like language that will make
less-experienced users happy. To me, that means making Python
simpler, not necessarily more powerful.

I've got some ideas for how to handle the classes vs classless thing.
See the "Python4" document at http://ece.arizona.edu/~edatools/Python
Mainly, I want to keep classes, because I see them as very useful in
most programs, where a "two-tier" organization is helpful. But I also
think I understand the needs of those who don't want a two-tier
organization. Seems like the answer is to make classes optional.

Comments are welcome. Suggestions are even better.

-- Dave

************************************************************* *
* David MacQuigg, PhD * email: dmq at gain.com * *
* IC Design Engineer * phone: USA 520-721-4583 * * *
* Analog Design Methodologies * * *
* * 9320 East Mikelyn Lane * * *
* VRS Consulting, P.C. * Tucson, Arizona 85710 *
************************************************************* *

 

[Josiah Carlson]

The problem with this suggestion ( and many other similar uses of

existing functions ) is that it relies on a regular mathematical
sequence. Think of a more general case, something like:

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]
Ick.


Q1) Is it worth having a "lambda" syntax like this, or should we just
deprecate lambda functions entirely and use:

def f1(x): return x**2
def f2(x): return x+4
def f3(x): return x/5
def f4(x): return 2-x
def f5(x): return x*7
L = [ f1, f2, f3, f4, f5 ]

Don't deprecate lambda. I know Guido is hot to do so, and believes it a
mistake to have in the first place, but is it really hurting anyone?
Sure, those who don't know how to use them, but the barrier for entry is
quite low.

Q2) Will it help new users to have the "lambda" syntax be as close as
possible to a normal function definition? i.e.

f x): return x**2 # a simple function
:x:x**2 # equivalent lambda expression

-- or --

f = def(x): return x**2
def x:x**2

Ick on the four options just given.

I am especially interested in feedback from users who have recently
learned Python. I suspect that many experienced users will have long
forgotten any difficulties they had while learning.

Lerning lambda expressions are trivial when you have experience with
derivatives of LISP. While I generally don't like to point users off to
go RTFM, in this case, 5 minutes of manual reading (without LISP
experience) will go a long way toward understanding lambda expressions.

- Josiah

 

[David MacQuigg]

The problem with this suggestion ( and many other similar uses of
existing functions ) is that it relies on a regular mathematical
sequence. Think of a more general case, something like:

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]

Ick.

Could you be more specific? :>)

Q1) Is it worth having a "lambda" syntax like this, or should we just
deprecate lambda functions entirely and use:

def f1(x): return x**2
def f2(x): return x+4
def f3(x): return x/5
def f4(x): return 2-x
def f5(x): return x*7
L = [ f1, f2, f3, f4, f5 ]

You seem to be suggesting that the current syntax is preferable. Is
this really what you would prefer:

L = [(lambda x:x**2), (lambda x:x+4), (lambda x:x/5),
(lambda x:2-x), (lambda x:x*7)]

Don't deprecate lambda. I know Guido is hot to do so, and believes it a
mistake to have in the first place, but is it really hurting anyone?
Sure, those who don't know how to use them, but the barrier for entry is
quite low.

The barrier *should* be low, but it isn't. In Learning Python, 2nd
ed., there is a 5-page section under "Advanced Topics" devoted to
lambdas. Some experts like the association with lambda calculus, even
though that doesn't help beginners. In fact, it only serves to make
lambdas seem even more mysterious. The benefit of lambdas *could* be
provided in a simple, self-explanatory syntax that requires zero pages
in a textbook and has none of the mystique that turns off beginners.

Lamdas add an unnecessary keyword and unnecessary burdens to the
syntax of the language. The benefit is very small -- being able to
cram a function definition in a tight space. Due to the mystique of
lambdas, it took me a while to realize that was their only benefit.

Ick on the four options just given.

The parentheses are optional when we have colons around the arguments.
Leaving them out is my preference, but I would be just as happy with

L = [x):x**2, x):x+4, x):x/5, x):2-x, x):x*7 ]

This would more strongly highlight the argument x, and still have a
form that parallels the standard function definition.

How about this:

f(x): return x**2
(x):x**2

Neat and clean, but I don't know if there would be parser problems
with the short form. Unlike the long form, which can only occur at
the beginning of a line, the short form might occur in a dictionary
item, where the colon could lead to ambiguity. Maybe we could say
lambdas in dictionaries must be enclosed in parentheses, or maybe just
not allow them at all where they might cause ambiguity.

Lerning lambda expressions are trivial when you have experience with
derivatives of LISP. While I generally don't like to point users off to
go RTFM, in this case, 5 minutes of manual reading (without LISP
experience) will go a long way toward understanding lambda expressions.

My users ( EE students and professional design engineers ) have no
experience with LISP. I agree, 5 minutes should be enough to explain
lambdas properly, but unfortunately, they are not explained properly
in the texts I have seen.

In my humble opinion, GvR should have ignored the experts who told him
lambdas were great, and just applied some simple common sense to find
a better solution.

-- Dave

 

[Ville Vainio]

David> In my humble opinion, GvR should have ignored the experts
David> who told him lambdas were great, and just applied some
David> simple common sense to find a better solution.

Why don't we just let lambdas be? They got a bad rep because of the
lack of lexical closures (and hence the default variable hack), but
the issue is solved already. Lambdas work well enough for simple
currying/whatever tasks, and don't really hurt anyone.

 

[Josiah Carlson]

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]

Ick.

Could you be more specific? :>)

Sure, the general format of your anonymous function syntax given above
does not offer anything that a new user can search for.

Generally, new users of any language are one of two types:
Self starter (looks in the help files)
Asker (asks people before even checking the help files, searching
google, etc.)

The asker may become a self-starter, but usually only after repeated
"dude, use google" replies on c.l.py.

Your proposed syntax removes the ability for the self-starter to search
the docs for a keyword (which is currently 'lambda'), forcing them to
become an asker.

You seem to be suggesting that the current syntax is preferable. Is
this really what you would prefer:

L = [(lambda x:x**2), (lambda x:x+4), (lambda x:x/5),
(lambda x:2-x), (lambda x:x*7)]

You fail to notice that:
L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7]
needs to be:
L = [x:x**2), x:x+4), x:x/5), x:2-x), x:x*7)]
....unless of course you want to remove the ability for anonymous
functions/lambdas to return tuples.

The only difference between the syntax you offer is the replacement of
'lambda ' with ':', which I don't believe is an advancement in the language.

The barrier *should* be low, but it isn't. In Learning Python, 2nd
ed., there is a 5-page section under "Advanced Topics" devoted to
lambdas. Some experts like the association with lambda calculus, even
though that doesn't help beginners. In fact, it only serves to make
lambdas seem even more mysterious. The benefit of lambdas *could* be
provided in a simple, self-explanatory syntax that requires zero pages
in a textbook and has none of the mystique that turns off beginners.

How about this for a manual page for lambda...

In other languages, Python's lambda would be considered an 'anonymous
function', that is, a function that does not require a name.:
81

Certainly you can give lambdas names with standard assignments.:
81

The equivalent function definition is below.:
... return arg*arg
... 81

Generally, lambdas are functions with a single expression in its body
whose value is returned. Just like normal function definitions, lambdas
can take multiple arguments, contain keyword arguments, return any
Python type, etc., as long as the function body is a single expression,
and whose parameters match standard function definition syntax, the
lambda is valid. (leave annotation and/or link to what an expression is)

An ugly example of this is as follows.:
(1, 2, (3,), {'c': 4})

Which is equivalent to:
... return (a, b, args, kwargs)
... (1, 2, (3,), {'c': 4})

Lamdas add an unnecessary keyword and unnecessary burdens to the
syntax of the language. The benefit is very small -- being able to
cram a function definition in a tight space. Due to the mystique of
lambdas, it took me a while to realize that was their only benefit.

And removing the keyword would remove their 'mystique'? No, all it
would do is remove a keyword from Python. If we used your alternative
syntaxes, the 'mystique' would still exist and be unsearchable.
Removing the functionality entirely would result in no longer seeing the
below (which you use as an example):

L = [(lambda...),
(lambda...),
...]

But it being replaced with:

def fun1(arg): return ...
def fun2(arg): return ...
...
L = [fun1, fun2,...]

Neither of which are terribly attractive, but I prefer the lambda version.

Ick on the four options just given.

The parentheses are optional when we have colons around the arguments.
Leaving them out is my preference, but I would be just as happy with

L = [x):x**2, x):x+4, x):x/5, x):2-x, x):x*7 ]

I'm not icking on the parenthesis, I'm icking on the general syntax.
While Python 3 is supposed to be a mythical creature that fixes all of
the problems with previous versions, I don't believe that the syntax
options you provide are a fix. In fact, what about the following...

L[:x:x**.5]

Using current python syntax, that is a slice into a sequence. With your
syntax, that is an anonymous function that takes an argument and returns
its square root, that is used as an index into some mappable type. Are
you also talking about changing slice syntax?

As for
a. f x): return x**2
b. f = def(x): return x**2
c. def x:x**2

a. Also looks like a bad slice to me.
b. What was wrong with:
def f(x): return x**2
c. Now you're just replacing the lambda keyword with the def keyword.

This would more strongly highlight the argument x, and still have a
form that parallels the standard function definition.

How about this:

f(x): return x**2
(x):x**2
>
> Neat and clean, but I don't know if there would be parser problems

First looks like magic.
Second looks like a slice.
Neither are neat and clean.

with the short form. Unlike the long form, which can only occur at
the beginning of a line, the short form might occur in a dictionary
item, where the colon could lead to ambiguity. Maybe we could say
lambdas in dictionaries must be enclosed in parentheses, or maybe just
not allow them at all where they might cause ambiguity.

With the 'lambda' (or other equivalent) keyword, there does not exist
ambiguity. Your removal of the keyword seems to not add any
understandability to the syntax (or the one-line-function 'problem'),
but adds ambiguity to the meaning of an equivalent anonymous function.
I thought Python was about removing ambiguity, not encouraging it.

My users ( EE students and professional design engineers ) have no
experience with LISP. I agree, 5 minutes should be enough to explain
lambdas properly, but unfortunately, they are not explained properly
in the texts I have seen.

So why are you explaining lambdas to them? If they are having
difficulty understanding them, then don't teach it. Since you are also
advocating the removal of the lambda functionality entirely, I see no
reason to show them something that they are going to struggle with
understanding.

If you are still going to teach them lambdas, then do it by example.
Give a simple function definition, translate it into a lambda, then have
them do it. I find that learn-by-example works pretty well, at least
for simple algorithms like definition-to-lambda. If your students can't
translate a few simple function definitions to lambdas, then Iyou should
ask yourself if they deserve to get degrees in their field.

In my humble opinion, GvR should have ignored the experts who told him
lambdas were great, and just applied some simple common sense to find
a better solution.

I don't believe that lambdas were a solution to a problem. I believe
the /desire/ was to have a way of defining simple functions in a general
fashion. They do just that, allow simple functions to be defined in a
general fashion, albeit using a slightly altered function syntax. Their
ability to be placed in lists, gain names, etc., was a side-effect of
them being Python objects.

- Josiah

 

[Peter Otten]

How about this for a manual page for lambda...

In other languages, Python's lambda would be considered an 'anonymous
function', that is, a function that does not require a name.:

81

Certainly you can give lambdas names with standard assignments.:

81

The equivalent function definition is below.:

... return arg*arg
...
81

Generally, lambdas are functions with a single expression in its body
whose value is returned. Just like normal function definitions, lambdas
can take multiple arguments, contain keyword arguments, return any
Python type, etc., as long as the function body is a single expression,
and whose parameters match standard function definition syntax, the
lambda is valid. (leave annotation and/or link to what an expression is)

I suggest that you submit the above as a patch for the tutorial
http://www.python.org/doc/current/tut/node6.html. Currently there is an
example with a nice trick which has nothing to do with lambda. Your more
straightforward example demonstrates how simple lambda really is.

An ugly example of this is as follows.:

(1, 2, (3,), {'c': 4})

Which is equivalent to:

... return (a, b, args, kwargs)
...
(1, 2, (3,), {'c': 4})

At this point of learning the language a newbie may or may not be
comfortable with the *args, **kw special arguments. As their usage is
completely orthogonal to functions/lambdas I wouldn't mention them here.

Peter

 

[John Roth]

I don't believe that lambdas were a solution to a problem. I believe
the /desire/ was to have a way of defining simple functions in a general
fashion. They do just that, allow simple functions to be defined in a
general fashion, albeit using a slightly altered function syntax. Their
ability to be placed in lists, gain names, etc., was a side-effect of
them being Python objects.

If I remember my Python history correctly, they showed up as part
of a "functional programming" package that included map, filter, apply
and reduce. The term "lambda" comes from that heritage.

I agree with the other comments in this thread that indicate that
lambdas are poorly explained, although the explanation in the tutorial
isn't bad.

They are also widely misused in callback function examples where
bound methods would be much more appropriate.

John Roth

 

[David MacQuigg]

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]

Ick.

Could you be more specific? :>)

Sure, the general format of your anonymous function syntax given above
does not offer anything that a new user can search for.

This is a good point, and one I hadn't thought of. The counter is
that function definition syntax is so basic and so prevalent that any
user of Python will already know it. Lambdas are seldom used, but by
making their syntax almost identical to normal functions, we can make
them self-explanatory. I would add one very short paragraph at the
end of an introduction to functions.
"""
Nameless Functions
------------------
There is a short form of a function definition, which is sometimes
used in lists or other places where space is tight. If you can write
your function as a single expression, you can use the short form by
just leaving off the function's name and the return keyword.
.... example above showing long form and short form.
"""
[snip further discussion on searchability]

You seem to be suggesting that the current syntax is preferable. Is
this really what you would prefer:

L = [(lambda x:x**2), (lambda x:x+4), (lambda x:x/5),
(lambda x:2-x), (lambda x:x*7)]

You fail to notice that:
L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7]
needs to be:
L = [x:x**2), x:x+4), x:x/5), x:2-x), x:x*7)]
...unless of course you want to remove the ability for anonymous
functions/lambdas to return tuples.

I'm not seeing the need for parentheses in the above example. The
commas clearly show this is a five-item list.

I think the parentheses should be optional, to be used in cases where
there is ambiguity, like the one you have identified below.

The only difference between the syntax you offer is the replacement of
'lambda ' with ':', which I don't believe is an advancement in the language.

The important difference is that it makes the short form (lambda)
almost identical to the normal form. This makes it self-explanatory,
and avoids five pages in an introductory text.

How about this for a manual page for lambda...

In other languages, Python's lambda would be considered an 'anonymous
function', that is, a function that does not require a name.:

81

Certainly you can give lambdas names with standard assignments.:

81

The equivalent function definition is below.:

... return arg*arg
...
81

Generally, lambdas are functions with a single expression in its body
whose value is returned. Just like normal function definitions, lambdas
can take multiple arguments, contain keyword arguments, return any
Python type, etc., as long as the function body is a single expression,
and whose parameters match standard function definition syntax, the
lambda is valid. (leave annotation and/or link to what an expression is)

An ugly example of this is as follows.:

(1, 2, (3,), {'c': 4})

Which is equivalent to:

... return (a, b, args, kwargs)
...
(1, 2, (3,), {'c': 4})

I would like to eliminate this explanation entirely, and just include
a simple paragraph at the end of the introduction to functions.

And removing the keyword would remove their 'mystique'?

Can you imagine some mathematician trying to foist "lambda calculus"
on us, when everyone can clearly see a nameless function is nothing
but a function without a name? Maybe you haven't seen some of the
discussions of lambda calculus.

No, all it
would do is remove a keyword from Python.

Which, in itself is a good thing.

If we used your alternative
syntaxes, the 'mystique' would still exist and be unsearchable.
Removing the functionality entirely would result in no longer seeing the
below (which you use as an example):

L = [(lambda...),
(lambda...),
...]

But it being replaced with:

def fun1(arg): return ...
def fun2(arg): return ...
...
L = [fun1, fun2,...]

Neither of which are terribly attractive, but I prefer the lambda version.

And I prefer the second form. It's a little more space, but much more
clarity. Space is rarely a worry for me.

f x): return x**2 # a simple function
:x:x**2 # equivalent lambda expression

-- or --

f = def(x): return x**2
def x:x**2

Ick on the four options just given.

The parentheses are optional when we have colons around the arguments.
Leaving them out is my preference, but I would be just as happy with

L = [x):x**2, x):x+4, x):x/5, x):2-x, x):x*7 ]

I'm not icking on the parenthesis, I'm icking on the general syntax.
While Python 3 is supposed to be a mythical creature that fixes all of
the problems with previous versions, I don't believe that the syntax
options you provide are a fix. In fact, what about the following...

L[:x:x**.5]

Using current python syntax, that is a slice into a sequence. With your
syntax, that is an anonymous function that takes an argument and returns
its square root, that is used as an index into some mappable type. Are
you also talking about changing slice syntax?

Nice work!!

No, I would say this is a rare case where we need to add the optional
parentheses if we really want a function not a slice. The slice
syntax is more important, and should have priority.

As for
a. f x): return x**2
b. f = def(x): return x**2
c. def x:x**2

a. Also looks like a bad slice to me.

I now think that f(x): would be the best option. A normal function
definition always starts at the beginning of a line, so this won't get
confused with a dictionary key or list index.

b. What was wrong with:
def f(x): return x**2

See pros and cons on this issue at
http://ece.arizona.edu/~edatools/Python/PrototypeSyntax.htm
Pro1: See all the variables at a glance in one column.
Pro2: Emphasize the similarity between data and functions as
attributes of an object.
Pro3: Eliminates the need for special syntax in lambda functions.

c. Now you're just replacing the lambda keyword with the def keyword.

Which users already understand. Also two letters shorter, since the
sole purpose of lambdas is to save space.

First looks like magic.
Second looks like a slice.
Neither are neat and clean.

I think we should leave these matters of personal preference and style
to GvR, and focus here on finding hidden problems, like -- This syntax
won't work because ..."

With the 'lambda' (or other equivalent) keyword, there does not exist
ambiguity. Your removal of the keyword seems to not add any
understandability to the syntax (or the one-line-function 'problem'),
but adds ambiguity to the meaning of an equivalent anonymous function.
I thought Python was about removing ambiguity, not encouraging it.

The ambiguity you have discovered ( and I appreciate these discoveries
) is still an edge case, best resolved by putting the burden of adding
parentheses on the least used syntax ( these nameless functions ).

So why are you explaining lambdas to them? If they are having
difficulty understanding them, then don't teach it. Since you are also
advocating the removal of the lambda functionality entirely, I see no
reason to show them something that they are going to struggle with
understanding.

The reason we need to include lambdas is because they are now a part
of the Python culture, and users will be seeing them, encountering
long discussions about "lambda calculus" and other useless
obfuscations. I will give a short explanation, and advice to avoid
them. Students should also be aware of the discussion on pages
219-224 of Learning Python, in case they need to decipher a really
nasty lambda.

If you are still going to teach them lambdas, then do it by example.
Give a simple function definition, translate it into a lambda, then have
them do it. I find that learn-by-example works pretty well, at least
for simple algorithms like definition-to-lambda. If your students can't
translate a few simple function definitions to lambdas, then Iyou should
ask yourself if they deserve to get degrees in their field.

Bad attitude. They have plenty of opportunites for mental
masturbation in their own field of study, and lots of pressure to do
something more useful with their time. :>)

I don't believe that lambdas were a solution to a problem. I believe
the /desire/ was to have a way of defining simple functions in a general
fashion. They do just that, allow simple functions to be defined in a
general fashion, albeit using a slightly altered function syntax. Their
ability to be placed in lists, gain names, etc., was a side-effect of
them being Python objects.

Are we talking about the same thing? The *sole purpose* of lambdas is
to squeeze a function into a tight space. There is no other advantage
over a simple, normal, general-purpose, function definition.

-- Dave

 

[Josiah Carlson]

L = [:x:x**2, :x:x+4, :x:x/5, :x:2-x, :x:x*7 ]

Sure, the general format of your anonymous function syntax given above
does not offer anything that a new user can search for.

This is a good point, and one I hadn't thought of. The counter is
that function definition syntax is so basic and so prevalent that any
user of Python will already know it. Lambdas are seldom used, but by
making their syntax almost identical to normal functions, we can make
them self-explanatory. I would add one very short paragraph at the
end of an introduction to functions.
"""
Nameless Functions
------------------
There is a short form of a function definition, which is sometimes
used in lists or other places where space is tight. If you can write
your function as a single expression, you can use the short form by
just leaving off the function's name and the return keyword.
.... example above showing long form and short form.
"""
[snip further discussion on searchability]

After reading the remainder of your post, I have come to the opinion
that of all your offered syntaxes, there exists only one syntax for
named and nameless functions that /doesn't/ have severe handicaps of one
kind or another.

def funct(arg):
... #body
return result

funct = def(arg): result

The above has both a searchable keyword, is easy to describe (you gave a
sufficient explanation), and doesn't suffer from the "looks like a
slice" (seq[:x:x**2]), "looks like a function call in a slice"
(seq[f(x):x**2]), due to the existance of the 'def' keyword.

Ultimately it will come down to what people find most intuitive to
program with. I think the "lamdba replaced with def adding parenthesis"
syntax is preferential to the other options (with removing lambdas a far
second, and all other options not even in the running), and I would
expect that other current Python users would agree.


I've read the link you posted, and I've previously taken a look at
Prothon. Prototype syntax does not suit my taste in programming
languages. If Python 3.0 becomes Prothon, I'll fork the most recent
Python 2.x codebase that I prefer, maintaining it myself if necessary.

Just as my anti-prototype perspective colors my opinions on what is
reasonable, I would imagine that your pro-prototype perspective colors
yours. I would also expect that someone who enjoys using lisp or
perhaps SML would have their opinion on what is reasonable, colored by
their preferences. Considering that Python is not a prototype-based
language today in 2.3 (or the forthcoming 2.4 in the fall), having it
become one in Python 3 would be quite a drastic change, quite literally
like going from Python 2.3 to Prothon.

- Josiah

 

[Michael Geary]

I've previously taken a look at Prothon. Prototype syntax
does not suit my taste in programming languages. If
Python 3.0 becomes Prothon, I'll fork the most recent
Python 2.x codebase that I prefer, maintaining it
myself if necessary.

Prothon syntax is changing daily (actually hourly right now). It's not going
to end up looking anything like what you see on the prothon.org website.
What's being discussed right now looks a lot better to me. You still may or
may not like the way it turns out, but it will be worthwhile to check back
in on it after things settle down a bit.

-Mike

 

[Josiah Carlson]

I've previously taken a look at Prothon. Prototype syntax

Prothon syntax is changing daily (actually hourly right now). It's not going
to end up looking anything like what you see on the prothon.org website.
What's being discussed right now looks a lot better to me. You still may or
may not like the way it turns out, but it will be worthwhile to check back
in on it after things settle down a bit.

Certainly looking at it will be worthwhile, though as it stands, Prothon
has sufficient syntactical ugly (IMO) to make me feel nauseated (even
with all the syntactical variations discussed on the mailing list). To
make Mark Hahn feel a bit better, I have the same opinion of the 'Self'
language, though Prothon does look quite a bit better than Self (what
was/is Sun thinking?)

- Josiah

 

[David MacQuigg]

I've read the link you posted, and I've previously taken a look at
Prothon. Prototype syntax does not suit my taste in programming
languages. If Python 3.0 becomes Prothon, I'll fork the most recent
Python 2.x codebase that I prefer, maintaining it myself if necessary.

Just as my anti-prototype perspective colors my opinions on what is
reasonable, I would imagine that your pro-prototype perspective colors
yours.

I am currently neutral on the idea of prototypes, still waiting for
anyone to show me a good use case, so I can include it in my webpage.
I still have "cloning" of instances in my proposal, but you will
notice they are at the bottom of the list of benefits at
http://ece.arizona.edu/~edatools/Python/PrototypeSyntax.htm

I typically get enthusiastic about something when I first see it, then
I learn a little more, and some of the new tricks/features drop to the
bottom of my list. What remains of my original enthusiasm for Prothon
is still in the unification of methods and functions.

The main thing that changed my mind on the need for Prothon-like
prototypes is Michele Simionato's posting on 4/28/04 "Prototypes in
Python". If anyone is seriously interested in prototyping, they can
do it in Python right now, using Michele's 'prototype.py' module.
Until we get some actual users with a demonstrated need, I'm not
pushing for anything beyond the current module.

The key to finding what is fundamentally good in these other
languages, and what is just someone's personal preference for being
different, is keeping an open mind. It's astonishing how few people
can do that in comparing computer languages. I guess it is just
easier to be "anti" or "pro" and skip the investigation and thinking.

I would also expect that someone who enjoys using lisp or
perhaps SML would have their opinion on what is reasonable, colored by
their preferences. Considering that Python is not a prototype-based
language today in 2.3 (or the forthcoming 2.4 in the fall), having it
become one in Python 3 would be quite a drastic change, quite literally
like going from Python 2.3 to Prothon.

I continue to find new surprises in Python. The ability to change
classes into prototypes by using descriptors is the latest exammple.
This seems like a drastic change, but it is all within the capability
of Python.

Prothon does have some changes that are more drastic, but in my
opinion have no real benefit over Python. I measure drastic by how
much effort it will take to translate existing Python programs to
Prothon. By this measure, the unification of functions and methods is
not a drastic change. I believe it will be possible to automatically
translate all Python methods into the new form. So if we categorize
syntax changes as ( Compatible / Migratable / Totally Different ), the
proposed changes are in the middle.

-- Dave