Unification of Methods and Functions

[David MacQuigg]

I'm not getting any feedback on the most important benefit in my
proposed "Ideas for Python 3" thread - the unification of methods and
functions. Perhaps it was buried among too many other less important
changes, so in this thread I would like to focus on that issue alone.

I have edited the Proposed Syntax example below to take out the
changes unecessary to this discussion. I left in the change of
"instance variable" syntax ( self.sound --> .sound ) because that is
necessary for the unification of all method forms. ( Compare the
forms of the 'show' and 'talk' methods below.)

I believe these changes in syntax will make teaching OOP in Python
much easier. See "Prototypes.doc" at
http://ece.arizona.edu/~edatools/Python/ The first eight pages are a
basic, but complete presentation of the new OOP syntax. I expect this
to expand to about 30 pages with more examples and exercises. This
compares to about 60 pages in Learning Python 2nd ed.

If we measure complexity by the number of pages needed for a "textbook
explanation" of OOP, then I believe the new syntax has some real
benefits. At this point in the learning process, students already know
functions, modules, and global variables. The concept of using a
global variable __self__ is no surprise at all. The only thing new in
a class, compared to a module, is the instance variables, and they can
be explained in one paragraph. All methods look just like normal
functions. There is no need to explain "static methods" or any other
form of method. In fact, I use the term "function" rather than
"method" to emphasize the similarity.

I'm especially interested in feedback from users who are now learning
or have recently learned Python. I already know these changes seem
trivial to many experts. I've also heard plenty from the people who
think Python is so complex that we need to start a whole new language
( www.prothon.org ). I'm looking for a middle ground. I believe it
is possible to adopt what is good about Prothon, and not lose ten
years of software and community development.

======= Syntax Examples =============

## Proposed Syntax:
class Cat(Feline):
numCats = 0
def __init__( n = "unknown", s = "Meow" ):
Feline.__init__()
Cat.numCats += 1
.name = n # Set instance variables.
.sound = s
def show(): # Define a "static method".
Feline.show()
print " Cats:", Cat.numCats
def talk():
print "My name is ...", .name
print "I am a %s from %s" % (.genus, .home)
Mammal.talk() # Call an unbound function.
print __self__ ### Diagnostic check.

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.


## Equivalent Python:
class Cat(Feline):
numCats = 0
def __init__(self, n = "unknown", s = "Meow" ):
Feline.__init__(self)
Cat.numCats += 1
self.name = n
self.sound = s
def show():
Feline.show()
print " Cats:", Cat.numCats
show = staticmethod(show)
def talk(self):
print "My name is ...", self.name
print "I am a %s from %s" % (self.genus, self.home)
Mammal.talk(self)
print self

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.

========= End of Examples =======

Thanks for your help.

-- Dave

 

[Nicolas Fleury]

I'm especially interested in feedback from users who are now learning
or have recently learned Python. I already know these changes seem
trivial to many experts. I've also heard plenty from the people who
think Python is so complex that we need to start a whole new language
( www.prothon.org ). I'm looking for a middle ground. I believe it
is possible to adopt what is good about Prothon, and not lose ten
years of software and community development.

I don't think Python is too complex and to me your suggestion adds more
complexity rather than simplying. Your proposal is also not
backward-compatible at all, so I don't see it can ever be accepted. I
agree that avoiding self make code lighter, but I don't think it unify
things or simplify them. It adds a magic and hidden parameter. The
good thing about the current syntax is that it shows what is really
happening. Actually, I find the statu quo more unified.

I would prefer self to be a keyword and be used to determine if the
function is static, but I guess we have to live with things like that.
I have given some python courses and the self parameter seems to be
clear for everybody. I think it even helps comprehension.

Regards,

Nicolas

 

[Michael Walter]

[...]

Pardon my ignorance, but I'm not quite sure how this unificates methods
and functions at all.

In my eyes, unification would remove method definitions from the "class"
scope or make it simply syntactic sugar for:

def my_method(SomeClass self):
...

(instead of:
class SomeClass:
def my_method(self):
...)

This would also add the possibility to provide multiple dispatch by
simply allowing a "type hint" for every parameter at every position.

Cheers,
Michael

 

[Jack Diederich]

I'm not getting any feedback on the most important benefit in my
proposed "Ideas for Python 3" thread - the unification of methods and
functions. Perhaps it was buried among too many other less important
changes, so in this thread I would like to focus on that issue alone.
======= Syntax Examples =============

## Proposed Syntax:
class Cat(Feline):
numCats = 0
def __init__( n = "unknown", s = "Meow" ):
Feline.__init__()
Cat.numCats += 1
.name = n # Set instance variables.
.sound = s
def show(): # Define a "static method".
Feline.show()
print " Cats:", Cat.numCats
def talk():
print "My name is ...", .name
print "I am a %s from %s" % (.genus, .home)
Mammal.talk() # Call an unbound function.
print __self__ ### Diagnostic check.

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.


## Equivalent Python:
class Cat(Feline):
numCats = 0
def __init__(self, n = "unknown", s = "Meow" ):
Feline.__init__(self)
Cat.numCats += 1
self.name = n
self.sound = s
def show():
Feline.show()
print " Cats:", Cat.numCats
show = staticmethod(show)
def talk(self):
print "My name is ...", self.name
print "I am a %s from %s" % (self.genus, self.home)
Mammal.talk(self)
print self

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.

========= End of Examples =======

Explicit is better than implicit.
or
Magic BAAAAAAAAD [Phil Hartman as Frankenstein]

I suggest you check out perl to see mixing instance/class/static methods
in practice. Becuase perl is weakly typed this 'makes sense' in perl, even if
it causes problems[1]. What happens in practice is bad, people write functions
that can be used in two or more ways. This makes type checking hard, and
makes code unreadable. Functions frequently do slightly different things when
called one way or another.

For a python version you could do type checking on the function by doing
static analysis of the code, but that would be unpythonic. I like it when
a static function breaks badly when some yahoo tries to use self -- it breaks
early and loudly. Your way might only break on a certain code path that
tries to access '.name' and turns the method from static to instance.
If you re-added staticmethod/classmethod to clear up the distinction then
the above example just becomes a new syntax for implicit 'self'.

A version of the ':vars: expression' lambda replacement gets suggested every
so often (and that exact syntax once by me). It ain't going to happnen, labmda
is more likely to be dropped than enhanced.

-jackdied

[1] standard perl interview question, what is the difference between these
calls? All of these end up calling meow() with one argument, but they all
behave differently - sometimes very subtly.

$ob = new Cat;
$ob->meow(); # intsance method
Cat->meow(); # class method
Cat::meow($ob); # static method with first argument a Cat instance

 

[David MacQuigg]

I'm not getting any feedback on the most important benefit in my
proposed "Ideas for Python 3" thread - the unification of methods and
functions. Perhaps it was buried among too many other less important
changes, so in this thread I would like to focus on that issue alone.
======= Syntax Examples =============

## Proposed Syntax:
class Cat(Feline):
numCats = 0
def __init__( n = "unknown", s = "Meow" ):
Feline.__init__()
Cat.numCats += 1
.name = n # Set instance variables.
.sound = s
def show(): # Define a "static method".
Feline.show()
print " Cats:", Cat.numCats
def talk():
print "My name is ...", .name
print "I am a %s from %s" % (.genus, .home)
Mammal.talk() # Call an unbound function.
print __self__ ### Diagnostic check.

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.


## Equivalent Python:
class Cat(Feline):
numCats = 0
def __init__(self, n = "unknown", s = "Meow" ):
Feline.__init__(self)
Cat.numCats += 1
self.name = n
self.sound = s
def show():
Feline.show()
print " Cats:", Cat.numCats
show = staticmethod(show)
def talk(self):
print "My name is ...", self.name
print "I am a %s from %s" % (self.genus, self.home)
Mammal.talk(self)
print self

cat1 = Cat() # Create instance.
bf = cat1.talk # Make a bound function.

========= End of Examples =======

Explicit is better than implicit.
or
Magic BAAAAAAAAD [Phil Hartman as Frankenstein]

I agree that magic is bad, but I disagree that the magically inserted
first argument on *some* function calls is less than the magic of
setting the current instance to a global variable __self__. Students
at this point already understand global variables. Yet they
frequently stumble on the different calling sequences for bound and
unbound functions. ( cat1.talk() vs Cat.talk(cat1) )

Using a global __self__ puts the current instance in a much more
convenient place, always available, but never in your way. This
"out-of-your-way" aspect of the proposed __self__ is the key to
getting a consistent calling sequence for all functions and methods.
If you want to call a method, but you don't have a current instance,
no problem. Just call it like any other function. We don't need
special syntax for a "static method".

It is hard to judge the complexity of a proposed syntax by looking
with a microscope at something as small as setting a global variable.
The way I would make the comparison is by looking at the length of a
"textbook" explanation of instance variables. I believe the proposed
syntax will take about half the number of lines to write a good
explanation. See the examples at the end of
http://ece.arizona.edu/~edatools/Python/PrototypeSyntax.htm

There is a discussion of this question under the subject "Explanation
of Instance Variables in Python". Greg Ewing wrote a shorter
explanation than the one I prefer or the one that is in the Python
tutorial. I would like to get some more opinions, especially from
users who remember their first encounter with instance variables in
Python.

I suggest you check out perl to see mixing instance/class/static methods
in practice. Becuase perl is weakly typed this 'makes sense' in perl, even if
it causes problems[1]. What happens in practice is bad, people write functions
that can be used in two or more ways. This makes type checking hard, and
makes code unreadable. Functions frequently do slightly different things when
called one way or another.

You seem to be saying that there is a lot of confusion in Perl due to
mixing instance/class/static methods. I'm not familiar with Perl, but
this doesn't surprise me. It seems to *support* my proposal that we
do away with these variations in method calls. They serve no purpose
other than patching holes in the syntax which arose only because we
have this special first argument in *some* cases.

For a python version you could do type checking on the function by doing
static analysis of the code, but that would be unpythonic. I like it when
a static function breaks badly when some yahoo tries to use self -- it breaks
early and loudly. Your way might only break on a certain code path that
tries to access '.name' and turns the method from static to instance.
If you re-added staticmethod/classmethod to clear up the distinction then
the above example just becomes a new syntax for implicit 'self'.

I'm not following this. In the proposed syntax, there is no
distinction between static and class or instance methods. All methods
are called the same way. If you call a method that has instance
variables, it will look for __self__ to resolve those variables. If
__self__ is set to None, or to some inappropriate type, you will get
the same error message you get now from Python:
TypeError: unbound method talk() must be called with Cat instance as
__self__ (got Mammal instance instead)

A version of the ':vars: expression' lambda replacement gets suggested every
so often (and that exact syntax once by me). It ain't going to happnen, labmda
is more likely to be dropped than enhanced.

I think there is some value in having an anonymous function syntax,
not enough to justify a keyword, but certainly enough if it can be
done with a standard function definition, just leaving off the name.
Here is my latest favorite function definition syntax:

f(x):
return x**2

Lambda form:
:x:x**2

The lambda form is close enough to the standard form that it is
self-explanatory. So we can drop three more pages from the standard
introductory text.

-jackdied

[1] standard perl interview question, what is the difference between these
calls? All of these end up calling meow() with one argument, but they all
behave differently - sometimes very subtly.

$ob = new Cat;
$ob->meow(); # intsance method
Cat->meow(); # class method
Cat::meow($ob); # static method with first argument a Cat instance

The equivalent question in Python is -- explain the difference in
these uses of the talk method:

cat1.talk()
Mammal.talk(cat1)
bound_func = cat1.talk
unbound_func = Mammal.talk

This is a clear violation of the "explicit" rule. Yet it is one that
is justified, and one that I decided to keep in the prposed syntax.
The alternative is a special syntax to provide explicit binding in all
situations. After weeks of debate on the Prothon list, we were not
able to come up with anything better than Python's syntax.

So I have decided to accept the simple implicit rule -- you get a
bound function from an instance and the equivalent unbound function
from its parent class. Practicality beats purity.

-- Dave

 

[Greg Ewing]

The concept of using a
global variable __self__ is no surprise at all.

Except that __self__ can't be a global variable.
Implementing it that way would be a disaster.

 

[David MacQuigg]

Except that __self__ can't be a global variable.
Implementing it that way would be a disaster.

It seems to work in Michele Simionato's 'prototype' module.
{ comp.lang.python, 4/28/04, "Prototypes in Python"} It is global
only to the called function, but the key requirement is met -- no
alteration of the standard function calling sequence.

Why can't it be a true global?

-- Dave

 

[Greg Chapman]

It seems to work in Michele Simionato's 'prototype' module.
{ comp.lang.python, 4/28/04, "Prototypes in Python"} It is global
only to the called function, but the key requirement is met -- no
alteration of the standard function calling sequence.

Why can't it be a true global?

If I understand correctly, one drawback of his approach is that methods cannot
rebind globals. A STORE_GLOBAL opcode will store its value into the globs
dictionary created when the method was fetched; this dictionary is not the same
as the module's dictionary, so the changed global will only be visible within
that method (and within that method call: the next time the method is fetched, a
new globs will be created).

 

[Michele Simionato]

If I understand correctly, one drawback of his approach is that methods cannot
rebind globals. A STORE_GLOBAL opcode will store its value into the globs
dictionary created when the method was fetched; this dictionary is not the same
as the module's dictionary, so the changed global will only be visible within
that method (and within that method call: the next time the method is fetched, a
new globs will be created).

I believe you are right (haven't checked yet) but I guess I can always
change the globals from inside the method with something like

currentmodule=sys.modules["__name__"]
currentmodule.myglobal="something"

Not that changing globals from a method is good style.

BTW, I do agree that my hack is an evil hack, the interesting thing
about it (for me) was the fact that inside the method I could use
"super.method(*args)" instead of the ugly
"super(cls,self).method(*args)" syntax. I didn't think that was
possible at all.

I just wonder if it would be technically possible to implement
something
like that in C with a decent performance (maybe caching the method
access).
What do you think?

Michele Simionato

 

[David MacQuigg]

If I understand correctly, one drawback of his approach is that methods cannot
rebind globals. A STORE_GLOBAL opcode will store its value into the globs
dictionary created when the method was fetched; this dictionary is not the same
as the module's dictionary, so the changed global will only be visible within
that method (and within that method call: the next time the method is fetched, a
new globs will be created).

I believe you are right (haven't checked yet) but I guess I can always
change the globals from inside the method with something like

currentmodule=sys.modules["__name__"]
currentmodule.myglobal="something"

Not that changing globals from a method is good style.

The one limitation I see in the prototype module is that I can't save
an unbound function and later call it with my choice of bind object.
A global __self__ might help in this situation. e.g.

## This works:
bf = cat2.talk # Save a bound function.
def func(bf): # Pass it to another scope.
bf()
func(bf) # Call it.

## This does not:
uf = Cat.talk # Save an unbound function.
uf() #=>
# AttributeError: type object 'Cat' has no attribute 'name'

I would like to be able to do something like:
__self__ = cat2
uf()

Maybe there is some other trick to get this to work. It doesn't have
to be pretty. This is a seldom-needed call, mostly for debugging.

BTW, I do agree that my hack is an evil hack, the interesting thing
about it (for me) was the fact that inside the method I could use
"super.method(*args)" instead of the ugly
"super(cls,self).method(*args)" syntax. I didn't think that was
possible at all.

I don't care how evil the underlying machinery is. It works nicely at
the user level. This is something the prototype advocates have
missed. They see metaclasses or descriptors and they freak. If the
module had been written in C, they would not be complaining.

super.func() is cool. I would not bother my students with
super(cls,self).func(), but the way super is used, it looks simple,
and it is simple. I can have a hierarchy of classes, each one calling
a function from its parent, and without modifying anything but the
class headers, insert a new class in the middle of the hierarchy.

## New class:
class Feline(Animal):
....
def show():
super.show()
print "Felines:", Feline.numFelines
....
Animals: 2
Felines: 2 <== Line added to previous output.
Cats: 2

My interest in this "prototype" module is not the ability to clone
prototypes, but the simplification and unification of all function and
method calling styles. If we leave out the cloning stuff, does it
make the implementation problems simpler?

-- Dave

 

[James Moughan]

< The concept of using a

global variable __self__ is no surprise at all. The only thing new in
a class, compared to a module, is the instance variables, and they can
be explained in one paragraph. All methods look just like normal
functions. There is no need to explain "static methods" or any other
form of method. In fact, I use the term "function" rather than
"method" to emphasize the similarity.

I'm especially interested in feedback from users who are now learning
or have recently learned Python.

I'd fit there - I've been learning Pyhton for a few weeks, though it's
now my main language.

The first time I saw the 'self' syntax in Python I recoiled slightly,
I'll admit, and I took, oooh, a few tens of minutes to get used to
including it. But then I've been programming in Java since 'the
beginning' so loosing the long habits from that may have slowed me
down.

IMO the current syntax is very, very clear and makes it *easier* to
understand OOP than any other syntax which I've seen. If you
understand functions then you understand arguments; it's obvious how
that function would work if it weren't bound to a class, and it's a
simple leap to see the effects of placing a function into a class.

Also, it illuminates your programming in other languages. Once I saw
the Pythonic methods, it immediately occured to me how to do OOP in a
non-OOP language like C.

Overall IMO this particular syntax is one of the things which makes
Python ideal for teaching OOP. It's definitely better than waffling
about global variables which change depending on where you are in the
code. I think if I heard that from a lecturer then I'd switch off
like a light.

I already know these changes seem
trivial to many experts. I've also heard plenty from the people who
think Python is so complex that we need to start a whole new language
( www.prothon.org ).

I thought that was just some guy who thought it was too complex for
his compiler?

Python is a very deep language, but the complexity hits you so late,
and it's so comprehensible in terms of what's gone before that there's
no pain involved. ( esp. contrasted with something like C++ where I
had to set up rigorous empirical tests to work out what the heck the
basic structures of the laguage were doing...)

Just MHO.

Jam

 

[David MacQuigg]

This is exactly the kind of feedback I'm looking for. Thank you for
taking the time to recall your experiences in learning Python.

David MacQuigg wrote:
< The concept of using a

I'd fit there - I've been learning Pyhton for a few weeks, though it's
now my main language.

The first time I saw the 'self' syntax in Python I recoiled slightly,
I'll admit, and I took, oooh, a few tens of minutes to get used to
including it. But then I've been programming in Java since 'the
beginning' so loosing the long habits from that may have slowed me
down.

IMO the current syntax is very, very clear and makes it *easier* to
understand OOP than any other syntax which I've seen. If you
understand functions then you understand arguments; it's obvious how
that function would work if it weren't bound to a class, and it's a
simple leap to see the effects of placing a function into a class.

Also, it illuminates your programming in other languages. Once I saw
the Pythonic methods, it immediately occured to me how to do OOP in a
non-OOP language like C.

Overall IMO this particular syntax is one of the things which makes
Python ideal for teaching OOP. It's definitely better than waffling
about global variables which change depending on where you are in the
code. I think if I heard that from a lecturer then I'd switch off
like a light.

The statement about global variables is the only thing that surprises
me. Have you read the explanations of instance variables at the end
of http://ece.arizona.edu/~edatools/Python/PrototypeSyntax.htm
Does this change your thinking?

Explanation of Instance Variables in Python
-------------------------------------------
""" Some of the variables inside the functions in a class have a
self. prefix. This is to distinguish local variables in the function
from "instance variables". These instance variables will be found
when the function is called, by searching the instance which called
the function. The way this works is that calling the function from an
instance causes that instance to be passed as the first argument to
the function call. So if you call cat1.talk(), that is equivalent to
Cat.talk(cat1) If you call cat1.set_vars( "Garfield", "Meow"), that is
equivalent to Cat.set_vars(cat1, "Garfield", "Meow")

The "current instance" argument is automatically inserted as the first
argument, ahead of any other arguments that you may provide in calling
a method that is "bound" to an instance. Note: The distinction
between instances and classes is important here. If you call a
function from a class, that function is not bound to any instance, and
you have to supply the instance explicitly in the first argument (
Cat.talk(cat1) )

The variable name self is just a convention. As long as you put the
same name in the first argument as in the body of the definition, it
can be self or s or even _ The single underscore is handy if you
want to maximally suppress clutter. """

Explanation of Simplified Instance Variables
--------------------------------------------
""" Some of the variables inside the functions in a prototype have a
leading dot. This is to distinguish local variables in the function
from "instance variables". When a function is called from an instance
( cat1.talk() ) a special global variable __self__ is automatically
assigned to that instance ( __self__ = cat1 ) Then when the function
needs an instance variable ( .sound ) it uses __self__ just as if you
had typed it in front of the dot ( __self__.sound ) The leading dot
is just an abbreviation to avoid typing __self__ everywhere. """

( For more discussion of this topic see the thread "Explanation of
Instance Variables in Python", comp.lang.python, 4/28/04. )

Python is a very deep language, but the complexity hits you so late,
and it's so comprehensible in terms of what's gone before that there's
no pain involved. ( esp. contrasted with something like C++ where I
had to set up rigorous empirical tests to work out what the heck the
basic structures of the laguage were doing...)

Python is clearly superior to C++ and Java for what I need ( teaching
EE students and professional engineers ). Still, it isn't perfect. I
believe a good presentation of OOP could be done in 50 pages ( about
half of what is now in Learning Python, 2nd ed.) I've written an
explanation of everything basic in the proposed syntax, including
bound and unbound methods, in 7 pages. ( pages 2-8 of
http://ece.arizona.edu/~edatools/Python/Prototypes.doc ) It still
needs examples and exercises, but I probably won't continue if there
is not more interest from the Python community.

The key to this simplification is not having to discuss static methods
or any other method form. All methods are called just like normal
functions, which the students already understand at this point in the
course.

---------------------------

I think the learning process you describe above is quite common. I
went through the same initial steps. At first I was puzzled by the
strange 'self' first argument, and annoyed at typing 'self' in so many
places. Then I understood the magic first argument, and got used to
the extra typing. It did not occur to me that there was some
unnecessary complexity yet to come. I had not yet seen how 'self' is
handled in Ruby and in Prothon. I did not learn about static methods
and other forms until much later, and 'lambdas' were something weird
related to "lambda calculus". Part of my problem was that the
pressures from my regular job did not allow much time to be studying
Python - exactly the kind of pressure that my students and clients are
now facing.

I did not start using static methods until recently. I just did what
the textbook examples showed, and steered around the need for these
"advanced" method forms. A problem with this strategy is that the
need for static methods arises quite naturally in some fairly simple
programs. ( See the Animals_2.py example in the Prototypes document
above.) This is not like metaclasses, which really can be avoided by
beginners. It is more like telling new users they can't use the
letters q and w in their variable names. Sure, you can work around
it, but it constrains what you would naturally do just sitting down to
write your first Python programs.

The second problem with the "avoidance/denial" strategy is that it
involves moving the troublesome functions outside of the class
definition. This results in a less-than-optimal structure for your
program. Look at the show() methods in Animals_2.py They must be
callable without any specific instance. Moving them outside the
appropriate classes is not good programming. When you want to change
a class later, you will need to remember to look in some other place
for any "static methods" associated with the class.

If you could do me one more favor, please take a look at the
Animals_2.py example starting on page 7 of the Prototypes document at
http://ece.arizona.edu/~edatools/Python Can you write this in Python?
Give it a try, and let me know if you had any difficulties. Were
there any parts you had to stop and think, or look up in a manual?

I appreciate the effort it takes to thoroughly investigate this issue.

-- Dave

 

[Donn Cave]

Quoth David MacQuigg <[email protected]>:
....
| The second problem with the "avoidance/denial" strategy is that it
| involves moving the troublesome functions outside of the class
| definition. This results in a less-than-optimal structure for your
| program. Look at the show() methods in Animals_2.py They must be
| callable without any specific instance. Moving them outside the
| appropriate classes is not good programming. When you want to change
| a class later, you will need to remember to look in some other place
| for any "static methods" associated with the class.
|
| If you could do me one more favor, please take a look at the
| Animals_2.py example starting on page 7 of the Prototypes document at
| http://ece.arizona.edu/~edatools/Python Can you write this in Python?
| Give it a try, and let me know if you had any difficulties. Were
| there any parts you had to stop and think, or look up in a manual?

Mine came out 66 lines, most of it very introductory level
and nothing exotic. The first dozen lines deal with the
inventory problem, which I gather is the issue you're interested
in. I don't maintain a count in each class, but rather keep
the counts in a single data structure in the base class, and
I provide a single show() function that prints them. If I were
writing this in C++, I might make that a static function.
But then, I hate C++.

Now, you could argue that I changed the problem to suit my
purposes, but your approach repeats some stuff all over the
class hierarchy in a kind 'similar but not quite the same'
way that I don't like at all. If we need to implement some
functionality that's general to the hierarchy, then let's
implement it once. I can imagine in principle the need for
a class specific function. In such a case, I would define
the function above the class definition, and refer to it
inside a data structure at the class level. I have a vague
notion that there are newer Python features that address
these issues, but I'm not interested - it isn't important.

Donn Cave, (e-mail address removed)
-----------------------------------
def show():
for k, v in Animal.inventory.items():
if v == 1:
f = '%s: %s'
else:
f = '%ss: %s'
print f % (k, v)

class Animal:
home = 'Earth'
inventory = {}
def __init__(self):
self.register('Animal')
def register(self, name):
i = self.inventory.get(name, 0)
self.inventory[name] = i + 1

class Reptile(Animal):
pass

class Mammal(Animal):
def __init__(self, sound = 'Maa... Maa...'):
self.sound = sound
self.register('Mammal')
Animal.__init__(self)
def talk(self):
print 'Mammal sound: ', self.sound

class Bovine(Mammal):
pass

class Canine(Mammal):
pass

class Feline(Mammal):
genus = 'feline'
def __init__(self):
Mammal.__init__(self)
self.register('Feline')

class Cat(Feline):
def __init__(self, name='unknown', sound='Meow'):
Feline.__init__(self)
self.register('Cat')
self.name = name
self.sound = sound
def talk(self):
print 'My name is ...', self.name
print 'I am', self.genus, 'from', self.home
Mammal.talk(self)

a = Animal()
m = Mammal()
print 'm:',
m.talk()
f = Feline()
print 'f:',
f.talk()
c = Cat()
print 'c:',
c.talk()

show()

cat1 = Cat('Garfield', 'Purr')
cat1.talk()

 

[Greg Ewing]

Explanation of Instance Variables in Python
-------------------------------------------
""" Some of the variables inside the functions in a class have a
self. prefix. This is to distinguish local variables in the function
from "instance variables". These instance variables will be found
when the function is called...

Just because a particular writer uses more waffle than
necessary to explain something about Python doesn't mean
the thing being explained is complicated enough to require
that many words.

 

[David MacQuigg]

Quoth David MacQuigg <[email protected]>:
...
| The second problem with the "avoidance/denial" strategy is that it
| involves moving the troublesome functions outside of the class
| definition. This results in a less-than-optimal structure for your
| program. Look at the show() methods in Animals_2.py They must be
| callable without any specific instance. Moving them outside the
| appropriate classes is not good programming. When you want to change
| a class later, you will need to remember to look in some other place
| for any "static methods" associated with the class.
|
| If you could do me one more favor, please take a look at the
| Animals_2.py example starting on page 7 of the Prototypes document at
| http://ece.arizona.edu/~edatools/Python Can you write this in Python?
| Give it a try, and let me know if you had any difficulties. Were
| there any parts you had to stop and think, or look up in a manual?

Mine came out 66 lines, most of it very introductory level
and nothing exotic. The first dozen lines deal with the
inventory problem, which I gather is the issue you're interested
in. I don't maintain a count in each class, but rather keep
the counts in a single data structure in the base class, and
I provide a single show() function that prints them. If I were
writing this in C++, I might make that a static function.
But then, I hate C++.

Now, you could argue that I changed the problem to suit my
purposes, but your approach repeats some stuff all over the
class hierarchy in a kind 'similar but not quite the same'
way that I don't like at all. If we need to implement some
functionality that's general to the hierarchy, then let's
implement it once.

The "similar but not quite the same" for all the show() methods in my
example was deliberate, though somewhat artificial in this short
example. I wanted to discourage the "do it all with one external
function" idea, but it seems I failed.

In a more realistic, but longer example, each class will have a large
number of attributes that need a completely different display than any
other class. For example, the Mammal.show() method might output a
format with all sorts of information on temperature, pulse, and
respiration, and other mammal characteristics. This format would be
completely different than the format for Cat.show(), and different in
a way that you can't find a simple pattern for your all-in-one
function.

I can imagine in principle the need for
a class specific function. In such a case, I would define
the function above the class definition, and refer to it
inside a data structure at the class level. I have a vague
notion that there are newer Python features that address
these issues, but I'm not interested - it isn't important.

We can put the function outside the class definition, or leave it
inside and add the 'staticmethod' wrapper. I prefer the latter.
Either way its a pain. The pain is especially severe for my students
and clients, who don't have time to get proficient in Python, and only
figure out that they need a workaround *after* writing some code and
discovering the problem. Then they have to "refactor" their program,
or search for information on staticmethod (not remembering that
keyword of course).

This pain can be avoided by eliminating the need for static methods
entirely. That is one benefit of the proposed syntax. Static methods
add nothing to the fundamental capabilities of Python. They are there
to make up for a deficiency in the syntax.

PEP 318 adds function "decorators", which are basically syntax sugar
for staticmethod and other extensions of the normal function syntax.

Donn Cave, (e-mail address removed)
-----------------------------------
def show():
for k, v in Animal.inventory.items():
if v == 1:
f = '%s: %s'
else:
f = '%ss: %s'
print f % (k, v)

You left out the parts that are unique to each class.

class Animal:
home = 'Earth'
inventory = {}
def __init__(self):
self.register('Animal')
def register(self, name):
i = self.inventory.get(name, 0)
self.inventory[name] = i + 1

class Reptile(Animal):
pass

class Mammal(Animal):
def __init__(self, sound = 'Maa... Maa...'):
self.sound = sound
self.register('Mammal')
Animal.__init__(self)
def talk(self):
print 'Mammal sound: ', self.sound

class Bovine(Mammal):
pass

class Canine(Mammal):
pass

class Feline(Mammal):
genus = 'feline'
def __init__(self):
Mammal.__init__(self)
self.register('Feline')

class Cat(Feline):
def __init__(self, name='unknown', sound='Meow'):
Feline.__init__(self)
self.register('Cat')
self.name = name
self.sound = sound
def talk(self):
print 'My name is ...', self.name
print 'I am', self.genus, 'from', self.home
Mammal.talk(self)

How did you know to add self in Mammal.talk(self)? This is a perfect
example of an experienced Python programmer sailing right past a
problem that trips up less proficient users, and not even being aware
that the problem exists. What surprises me is that even after
pointing out a problem like this, I will get someone telling me --
It's real simple, dummy. Mammal is a class. When you call a method
from a class, you get an unbound method, which requires an explicit
'self'.

Yes, I know that. I remember it because I use Python almost every
day. My students and clients, three months after learning it, will
not remember.

I appreciate your time in working on this example. I hope that I was
able to communicate the reasons for my continuing dis-satisfaction
with existing method syntax.

I am beginning to see that my needs are somewhat special. Most users
get past these difficulties and never look back. My clients and
students, however, will always be in the "never fully proficient"
mode. This is not a lack of intelligence or motivation. IC designers
are some of the smartest and hardest-working technical professionals
on the planet. It is simply that their minds are 110% occupied with
the intricacies of their designs. Anything we can do to reduce the
time they spend on "tool problems" will be worth millions in
productivity.

-- Dave

 

[David MacQuigg]

Just because a particular writer uses more waffle than
necessary to explain something about Python doesn't mean
the thing being explained is complicated enough to require
that many words.

We disagree on what is "waffle" and what are beneficial extra words.
Your explanation, to me, seems more like a "man page" than a textbook
explanation.

For a more complete discussion of this topic, with some alternative
paragraphs, see the thread "Explanation of Instance Variables in
Python", 4/28/04.

-- Dave

 

[James Moughan]

The statement about global variables is the only thing that surprises
me.

You mentioned them in your first post in the thread, which gave me the
impression I expressed there; my bad, if that's not what you intended.

Have you read the explanations of instance variables at the end
of http://ece.arizona.edu/~edatools/Python/PrototypeSyntax.htm
Does this change your thinking?

Explanation of Simplified Instance Variables
--------------------------------------------
""" Some of the variables inside the functions in a prototype have a
leading dot. This is to distinguish local variables in the function
from "instance variables". When a function is called from an instance
( cat1.talk() ) a special global variable __self__ is automatically
assigned to that instance ( __self__ = cat1 ) Then when the function
needs an instance variable ( .sound ) it uses __self__ just as if you
had typed it in front of the dot ( __self__.sound ) The leading dot
is just an abbreviation to avoid typing __self__ everywhere. """

The changes would go beyond a simple change of syntax; it would break
the python functional object model, e.g. as you couldn't write a
global function and assign it to a class at runtime. (I'm not saying
this is a good thing to do I'm saying that this is not a trivial
matter of syntax.)

Python is clearly superior to C++ and Java for what I need ( teaching
EE students and professional engineers ). Still, it isn't perfect. I
believe a good presentation of OOP could be done in 50 pages ( about
half of what is now in Learning Python, 2nd ed.)

I believe a good working introduction could be done in 10, at most.
OOP is not complex. It's even simple enough for managers to (think
they) understand, which leads the slightly bizzare state of
programming today.

The key to this simplification is not having to discuss static methods
or any other method form. All methods are called just like normal
functions, which the students already understand at this point in the
course.

They are already called like normal functions, though. You can
copy-paste a method as a global function and it will still work. Your
syntax introduces a special case which makes them distinct, by use of
a specific method-only dot notation.

---------------------------

I think the learning process you describe above is quite common. I
went through the same initial steps. At first I was puzzled by the
strange 'self' first argument, and annoyed at typing 'self' in so many
places. Then I understood the magic first argument, and got used to
the extra typing. It did not occur to me that there was some
unnecessary complexity yet to come. I had not yet seen how 'self' is
handled in Ruby and in Prothon. I did not learn about static methods
and other forms until much later, and 'lambdas' were something weird
related to "lambda calculus".

There are static methods in Python? In my coding the major reason
for static methods is for data encapsulation for singleton-style
elements. Python has no data encapsulation worth anything, so static
methods don't do much that's useful.

The second problem with the "avoidance/denial" strategy is that it
involves moving the troublesome functions outside of the class
definition. This results in a less-than-optimal structure for your
program. Look at the show() methods in Animals_2.py They must be
callable without any specific instance. Moving them outside the
appropriate classes is not good programming. When you want to change
a class later, you will need to remember to look in some other place
for any "static methods" associated with the class.

If you could do me one more favor, please take a look at the
Animals_2.py example starting on page 7 of the Prototypes document at
http://ece.arizona.edu/~edatools/Python Can you write this in Python?
Give it a try, and let me know if you had any difficulties. Were
there any parts you had to stop and think, or look up in a manual?

I am not copy-pasting the whole thing, but I think this demonstrates
the functionality you want. It ain't pretty, but I'll discuss that in
a second.

class static:
def __init__(self, fun):
self.__call__ = fun

class Mammal:

numMammals = 0

def __init__(self):
Mammal.numMammals += 1

def show():
print "Inventory:"
print " Mammals:", Mammal.numMammals

show = static(show)

class Feline(Mammal):

numFelines = 0

def __init__(self):
Feline.numFelines += 1
Mammal.__init__(self)


def show():
Mammal.show()
print " Felines:", Feline.numFelines

show = static(show)


m = Mammal()
f = Feline()
print Mammal.numMammals, Feline.numFelines, '\n'

Feline.show()


I must note that Animals_2 is a total abuse of OOP techniques. You
continuously define classes to do the same thing, repeat the same code
in each, then give them slightly different names.

Also, your show method is IMO more than dubious. show does not
logically belong to Feline. As a result you are using a class to
display data about other classes to which it is not connected. This
is not OOP.

If I were teaching someone and they produced this structure then I'd
go over the rationale with them and try to figure out where they went
wrong in their planning of the program, and maybe go back and tutor
them on the basic ideas of OOP. I would not change the language to
make it easier to do in future.

 

[Donn Cave]

How did you know to add self in Mammal.talk(self)? This is a perfect
example of an experienced Python programmer sailing right past a
problem that trips up less proficient users, and not even being aware
that the problem exists. What surprises me is that even after
pointing out a problem like this, I will get someone telling me --
It's real simple, dummy. Mammal is a class. When you call a method
from a class, you get an unbound method, which requires an explicit
'self'.

Yes, I know that. I remember it because I use Python almost every
day. My students and clients, three months after learning it, will
not remember.

Maybe they won't, I don't know. But if it's difficult, it's
a benign sort of difficulty. You CAN understand how it works,
you can't use it WITHOUT understanding how it works, and the
more you use it, the more obvious it becomes (sail right past.)
This is what Python is all about - a consistent, explicit
system. If you're searching for unbearable difficulties, I'd
look for things you can use without understanding, like the
object/reference storage model, and its interactions with
mutable default parameters, +=, etc.

But at any rate, from my perspective, this is your problem,
not your students and clients problems. I've been hearing
for years from sophisticated users who think they know how
to speak for the unsophisticated users, and after a couple
decades I have come to be highly skeptical. I've dealt with
scientists myself, and I think I have an idea of what you're
talking about, the principles make sense, but the connection
between the general principles and the specific details ...
a black art. You're not unique here because you represent
non-programmers, you have a unique idea of how that relates to
the language. As would anyone. My own perspective is, present
the core language and leave them to solve problems in that model,
"bad programming" or not. The static method, for example, is
not really a core concept - not just the implementation, but
the basic principle. So don't go there. You'll be surprised
at how little pain they feel over software engineering issues.
Actually, my only Python-using colleague here, really the only
one who actively uses Python for our core software, doesn't even
use "class".

Donn Cave, (e-mail address removed)

 

[David MacQuigg]

The changes would go beyond a simple change of syntax; it would break
the python functional object model, e.g. as you couldn't write a
global function and assign it to a class at runtime. (I'm not saying
this is a good thing to do I'm saying that this is not a trivial
matter of syntax.)

I don't want to argue implementation details, as I am no expert, but I
think you are saying something is wrong at the user level, and that
puzzles me.

A global function, if I understand your terminology correctly, is one
defined at the module level, outside of any class. Such a function
cannot have instance variables. If you were to reference that
function from within a class, it would just act as a normal function
(a static method in Python terminology). I can't see the problem.
f = global_func
f = staticmethod(f)
hello

The difference in the proposed syntax is that it doesn't need the
staticmethod wrapper to tell the interpreter -- don't expect a special
first argument. In the new syntax all functions/methods will have the
same calling sequence.

I believe a good working introduction could be done in 10, at most.
OOP is not complex. It's even simple enough for managers to (think
they) understand, which leads the slightly bizzare state of
programming today.

I've looked at a few introductions to Python, and in my opinion
Learning Python, 2nd ed, by Mark Lutz and David Ascher is the best.
It strikes a good balance between the minimal presentation that
tersely covers the essentials for an experienced programmer vs the
long, windy introductions that put you to sleep with analogies to car
parts and other "objects". Lutz takes 95 pages to cover OOP. I think
I could do a little better, maybe 70 pages, but that may be just my
ego :>)

When you say ten pages, you must be making some very different
assumptions about the students or their prior background, or the
desired level of proficiency. The least I can imagine is about 30
pages, if we include exercises and examples. And that 30 assumes we
get rid of all the unneccesary complexity (static methods, lambdas,
etc.) that currently fills the pages of Learning Python.

They are already called like normal functions, though. You can
copy-paste a method as a global function and it will still work. Your
syntax introduces a special case which makes them distinct, by use of
a specific method-only dot notation.

Wow!! This is the complete opposite of what I am seeing. I see
functions and methods *not* having the same calling sequence. You
need to add the special first argument with most methods. You
*cannot* just copy and paste a global function into a class without
adding that special first argument, or the staticmethod wrapper.

There are static methods in Python? In my coding the major reason
for static methods is for data encapsulation for singleton-style
elements. Python has no data encapsulation worth anything, so static
methods don't do much that's useful.

We have the usual dose of terminology problems here. The term 'static
method' in Python may be different than in other languages. In Python
it just means a method that has no instance variables, no special
first argument, and an extra 'staticmethod' line, to tell the
interpreter not to expect a special first argument.

Python does have "encapsulation" but does not have "hiding", by my
understanding of these words. The idea is that __private variables
are easily identified to avoid accidents, but there is no attempt to
stop deliberate access to such variables. This is a design philosophy
that I like.

I am not copy-pasting the whole thing, but I think this demonstrates
the functionality you want. It ain't pretty, but I'll discuss that in
a second.

class static:
def __init__(self, fun):
self.__call__ = fun

class Mammal:

numMammals = 0

def __init__(self):
Mammal.numMammals += 1

def show():
print "Inventory:"
print " Mammals:", Mammal.numMammals

show = static(show)

class Feline(Mammal):

numFelines = 0

def __init__(self):
Feline.numFelines += 1
Mammal.__init__(self)


def show():
Mammal.show()
print " Felines:", Feline.numFelines

show = static(show)


m = Mammal()
f = Feline()
print Mammal.numMammals, Feline.numFelines, '\n'

Feline.show()


I must note that Animals_2 is a total abuse of OOP techniques. You
continuously define classes to do the same thing, repeat the same code
in each, then give them slightly different names.

This is a textbook introduction, not a real program. The purpose of
the example is to show a complete OOP hierarchy in a small example,
with a good selection of the method styles most needed in a real
program. The similarity between the show() methods in different
classes would not be so tempting to reduce to one global function if I
had made a larger example, with more radically different outputs from
each show function. I thought that just changing one string in each
function would be enough to say "This function is different."

You are not the only one who had this reaction. See my reply to Don
Cave above. I guess I need to thow in a little more "meat", so that
experienced programmers don't get distracted by the possibility of
making the whole program simpler by taking advantage of its
regularities. This is the same problem I've seen in many texts on
OOP. You really can't see the advantages of OOP in a short example if
you look at it with the attitude -- I can do that much more easily
without classes. It's when you get to really big complex hierarchies
that the advantages of OOP become clear.

Also, your show method is IMO more than dubious. show does not
logically belong to Feline. As a result you are using a class to
display data about other classes to which it is not connected. This
is not OOP.

I thought this part was pretty clear. The show() method at each level
calls the show() method one level up, then adds its own stuff at the
end. Feline.show() calls Mammal.show(), which prints lots of stuff
characteristic of mammals, all in a format unique to the Mammal class.
Mammal.show() in turn calls Animal.show(). At each level we have some
unique display of characteristics. The purpose is to have a call at a
particular level show all characteristics of the animal from that
level up.

If I were teaching someone and they produced this structure then I'd
go over the rationale with them and try to figure out where they went
wrong in their planning of the program, and maybe go back and tutor
them on the basic ideas of OOP. I would not change the language to
make it easier to do in future.

Both responses I have on this are basically experts saying -- you can
solve this particular problem more easily by restructuring it. I
should have been more clear. Imagine that each of these classes and
methods is fully expanded with lots of parameters to make each one
unique. Don't even think about re-structuring, unless you are trying
to tell me that the whole idea of having these structures in any
program is wrong. That would surprise me, since this "animals"
example is a common illustration of OOP.

What I'm looking for is not clever re-structuring, but just a
straightforward translation, and some comments along the way -- oh
yes, that is a little awkward having to use a staticmethod here. Wow,
you mean staticmethods aren't fundamentally necessary, just a bandaid
to make up for Python's deficiencies? That was my reaction when I
first saw Prothon.

Thanks again for your feedback.

-- Dave

 

[David MacQuigg]

Maybe they won't, I don't know. But if it's difficult, it's
a benign sort of difficulty. You CAN understand how it works,
you can't use it WITHOUT understanding how it works, and the
more you use it, the more obvious it becomes (sail right past.)

My point is that you don't *have to* deal with these difficulties,
even if they are "benign". There is nothing fundamentally missing if
we use a syntax that makes all function/method calls the same. Taking
out unnecessary complexity is a good thing, even something as benign
as the confusion over dividing two integers, and not getting a
fractional result.

Currently, I'm not including OOP in my lesson plan. For most design
engineers, it will be sufficient that they understand functions.
Mostly they will be modifying scripts to run the design tools, and we
can avoid using classes in any code the user is likely to be working
with.

Still, it would be nice if I could include OOP with just four more
hours total ( lecture, reading, and working exercises ). I believe
that is possible with simpler syntax. See the first ten pages of the
Prototypes chapter at http://ece.arizona.edu/~edatools/Python/ With
the current syntax there is no way I can get students to the level
they can "sail past" the problems in my Animals.py example.

This is what Python is all about - a consistent, explicit
system. If you're searching for unbearable difficulties, I'd
look for things you can use without understanding, like the
object/reference storage model, and its interactions with
mutable default parameters, +=, etc.

I agree Python is better than other languages in consistency and
simplicity, just not perfect.

I'm not aware of a simpler model for object references, but then I
wasn't aware that all functions/methods could be unified until I
looked at Prothon. Do you have an idea for simplification of the
object/reference model?

But at any rate, from my perspective, this is your problem,
not your students and clients problems.

I guess you know them better than I do. :>(

I've been hearing
for years from sophisticated users who think they know how
to speak for the unsophisticated users, and after a couple
decades I have come to be highly skeptical.

I don't consider myself a sophisticated user at all. I *have*
encountered many sophisticated users, not just Python, but programming
in general, who have not a clue as to what is easy or hard for users.
I suspect that there is a bit of elitism in the attitudes I have seen.
Now that they have learned their "stock-in-trade" they really don't
want to make it easier on others. In general, Python and its
community is far more open than others I have seen.

I've dealt with
scientists myself, and I think I have an idea of what you're
talking about, the principles make sense, but the connection
between the general principles and the specific details ...
a black art. You're not unique here because you represent
non-programmers, you have a unique idea of how that relates to
the language. As would anyone.

Whenever I hear that program usability is a "black art" I think of all
the really excellent programs I have used, and the straight-forward
things they do to improve usability, or I pick up a book like The Art
of Unix Programming, and get a refreshing discussion showing how
principles relate to specific examples. Or I look at what Linus
Torvalds has done with Linux, or GvR with Python. Sometimes you have
to swim against a flood of ignorance and apathy.

At the same time I'm acutely aware of my own limitations in designing
the perfect interface. I try to always keep an open mind to a better
idea coming from the most unexpected place. And I try to keep my
interfaces separable, so we can tear it all up and start over after
the first real user points out some obvious mistakes.

I've come up with an idea which simplifies Python's function/method
syntax. Actually, it wasn't my idea. It was Prothon (and they got it
from Ruby, Self, whatever). I am ruthlessly stealing Prothon's good
ideas, and I hope ignoring the bad. They are ruthlessly ripping off
Python. Long live open source!!

My own perspective is, present
the core language and leave them to solve problems in that model,
"bad programming" or not.

This could be an argument for Perl. :>)

The static method, for example, is
not really a core concept - not just the implementation, but
the basic principle. So don't go there. You'll be surprised
at how little pain they feel over software engineering issues.

I can accept this argument for metaclasses, and other advanced tricks
that the basic user really doesn't need. But I would include in the
core, most of what is in Learning Python by Mark Lutz. Static methods
are necessary because it is a very natural thing to write a method in
a class that needs to work without being bound to any particular
instance.

Yes, you can re-structure your program to avoid the need, but that is
like re-writing variable names to avoid the letter q. It's a pain,
and the resulting program is not as clear as the original. But worst
of all, it adds to the "benign" difficulties faced by the users I care
most about, those who are too busy to learn Python, but could benefit
enormously from it.

Again, thanks for all your help.

-- Dave