Another try at Python's selfishness

[Steven D'Aprano]

Yes, I 100% agree to that point!
But the point is, the current situation is not newbie-friendly (I can
tell, I am a newbie): I declare a method with 3 parameters but when I
call it I only pass 2 parameters. That's confusing.

Yes, I understand what you mean now. When I was a newbie, it took me a
little while to get the difference between ordinary functions and methods
too, although I had little OO experience before Python. I don't know if
that helped or hindered the learning experience.

With the current syntax, you have to learn some special behaviour:

class Foo:
def bar(self, x, y):
pass

but you call Foo().bar(x, y)

But with your syntax, you still have to learn special behaviour:


# outside the class definition
self.bar # look up "bar" in self's namespace. self must already exist.

# inside a class definition
class Foo:

self = Something() # normal class attribute "self" created
# this works because self is not a keyword
# Foo now has an attribute "self"

def self.bar(x, y):
# create a method "bar" with "self" in bar's namespace
# Foo now has an attribute "bar" (which is a method)
pass

self.bar = None # normal attribute access
# Foo attribute "self" is modified to have an attribute "bar"

# there is no conflict between self the attribute and self the
# special parameter because they live in different namespaces
# but looking at them, there is an apparent conflict


So, most of the time, foo.bar looks up attribute "bar" in foo; but in a
method definition, foo.bar assigns attribute "foo" in bar. That's special
behaviour too, and it seems to me probably harder to live with and even
more confusing than the behaviour you aim to remove.

 

[Ben Sizer]

The main reason (at least for me) is that there's simply too much
"magic" in it. Why does the expression left of the '.' get promoted to
the first parameter?

One of the reasons I like Lua is because it doesn't do this, instead
using the : operator to designate method-style calls.

eg.
a:foo(b, c) -- looks up foo within a, and calls it with (a, b, c) as
parameters
a.foo(b, c) -- looks up foo within a, and calls it with (b,c) as
parameters

This means there doesn't need to be a distinction between methods and
functions, just a different operator to treat a function as if it was a
method.

When I started out in Python I figured that I could just assign
functions to objects and treat them then as if they were methods, as I
would in Lua, but quickly learned that it wasn't that simple.

 

[Antoon Pardon]

One of the reasons I like Lua is because it doesn't do this, instead
using the : operator to designate method-style calls.

eg.
a:foo(b, c) -- looks up foo within a, and calls it with (a, b, c) as
parameters
a.foo(b, c) -- looks up foo within a, and calls it with (b,c) as
parameters

This means there doesn't need to be a distinction between methods and
functions, just a different operator to treat a function as if it was a
method.

That is nice. I wonder if the following is possible in Lua:

fun = a:foo
fun(b, c)

with the same effect as:

a:foo(b, c)

 

[bruno at modulix]

Having read previous discussions on python-dev I think I'm not the only
Python programmer who doesn't particularly like python's "self"
parameter:

bang ! You're dead !

(no no, just kidding !-)

class Foo:

old-style classes are deprecated.

class Foo(object):

def bar(self, a,b):
return a+b

(snip)

The point is, I _do_ think it's a good idea to explicitly write
"self.SomeMember" for member-access,

With Python's lookup rules, it couldn't be otherwise anyway !-)

so I thought: why can't we be
equally explicit about member function declaration?

Because there's no such thing as a "member function" in Python.

Wouldn't it be nice
if I could write (say, in Python 3k, or maybe later):

class Foo:
def self.bar(a,b):
return a+b
Foo().bar(1,2) => 3

'bar' is not an instance attribute, but a class attribute - it belongs
to *class* Foo, not to an instance of Foo.

BTW, 'self' does not yet exist (and cannot possibly exist) when this
code is eval'd (how could an instance of class Foo exists before class
Foo itself exists ?).

That way, the declaration would match the invocation (at least
syntactically), and the "magic"-feeling is gone. In the long run, the
"old-style" syntax (i.e. if there's no '.' in the method name) could be
used for static methods.
s/static/class/

What do you think?

That you should learn more about the inners of Python's object model
(and specially at the Descriptor protocol).

class Foo(object):
def __init__(self, a):
self.a = a

def bar(obj, b):
return obj.a + b

f = Foo(1)
bar(f, 2)

Foo.bar = bar
Foo.bar(f, 2)
f.bar(2)

The only 'magic' here is that when bar() is called *as an attribute of a
Foo instance*, the instance is passed as the first parameter. In fact,
this is much more explicit (well, IMHO) than the 'this' reference in
Java or C++.

 

[Alex Martelli]

which isn't needed at all. So far, the existence of x.y somewhere
in Python always implied that x was already introduced explicitly
in the program, and you suggest that we violate that both in the

Almost... import (and from) statements are exceptions to this.

import x.y

binds or rebinds name x "on the fly", as well as attribute y of x.


Alex

 

[Alex Martelli]

Sure, but now the call foo.bar
"call"?

has special meaning inside a def statement
than elsewhere. Elsewhere, foo.bar is an attribute access, looking up
attribute bar in foo's namespace.

or setting it, as in foo.bar=23, or setting both names, as in

import foo.bar

Using your syntax, in a def statement
foo.bar is a pair of declarations: it declares a name "foo", and it
declares a second name "bar".

"declares" isn't really pythonic -- let's talk about binding or setting
names, instead.

This inconsistency is, I think, worse than the implicit use of self.

I don't think there's any inconsistency in deciding that syntax x.y has
different meanings (as to what gets looked up or bound) in different
contexts, because it already does: mostly "look up y in namespace x",
but in x.y=... it's "bind y in namespace x" and in "import x.y" it's
"bind x AND then bind y in namespace y".

Since "def x.y(..." is currently a syntax error, it would introduce no
backwards compatibility to assign a meaning to it. Since the 'def'
statement binds something to the name that follows it (which currently
must be a plain identifier), that's what it should do if it allowed the
following name to be a compound one, too.

Unfortunately, none of this suggests that it's reasonable to have

def x.y(z): ...

mean the same as

def y(x, z): ...

and I have no idea of how it would generalize to def x.y.z(t): ...
(while import x.y.z generalizes in a pretty obvious way wrt import x.y).

So, I'm not supporting the OP's idea; just rejecting the specific
objections to it.


Alex

 

[Steven D'Aprano]

"call"?

Call? Who said anything about a call?

*wink*

Okay, poor choice of words. I'm not exactly sure what a better choice
would be. Token? Too specific. Maybe it would have been better to just
have just said "...but now foo.bar has ...".

or setting it, as in foo.bar=23,

Conceptually, both setting and getting an attribute involves a "look up"
in the general sense: you need to look up the attribute to get its value,
or to find out where to put its value. You are correct (of course!) that
foo.bar can either be a get or a set, but I'm doing lots of hand-waving
here and I didn't think it was necessary to get bogged down in too much
detail.

or setting both names, as in

import foo.bar

Ah, I completely forgot about import. But see below.

"declares" isn't really pythonic -- let's talk about binding or setting
names, instead.

Yes, you're right, it is a bad habit. Years of Pascal don't die easily.

I don't think there's any inconsistency in deciding that syntax x.y has
different meanings (as to what gets looked up or bound) in different
contexts, because it already does: mostly "look up y in namespace x",
but in x.y=... it's "bind y in namespace x"

Which from my perspective are conceptually two sides of the coin. The
model I have is "y" is a label in some namespace x, and you have to (in
some sense) look up where "y" should go regardless of whether you are
setting the value or getting the value.

Do you think this model is so far from the actual behaviour of Python that
it is useless? Or is it fair to lump getting and setting attributes/names
together?

and in "import x.y" it's
"bind x AND then bind y in namespace y".

I assume that's a typo and you mean "bind y in namespace x".

But even "import x.y" is conceptually a lookup, equivalent to x.y =
__import__("x.y"), with the understanding that x = __import__("x") is
automagically run first.

[hand-waving, hand-waving, hand-waving... of course imports do a lot more
than just setting a name in a namespace]

But in all three of:

foo.bar = something
something = foo.bar
import foo.bar

the hierarchy goes from left to right, with bar being "inside" foo, for
some meaning of inside. The Original Poster's syntax would reverse that,
with def foo.bar(x,y) creating parameter foo "inside" method bar.

 

[n.estner]

...

Unfortunately, none of this suggests that it's reasonable to have

def x.y(z): ...

mean the same as

def y(x, z): ...

Actually, it shouldn't. The idea was, that
def x.y(z): ...
(explicitly) introduces an unbound method. That's not introducing a new
conect to python, it's just making the difference between an unbound
method and a not-bindable function explicit.

Currently, "def(x,y): ..." can mean two different things: In the
context of a class, it introduces an unbound method, in global or local
contexts it introduces a function. I don't want to have a new syntax
for that, I want two different syntaxes for these two different
meanings.

and I have no idea of how it would generalize to def x.y.z(t): ...

Nor do I. Is that a problem?

 

[Donn Cave]

Probably my example wasn't clear, let's try another:

class A:
def test(a, **kwargs): return 1
class B:
def test(b, **kwargs): return 2
test(a=A(), b=B())

"self" isn't a keyword, so nothing should forbid this code. What is the
interpreter to do if it stumbles across this "test" call? I mean,
named-argument lookup is a tricky thing even if you do know what
function you're calling. If this would depend on one of the parameters,
I think it would become completely unintelligible. (you can think of
more complex examples yourself)

Still see no problem. Of course, it goes without saying that
Python 2.4 doesn't work this way, but given that it's theoretically
possible for f(a) to be resolved similarly to a.f, then I really
do not see what you're seeing here. The kwargs parameter appears
irrelevant from where I'm sitting.

That's because they're doing two different things: object.method() does
an attribute lookup, while function(parameter) looks for the function
in the current scope.

But current scope is actually a compound lookup - function scope,
global scope etc. Generalize it to object scope, and then you
can have a notation that expresses these things consistently - QED.

Donn Cave, (e-mail address removed)

 

[Terry Reedy]

(e-mail address removed) wrote:
Today, Python has a syntactic shortcut. If 'a' is an
instance of class 'A', a.f(x,y,z) is a shortcut for
A.f(a,x,y,z). If you don't use the shortcut, there is
no magic at all, just the unusual occurence of a type
check in Python!

As was once pointed out to me some years ago, when I wrote something
similar, a.f() is not just a shortcut for A.f(a) [a.__class__.f(a)]. The
latter only looks for f in the class A namespace while the former also
looks in superclass namespaces. The 'magical' part of accessing functions
via instances is the implementation of dynamic inheritance.

Terry Jan Reedy

 

[n.estner]

Still see no problem. Of course, it goes without saying that

Python 2.4 doesn't work this way, but given that it's theoretically
possible for f(a) to be resolved similarly to a.f, then I really
do not see what you're seeing here. The kwargs parameter appears
irrelevant from where I'm sitting.

Then either you didn't understand my answers, or I didn't understand
your idea. Could you summarize how exactly "f(x,y=z)" should be
resolved, i.e. where it should look for "f"?

 

[Alex Martelli]

would be. Token? Too specific. Maybe it would have been better to just
have just said "...but now foo.bar has ...".
Agreed.

model I have is "y" is a label in some namespace x, and you have to (in
some sense) look up where "y" should go regardless of whether you are
setting the value or getting the value.

Do you think this model is so far from the actual behaviour of Python that
it is useless? Or is it fair to lump getting and setting attributes/names
together?

I think it's reasonable: Python does first check if type(x).y is an
overriding descriptor (==one with a __set__ method), at least in the
mainstream case (oldstyle classic are weird;-).

I assume that's a typo and you mean "bind y in namespace x".

Yep. Differently from the other bots (tim and /f), I'm programmed to
randomly make errors once in a while -- makes it easier to pass the
Turing test (many people mistakenly think I'm human, since "to err is
human", while nobody would think THAT of impeccabile timbot and effbot).

But even "import x.y" is conceptually a lookup, equivalent to x.y =
__import__("x.y"), with the understanding that x = __import__("x") is
automagically run first.

[hand-waving, hand-waving, hand-waving... of course imports do a lot more
than just setting a name in a namespace]

Of course, but so do some other statements that ALSO bind a name (in the
current namespace only, in today's Python), such as class and def.
We're just focusing on the binding part;-).

But in all three of:

foo.bar = something
something = foo.bar
import foo.bar

the hierarchy goes from left to right, with bar being "inside" foo, for
some meaning of inside. The Original Poster's syntax would reverse that,
with def foo.bar(x,y) creating parameter foo "inside" method bar.

True. Of course, the OP might argue that if x.f() can CALL f(x)
[placing x ``inside'' f], it's fair that a def of a composite name
should also ``swap insideness and outsideness'' similarly. But, as I
said, "I'm not supporting the OP's idea; just rejecting the specific
objections to it". For once, I have a hard time articulating exactly why
I'd dislike such semantics for hypothetic syntax "def x.y", besides
minor points such as the difficulties wrt generalizing to "def x.y.z"...
but I do know which objections are NOT the ones which make me feel such
dislike!-)


Alex

 

[Alex Martelli]

your idea. Could you summarize how exactly "f(x,y=z)" should be
resolved, i.e. where it should look for "f"?

Lexically: local scope, outer functions outwards, globals, builtins.


Alex

 

[Alex Martelli]

As was once pointed out to me some years ago, when I wrote something
similar, a.f() is not just a shortcut for A.f(a) [a.__class__.f(a)]. The
latter only looks for f in the class A namespace while the former also
looks in superclass namespaces.
Nope:

.... def f(self): print 'b.f'
....
Inheritance applies in any lookup of an attribute in a class, just as
well as on an instance of the class.

Alex

 

[Dennis Lee Bieber]

Then either you didn't understand my answers, or I didn't understand
your idea. Could you summarize how exactly "f(x,y=z)" should be
resolved, i.e. where it should look for "f"?

About the only way I can see what you ask for working would require
the interpreter to do the following sequence:

look in current namespace for object "f";
if found, call it with arguments of "x" and (something -- "y=z" is
only meaningful in the declaration of a default value).
if NOT found, retrieve the object "x";
examine object "x" namespace for "f";
if found, call it with arguments (I suspect it will still have pass
both "x" and whatever, to ensure that the method knows which object the
invocation is to affect); [NOTE: this is no different from current
notation of x.f(whatever) -- f() still receives (x, whatever) as
parameters]
if NOT found, raise exception.
--

 

[Donn Cave]

Quoth (e-mail address removed):
| > Still see no problem. Of course, it goes without saying that
| > Python 2.4 doesn't work this way, but given that it's theoretically
| > possible for f(a) to be resolved similarly to a.f, then I really
| > do not see what you're seeing here. The kwargs parameter appears
| > irrelevant from where I'm sitting.
|
| Then either you didn't understand my answers, or I didn't understand
| your idea. Could you summarize how exactly "f(x,y=z)" should be
| resolved, i.e. where it should look for "f"?

a.f ==> f(a)

I would agree that I didn't understand your answers, but they weren't
really answers so much as questions, along the lines of ``well then,
how would this work?'' I seem to have missed what you were driving at,
but maybe if you were to just came out and explain the point?

Of course the whole business is kind of a joke, since there is no way
anyone in their right mind would wish to change Python's notation for
such trivial reasons, but there actually are languages that resolve
functions based on argument types. Overloading in C++ is somewhat
like this, Haskell's typeclass mechanism, and there is a ``multiple
dispatch'' model that I have no experience with but is not without its
supporters.

Donn Cave, (e-mail address removed)

 

[Jens Theisen]

Yes, I 100% agree to that point!
But the point is, the current situation is not newbie-friendly (I can
tell, I am a newbie): I declare a method with 3 parameters but when I
call it I only pass 2 parameters. That's confusing. If I declare a
member variable, I write: "self.x = ValueForX", why can't I be equally
explicit for declaring member functions?

For someone new to OO in general it might as well be something good, so he
realises that there actually really is a hidden parameter. After all,
there is something to understand with "self", and this discrapency between
the number of arguments and parameters puts newbies to it.

Jens

 

[n.estner]

Quoth (e-mail address removed):
| > Still see no problem. Of course, it goes without saying that
| > Python 2.4 doesn't work this way, but given that it's theoretically
| > possible for f(a) to be resolved similarly to a.f, then I really
| > do not see what you're seeing here. The kwargs parameter appears
| > irrelevant from where I'm sitting.
|
| Then either you didn't understand my answers, or I didn't understand
| your idea. Could you summarize how exactly "f(x,y=z)" should be
| resolved, i.e. where it should look for "f"?

a.f ==> f(a)

So, if the compiler recognizes the tokens "a.f", it should treat them
as if it found the tokens "f(a)". But that wouldn't do what you want.
You want it to do something different if it found the tokens "f(a)".

I would agree that I didn't understand your answers, but they weren't
really answers so much as questions, along the lines of ``well then,
how would this work?'' I seem to have missed what you were driving at,
but maybe if you were to just came out and explain the point?

You do know what a rethorical question is?

The first point is: Python has global functions, as well as methods. If
f(a) should look up f inside a first, that would shadow any global or
local f. That's bad, because python is dynamically typed, and you
sometimes down' know what "a" is. Things like "open(filename)" or
"tuple(a,b,c)" could behave completely unexpected, if "filename" had an
"open" or "a" a "tuple" attribute.

The other point is: Python supports named arguments. The algorithm to
put positional and named arguments into the right parameters is rather
tricky, and it _must_ know the function that's being called to work.
So, you simply can't make function lookup depend on the parameters,
it's a hen-egg problem.

Of course the whole business is kind of a joke, since there is no way
anyone in their right mind would wish to change Python's notation for
such trivial reasons, but there actually are languages that resolve
functions based on argument types. Overloading in C++ is somewhat
like this, Haskell's typeclass mechanism, and there is a ``multiple
dispatch'' model that I have no experience with but is not without its
supporters.

Yes, but both C++ and Haskell are statically typed, and neither
supports named arguments.
(We're talking about one function operating on one object, so I don't
see what multiple dispatch has to do with this)

 

[Donn Cave]

Quoth (e-mail address removed):
....
| The first point is: Python has global functions, as well as methods. If
| f(a) should look up f inside a first, that would shadow any global or
| local f. That's bad, because python is dynamically typed, and you
| sometimes down' know what "a" is. Things like "open(filename)" or
| "tuple(a,b,c)" could behave completely unexpected, if "filename" had an
| "open" or "a" a "tuple" attribute.

Well, of course we have to assume as part of the proposal that
it would documented and actually would be expected, to the extent
that one is entitled to expect anything in Python. It seems to
me that if you bring an even slightly open mind to the question,
when we apply "open" to an object that supports "open" with its
own function, what could be more appropriate than to call that
function? I will allow that there is some potential for problems
here, but there may be ways to finesse them. Haven't thought real
hard about it.

| The other point is: Python supports named arguments. The algorithm to
| put positional and named arguments into the right parameters is rather
| tricky, and it _must_ know the function that's being called to work.
| So, you simply can't make function lookup depend on the parameters,
| it's a hen-egg problem.

Ah. I see no reason this system needs to support a named first
parameter, though, so I suppose a function application like f(a=x,...)
would be exempt from parameter lookup. This seems like an implementation
detail.

| > Of course the whole business is kind of a joke, since there is no way
| > anyone in their right mind would wish to change Python's notation for
| > such trivial reasons, but there actually are languages that resolve
| > functions based on argument types. Overloading in C++ is somewhat
| > like this, Haskell's typeclass mechanism, and there is a ``multiple
| > dispatch'' model that I have no experience with but is not without its
| > supporters.
|
| Yes, but both C++ and Haskell are statically typed, and neither
| supports named arguments.
| (We're talking about one function operating on one object, so I don't
| see what multiple dispatch has to do with this)

Come on, you know that in the unlikely event this idea were to be seriously
considered, someone would be hot to extend it to multiple objects before
it even got implemented on one, but in any case, single dispatch sounds
like just a restricted case of multiple dispatch, so if the latter is
feasible, so is the former. I've heard talk about a form of static typing
for Python, and obviously that has the potential to considerably enhance
the possibilities in this area.

Donn Cave, (e-mail address removed)

 

[Nick Craig-Wood]

After messing around with Javascript (many magical
variables that suddenly show up in your namespaces!), I
began to *seriously* appreciate Python's design. Having
self as an explicit parameter is beautiful self-documenting
design.

I have to agree...

Here is my experience with the same idea in C++...

I've converted lots of C code written in an object oriented style into
C++. In C you pass round a pointer to a struct, which gets
transformed into the new class on conversion. Typically the code is
then full of this->member = that, etc. This is the python equivalent
of self.member = that. At this point I usually remove all the this->
from the code which then causes it to malfunction horribly at run-time
(it compiles fine) because of all the name space clashes between the
things in the class and the local variables. The compiler doesn't
warn about this - its perfectly legal C++.

In a lot of big C++ programs various conventions are used to try to
help with this - naming all parameters to functions _name rather than
name, or using this->member rather than member. The python way of
enforcing self.member is much cleaner and never comes back to bite you!