Another try at Python's selfishness

[n.estner]

Having read previous discussions on python-dev I think I'm not the only
Python programmer who doesn't particularly like python's "self"
parameter:

class Foo:
def bar(self, a,b):
return a+b
Foo().bar(1,2) => 3

The main reason (at least for me) is that there's simply too much
"magic" in it. Why does the expression left of the '.' get promoted to
the first parameter? It even goes further:

Foo.bar(Foo(), 1,2)

works, but:

Foo.bar(1,2,3)

doesn't, just because of the "magical first parameter" in a member
function. But:

Foo.__dict["bar"]__(1,2,3)

Does work.

The point is, I _do_ think it's a good idea to explicitly write
"self.SomeMember" for member-access, so I thought: why can't we be
equally explicit about member function declaration? Wouldn't it be nice
if I could write (say, in Python 3k, or maybe later):

class Foo:
def self.bar(a,b):
return a+b
Foo().bar(1,2) => 3

That way, the declaration would match the invocation (at least
syntactically), and the "magic"-feeling is gone. In the long run, the
"old-style" syntax (i.e. if there's no '.' in the method name) could be
used for static methods.

What do you think?

 

[Donn Cave]

Having read previous discussions on python-dev I think I'm not the only
Python programmer who doesn't particularly like python's "self"
parameter:

class Foo:
def bar(self, a,b):
return a+b
Foo().bar(1,2) => 3

The main reason (at least for me) is that there's simply too much
"magic" in it. Why does the expression left of the '.' get promoted to
the first parameter? It even goes further:

Foo.bar(Foo(), 1,2)

works, but:

Foo.bar(1,2,3)

doesn't, just because of the "magical first parameter" in a member
function. But:

Foo.__dict["bar"]__(1,2,3)

Does work.

The point is, I _do_ think it's a good idea to explicitly write
"self.SomeMember" for member-access, so I thought: why can't we be
equally explicit about member function declaration? Wouldn't it be nice
if I could write (say, in Python 3k, or maybe later):

class Foo:
def self.bar(a,b):
return a+b
Foo().bar(1,2) => 3

That way, the declaration would match the invocation (at least
syntactically), and the "magic"-feeling is gone. In the long run, the
"old-style" syntax (i.e. if there's no '.' in the method name) could be
used for static methods.

What do you think?

I think you make it too complicated. Why shouldn't all
functions be declared and called in the same way, that
would be the simplest thing for everyone.

class Foo
def bar(self, a, b):
return a + b
bar(Foo(), 1, 2) => 3

The virtues of this consistency become more apparent in a
more complex functional context:

sys.stdin.write(open(file, 'r').read().split(sep)[0])

vs.
write(sys.stdin, split(read(open(file, 'r')))[0])

Donn Cave, (e-mail address removed)

 

[Paul McGuire]

Having read previous discussions on python-dev I think I'm not the only
Python programmer who doesn't particularly like python's "self"
parameter:

How about this decorator-based approach (still need to pick *some* name for
self, I picked "__").

-- Paul



def memberFunction(f):
def func(self,*args,**kwargs):
globals()["__"] = self
return f(*args,**kwargs)
func.__name__ = f.__name__
func.__doc__ = f.__doc__
func.__dict__.update(f.__dict__)
return func


class Test:
@memberFunction
def __init__(x,y):
__.x = x
__.y = y

@memberFunction
def mult():
"Multiply the two member vars"
return __.x * __.y


t = Test(5,4)
print t.mult()
print dir(t)
print t.mult.__doc__

 

[n.estner]

write(sys.stdin, split(read(open(file, 'r')))[0])

So, if I got you right, the interpreter would have to interpret this
line like this:
1. Evaluate the first parameter of the first function (sys.stdin)
2. Look up the attribute "write" in this object
3. evaluate the first parameter of the split function ->
4. evaluate the first parameter of the read function ->
5. evaluate file (I guess this is a local string variable?)
6. try attribute lookup "open" on the string
7. fail -> call the global "open" function
8. lookup "read" in that object, call it
9. attribute lookup "split" on the returned object
10. call __getitem__(0)
11. pass the parameters to the write function from (1)

Is this correct?
My main problems are that you have to "guess" at step 6/7, and that the
order of evaluation makes me a little dizzy ;-) Also, member functions
seem to "shadow" global ones, what if I wanted to use some global
"split" function I've written myself instead of the string's one. If I
was mean, I'd ask you what this one does:
class A:
def test(self, this): return 1
class B:
def test(this, self): return 2
test(self=A(), this=B())

The current call syntax at least can be read from left-to-right, and
you always know whether you call a member function or a global one.

 

[n.estner]

Definitely looks interesting. I'd like it more if it was more explicit,
but still, it does look nice.

I guess you could make it recursion-safe if you saved/restored the
global "__" variable before/after calling the actual function, and
probably there's a way to make it thread-safe, too. But how would you
make it generator-safe? Also, wouldn't lambda's you pass around from
object to object be bound to use the wrong instance? How would you fix
this code:

class TestA:
@memberFunction
def do(f):
print f()


class TestB:
@memberFunction
def do(x):
x.do(lambda : __)

TestB().do(TestA()) # should print out TestB, does print out TestA


There really should be something like a wiki for
"selfishness"-proposals with pros and cons, so I could have looked at
some before thinking about my own one...

 

[DH]

Having read previous discussions on python-dev I think I'm not the only
Python programmer who doesn't particularly like python's "self"
parameter:

class Foo:
def bar(self, a,b):
return a+b
Foo().bar(1,2) => 3

The main reason (at least for me) is that there's simply too much
"magic" in it. Why does the expression left of the '.' get promoted to
the first parameter? It even goes further:

Foo.bar(Foo(), 1,2)

works, but:

Foo.bar(1,2,3)

doesn't, just because of the "magical first parameter" in a member
function. But:

Foo.__dict["bar"]__(1,2,3)

Does work.

The point is, I _do_ think it's a good idea to explicitly write
"self.SomeMember" for member-access, so I thought: why can't we be
equally explicit about member function declaration? Wouldn't it be nice
if I could write (say, in Python 3k, or maybe later):

class Foo:
def self.bar(a,b):
return a+b
Foo().bar(1,2) => 3

That's similar to ruby. Really this isn't going to change in python,
at least not anytime soon. If it really bothers you, then ruby is
something to look into.
But I think most people who don't like the extraneous 'self' in python
just consider it a minor inconvenience and don't even notice it after
using python for a while. It is only when you switch between python and
other languages that you notice it again.

If python extends decorators to allow them to be applied to classes as
well as functions (which is more likely to happen perhaps), then you'll
see a bunch of useful hacks pop up to eliminate the need for 'self' in
method declarations.

 

[Donn Cave]

write(sys.stdin, split(read(open(file, 'r')))[0])

So, if I got you right, the interpreter would have to interpret this
line like this:
1. Evaluate the first parameter of the first function (sys.stdin)
2. Look up the attribute "write" in this object
3. evaluate the first parameter of the split function ->
4. evaluate the first parameter of the read function ->
5. evaluate file (I guess this is a local string variable?)
6. try attribute lookup "open" on the string
7. fail -> call the global "open" function
8. lookup "read" in that object, call it
9. attribute lookup "split" on the returned object
10. call __getitem__(0)
11. pass the parameters to the write function from (1)

Is this correct?
My main problems are that you have to "guess" at step 6/7, and that the
order of evaluation makes me a little dizzy ;-) Also, member functions
seem to "shadow" global ones, what if I wanted to use some global
"split" function I've written myself instead of the string's one. If I
was mean, I'd ask you what this one does:
class A:
def test(self, this): return 1
class B:
def test(this, self): return 2
test(self=A(), this=B())

That usage (self is second parameter to B.test) is bound
to cause trouble in general, but in this case doesn't have
any effect I can see. The function call "test" would be
resolved from its first parameter, instance of A, and that
function would return 1. One of us is missing something
here, could be me.

The current call syntax at least can be read from left-to-right, and
you always know whether you call a member function or a global one.

That's exactly the problem, it doesn't read from left to right,
because we swap back and forth between function(parameter and
parameter.function notation. It only works if you don't have
to mix the two. I'm saying, pick one and use it consistently.

Donn Cave, (e-mail address removed)

 

[Paul McGuire]

Definitely looks interesting. I'd like it more if it was more explicit,
but still, it does look nice.

I guess you could make it recursion-safe if you saved/restored the
global "__" variable before/after calling the actual function, and
probably there's a way to make it thread-safe, too. But how would you
make it generator-safe? Also, wouldn't lambda's you pass around from
object to object be bound to use the wrong instance? How would you fix
this code:

class TestA:
@memberFunction
def do(f):
print f()


class TestB:
@memberFunction
def do(x):
x.do(lambda : __)

TestB().do(TestA()) # should print out TestB, does print out TestA


There really should be something like a wiki for
"selfishness"-proposals with pros and cons, so I could have looked at
some before thinking about my own one...

Here's another pass:

__ = None #initialize global __
def memberFunction(f):
def func(self,*args,**kwargs):
global __
save__ = __
globals()["__"] = self
try:
return f(*args,**kwargs)
finally:
__ = save__

func.__name__ = f.__name__
func.__doc__ = f.__doc__
func.__dict__.update(f.__dict__)
return func


class Test:
@memberFunction
def __init__(x,y):
__.x = x
__.y = y

@memberFunction
def mult():
"Multiply the two member vars"
return __.x * __.y


t = Test(5,4)
print t.mult()
print dir(t)
print t.mult.__doc__

class TestA:
@memberFunction
def do(f):
print f()


class TestB:
@memberFunction
def do(x):
z = __ # lambda's shouldn't directly reference '__'
x.do(lambda : z)

TestB().do(TestA())

Prints out:
20
['__doc__', '__init__', '__module__', 'mult', 'x', 'y']
Multiply the two member vars
<__main__.TestB instance at 0x009DA4E0>

 

[Terry Hancock]

But I think most people who don't like the extraneous
'self' in python just consider it a minor inconvenience
and don't even notice it after using python for a while.

After messing around with Javascript (many magical
variables that suddenly show up in your namespaces!), I
began to *seriously* appreciate Python's design. Having
self as an explicit parameter is beautiful self-documenting
design.

The convention that the self parameter is always "self" with
4 characters instead of, say, __ or _ or s can get annoying,
but of course, you are actually free to break this
convention. The advantage to keeping it, of course, is that
anybody can google "python self" and immediately get an
explanation of what it is, whereas googling for "python _"
is less likely to be helpful.

In fact, I think this ease of finding documentation is one
of the reasons why Python's heavy use of keywords (rather
than symbolic operators) is so useful. Recent additions like
"list comprehensions" and "generators" create a lot of
newbie confusion not least because newbies simply don't know
that [x for x in mylist] is *called* a "list comprehension"
(if they knew what it was called, they'd answer the question
for themselves).

They see the code and don't have any idea what it is. It
adds to the basic or core knowledge that you have to wade
through before you can say you "know Python" even at the
trivial level.

One of the things that attracted me to Python (at v1.5.2)
was the fact that the core language was so small and simple
that I could learn it in just a few days. The more
syntactical sugar added, the less that will be true, though.

I don't think we've busted the system yet, but I think
there's some reason for caution against making things any
*more* complicated. I still see "newbie-friendliness" as a
MAJOR plus for Python -- it increases the chance that users
of your software will become contributors.

I mean, even Perl was probably pretty simple in the
beginning. ;-)

 

[Magnus Lycka]

What do you think?

The impression I get from your suggestion is that you
haven't really understood Python. I'm sure that some
things could be better designed or better documented,
but your suggestions would actually make things worse.
Sorry.

Today, Python has a syntactic shortcut. If 'a' is an
instance of class 'A', a.f(x,y,z) is a shortcut for
A.f(a,x,y,z). If you don't use the shortcut, there is
no magic at all, just the unusual occurence of a type
check in Python! If you use the shortcut, Python code
looks just like most other OO languages. This isn't
very magical.

What you suggest doesn't remove magic, but it introduces
a bunch of inconsistencies!

But:
Foo.__dict["bar"]__(1,2,3)
Does work.

I agree that this isn't completely clean, and there are
some other ways in which the distinction between functions,
unbound methods and bound methods seem a bit quirky to me.
I don't think the problem is in the syntax though, and if
it had been a real problem, it would had been solved.

class Foo:
def self.bar(a,b):
return a+b
Foo().bar(1,2) => 3

First of all, you are using a really poor example of a "method",
since it doesn't use any attributes of the Foo instance. That
function doesn't seem to belong in that class at all. If it
should be in the class, it should be a static method, so self
should not be involved at all. In modern Python it would look
like this:

class Foo(object):
@staticmethod
def bar(a, b):
return a+b

Anyway, my impression is that you haven't fully understood how
namespaces work in Python, at least not the distinction between
class namespaces and instance namespaces. Please study this a
bit more. Hopefully, you'll find enlightenment and things will
fall into place for you.

You are really giving "self" a magic meaning with your suggestion
which isn't needed at all. So far, the existence of x.y somewhere
in Python always implied that x was already introduced explicitly
in the program, and you suggest that we violate that both in the
"def self.x"-row, and inside the methods when we access attributes.

Provoking a language can be a way of learning it, and provoking
people can sometimes lead to new insights, but it's important to
have an open mind. When learning a new language, it's probably
better to try to adapt oneself to it, rather than the other way
around.

 

[n.estner]

That usage (self is second parameter to B.test) is bound

to cause trouble in general, but in this case doesn't have
any effect I can see. The function call "test" would be
resolved from its first parameter, instance of A, and that
function would return 1. One of us is missing something
here, could be me.

Probably my example wasn't clear, let's try another:

class A:
def test(a, **kwargs): return 1
class B:
def test(b, **kwargs): return 2
test(a=A(), b=B())

"self" isn't a keyword, so nothing should forbid this code. What is the
interpreter to do if it stumbles across this "test" call? I mean,
named-argument lookup is a tricky thing even if you do know what
function you're calling. If this would depend on one of the parameters,
I think it would become completely unintelligible. (you can think of
more complex examples yourself)

That's exactly the problem, it doesn't read from left to right,
because we swap back and forth between function(parameter and
parameter.function notation.

That's because they're doing two different things: object.method() does
an attribute lookup, while function(parameter) looks for the function
in the current scope.

 

[n.estner]

Yes, that's what I had in mind when I said it could be made
recursion-safe. It's still not thread-safe, but I think that could be
done too, using thread-local-variables instead of globals.

class TestB:
@memberFunction
def do(x):
z = __ # lambda's shouldn't directly reference '__'
x.do(lambda : z)

Yes, that's what I meant when I said it wasn't lambda-safe. That means
a completely legal and (in my code) common expression can behave
totally unexpected. Would be a no-go for me.

But I think it gets worse:

class TestA:
@memberFunction
def do():
yield 1
yield 2
yield __


class TestB:
@memberFunction
def bar():
for x in TestA().do():
print x

TestB().bar()

Doesn't behave as expected, and the only way I can see to fix it would
be to declare a local function inside TestA's "do" function...

 

[n.estner]

First of all, you are using a really poor example of a "method",

since it doesn't use any attributes of the Foo instance.

Agreed. I tried to post a short example, and it obviously was to short
to make my point clear. lets take a longer one. Current syntax:

class Pair:
def __init__(self, a,b):
self.a = a
self.b = b

def sum(self):
return self.a + self.b

def product (this):
return this.a + this.b

My alternative syntax suggestion would be this one:

class Pair:
def self.__init__(a,b):
self.a = a
self.b = b

def self.sum():
return self.a + self.b

def this.product ():
return this.a + this.b

You are really giving "self" a magic meaning with your suggestion
which isn't needed at all.

No. I hope this is clearer in the example above. "self" shouldn't be a
keyword. It's a special kind of argument now, so why shouldn't we
explicitly _declare_ that it's a special kind of argument? (as explicit
is better than implicit)

So far, the existence of x.y somewhere
in Python always implied that x was already introduced explicitly
in the program,

Yes, but y(x) also implies that x and y have been introduced explicitly
before, unless it's in a "def" statement. The def statement always
introduces new variables.

and you suggest that we violate that both in the
"def self.x"-row, and inside the methods when we access attributes.

def x(self):
declares two new identifiers, "x" and "self"

Why shouldn't
def self.x():
declare two new identifiers ("x" and "self"), too?

Attribute acces wouldn't change.

 

[Steven D'Aprano]

Today, Python has a syntactic shortcut. If 'a' is an
instance of class 'A', a.f(x,y,z) is a shortcut for
A.f(a,x,y,z).

It is easy to work around (break?) that behaviour:

class A(object):
def foo(self):
print "normal method foo"

a = A()

def foo(self):
print "special method foo"

from new import instancemethod
a.foo = instancemethod(foo, a)

Now we can see that a.foo() is no longer a shortcut for A.foo(a):
normal method foo


So instances can have methods that they don't inherit from their class!

 

[Steven D'Aprano]

Probably my example wasn't clear, let's try another:

class A:
def test(a, **kwargs): return 1
class B:
def test(b, **kwargs): return 2
test(a=A(), b=B())

"self" isn't a keyword, so nothing should forbid this code. What is the
interpreter to do if it stumbles across this "test" call?

You could try running it to see:
.... def test(a, **kwargs): return 1
........ def test(b, **kwargs): return 2
....Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'test' is not defined


Oops! You have defined a name "test" in two namespaces, the class A and
the class B, but there is no name "test" in the global namespace you are
trying to run it in.

Since namespaces are critical to Python, your test code is a problem that
just cannot happen in Python. It is a non-issue. Python will not get
confused between the two definitions of test.

I mean,
named-argument lookup is a tricky thing even if you do know what
function you're calling. If this would depend on one of the parameters,
I think it would become completely unintelligible. (you can think of
more complex examples yourself)


That's because they're doing two different things: object.method() does
an attribute lookup, while function(parameter) looks for the function
in the current scope.

No, function(parameter) does not look for a function. It looks for any
object at all, and then tries to call it.

Both calls do almost the same thing. object.method() looks up the name
"method" in the object namespace, function(parameter) looks up the name
"function" in the current namespace. Neither "method" nor "function" must
be methods or functions, although if they are not callable objects,
calling them will raise an exception. But regardless of what sort of
objects they are, the look up proceeds in the same fashion.

Unless you understand namespaces, you don't understand Python, and any
criticism is likely to be due to misunderstanding.


As near as I can tell, your original complaint might be solved simply: it
seems to me that you are concerned about that extraneous "self" parameter
for methods that don't need it:

class Foo:
def bar(self, a,b):
return a+b
Foo().bar(1,2) => 3

If bar doesn't need the instance or class Foo, perhaps it would be better
off as an ordinary function rather than bound to a class.

But if bar does need to be a method, perhaps you want something like this:


# for Python 2.2 and up
class Foo:
def bar(a,b):
return a+b
bar = staticmethod(bar)

# for Python 2.4 and up
class Foo:
@staticmethod
def bar(a,b):
return a+b

Foo().bar(1,2) works exactly the same as before, but your definition of
bar doesn't need that extraneous "self" parameter.

 

[n.estner]

I still see "newbie-friendliness" as a

MAJOR plus for Python -- it increases the chance that users
of your software will become contributors.

Yes, I 100% agree to that point!
But the point is, the current situation is not newbie-friendly (I can
tell, I am a newbie): I declare a method with 3 parameters but when I
call it I only pass 2 parameters. That's confusing. If I declare a
member variable, I write: "self.x = ValueForX", why can't I be equally
explicit for declaring member functions?

 

[n.estner]

You could try running it to see:

... def test(a, **kwargs): return 1
...

... def test(b, **kwargs): return 2
...

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'test' is not defined

Oops! You have defined a name "test" in two namespaces, the class A and
the class B, but there is no name "test" in the global namespace you are
trying to run it in.

Since namespaces are critical to Python, your test code is a problem that
just cannot happen in Python. It is a non-issue. Python will not get
confused between the two definitions of test.

I've been answering to Donn Cave's suggestion. Read it, and you will
understand what I mean.

As near as I can tell, your original complaint might be solved simply: it
seems to me that you are concerned about that extraneous "self" parameter
for methods that don't need it:

No, I wasn't talking about functions that don't use the "self"
parameter. I rarely ever have such functions. I just tried to make the
example as short as possible.

 

[Antoon Pardon]

Agreed. I tried to post a short example, and it obviously was to short
to make my point clear. lets take a longer one. Current syntax:

class Pair:
def __init__(self, a,b):
self.a = a
self.b = b

def sum(self):
return self.a + self.b

def product (this):
return this.a + this.b

My alternative syntax suggestion would be this one:

class Pair:
def self.__init__(a,b):
self.a = a
self.b = b

def self.sum():
return self.a + self.b

def this.product ():
return this.a + this.b


No. I hope this is clearer in the example above. "self" shouldn't be a
keyword. It's a special kind of argument now, so why shouldn't we
explicitly _declare_ that it's a special kind of argument? (as explicit
is better than implicit)

Self is not a special kind of argument. It is the accessing of the
method that provides for the magic. Simplified one could say the
following is happening.

def _Pair__init__(self, a, b):
self.a = a
self.b = b

def _Pair_sum(self):
return self.a + self.b

def _Pair_product(this):
return this.a * this.be

class Pair:

def __init__(self, ...):
self.__init__ = BoundMethod(self, _Pair__init__)
self.sum = BoundMethod(self, _Pair_sum)
self.product = BoundMethod(self, _Pair_product)
self.__init__(...)


So when p is an instance of Pair, p.sum is not your defined
function but a BoundMethod.

 

[Steven D'Aprano]

On Fri, 03 Feb 2006 04:14:33 -0800, n.estner wrote:

[snip]

I've been answering to Donn Cave's suggestion. Read it, and you will
understand what I mean.

Your reply came through as I was writing my reply, and it does help
explain your position better.

No, I wasn't talking about functions that don't use the "self"
parameter. I rarely ever have such functions. I just tried to make the
example as short as possible.

I understand that now

 

[Steven D'Aprano]

My alternative syntax suggestion would be this one:

class Pair:
def self.__init__(a,b):
self.a = a
self.b = b

def self.sum():
return self.a + self.b

def this.product ():
return this.a + this.b

This would be a fairly major change to Python, so I think we can say right
up front that the chances of this happening before Python 3 are zero, and
even in Python 3 the chances are about the same as the chances of Richard
Stallman suddenly announcing that he's taken a job for Microsoft writing
Digital Restrictions Management software.

But still, let's continue. After all, all you have to do is convince Guido
that this syntax is better...

No. I hope this is clearer in the example above. "self" shouldn't be a
keyword.

Which it isn't now either.

It's a special kind of argument now, so why shouldn't we
explicitly _declare_ that it's a special kind of argument? (as explicit
is better than implicit)

And a foolish consistency is the hobgoblin of little minds *wink*

[snip]

def x(self):
declares two new identifiers, "x" and "self"

Why shouldn't
def self.x():
declare two new identifiers ("x" and "self"), too?

Sure, but now the call foo.bar has special meaning inside a def statement
than elsewhere. Elsewhere, foo.bar is an attribute access, looking up
attribute bar in foo's namespace. Using your syntax, in a def statement
foo.bar is a pair of declarations: it declares a name "foo", and it
declares a second name "bar".

This inconsistency is, I think, worse than the implicit use of self.