Does Python really follow its philosophy of "Readability counts"?

[Luis Zarrabeitia]

Python programmers tend to not have a need for
properties. Quite honestly they are a waste of time.
They come from traditional OO approaches to software design
(and mostly from the Java world).

With statements like that, it's no wonder you don't understand the
value of encapsulation.
[snip]

If you didn't plan ahead and encapsulate the radius from the start,
properties allow you to save yourself and encapsulate it later without
breaking the client's code.

Python programmers don't _need_ to plan ahead and encapsulate the radius
from
the start. That's the whole point. No clairvoyance needed. I kind of like
that.

Oops. I didn't noticed we were agreeing on this last point.
Bad english... bad...

[btw, I highly doubt James doesn't understand the value of encapsulation. Don't
you mean "enforced data hiding" again?]

Cya!

 

[Russ P.]

Quoting "Russ P." <[email protected]>:

On Jan 19, 6:24 pm, "James Mills" <[email protected]>
wrote:

With statements like that, it's no wonder you don't understand the
value of encapsulation.
[snip]

If you didn't plan ahead and encapsulate the radius from the start,
properties allow you to save yourself and encapsulate it later without
breaking the client's code.

Python programmers don't _need_ to plan ahead and encapsulate the radius from
the start. That's the whole point. No clairvoyance needed. I kind of like that.

And I like it too. I was trying to explain that to Mr. Mills, who
claimed that properties are "a waste of time."

Properties reduce the need for encapsulation, but they do not
eliminate it, particularly for large projects.

In the circle example, properties are nice for guaranteeing
consistency between the radius and the area, but they are not of much
use if you decide later that the client should not be allowed to
change either one. Well... I suppose you could define the setter to do
nothing ... but you had better figure a way to notify the client about
it.

 

[Luis Zarrabeitia]

[removing david's message]

Why did you email your post to me? Did you really think I wanted to
see it in my inbox? I assure you I did not.

Don't get mad at him... and don't take it personal. The reply-to header of this
mailing list (usenet users: for some of us, this is only a mailing list, and
can't hope for more) points to the sender, and not the list. Many believe it is
the right thing to do, but if your mailer doesn't have a 'reply to list'
function, like the one I'm using right now (Horde/Imp), your only option is to
reply-to-all and then delete the original sender. I can't blame anyone for
forgetting about the last part.

 

[Luis Zarrabeitia]

You don't understand the purpose of properties -- and you tell me that
*I* am the one missing the point?

This line would make a lot more sense if you were talking about Java's getters
and setters, or about a language where accessing a property is different than
accessing an attribute (which would make little sense). If properties already
let you change from attribute to method without affecting the caller, why do you
need a property that does nothing?

 

[Luis Zarrabeitia]

In the circle example, properties are nice for guaranteeing
consistency between the radius and the area, but they are not of much
use if you decide later that the client should not be allowed to
change either one. Well... I suppose you could define the setter to do
nothing ... but you had better figure a way to notify the client about
it.

Well, you could make it raise an exception. There was one C#1.0 class that did
this, Liskov substitution principle be damned (it was for a subtype). Very
frustrating.

Once you decide for a public interface, you shouldn't change it... But, if you
absolutely have to do it, complain loudly and as soon as possible. In python,
I'd say that [a subclass of?] AttributeError would be the best solution for this
ill-advised situation.

 

[James Mills]

This line would make a lot more sense if you were talking about Java's getters
and setters, or about a language where accessing a property is different than
accessing an attribute (which would make little sense). If properties already
let you change from attribute to method without affecting the caller, why do you
need a property that does nothing?

My point exactly. Russ seems to agree with Python
but yet argue points for the sake of it. I'm not sure why

Having come from all kinda of programming backgrounds
and paradigms you learn to see the value in Python and the
kind of simplicity it has to offer. I will stand by my view
that there are many features of the traditional, strict
and academic features of the OO model that have little
practical value.

Python is a great mix of functional features, OO features
and has borrowed (what I believe) are the best of breed
features from all around.

One thing I find quite amazing is that we're having a discussion
over such low-level features of the OO model (and functional paradigm)
and how Python fits into it all ... And for what exactly ?

Russ - what is your point in all of this - You keep saying you don't have
time to waste with this - yet you keep making this thread grow
longer and longer and longer

As far as I'm concerned properties are just fancy functions to
retrieve and set attributes of an object. Consider the following
equivalent pieces of code:

#!/usr/bin/env python

from math import pi

class CircleA(object):

def __init__(self, radius):
self._radius = radius
self._area = pi * radius ** 2

def __getRadius(self):
return self._radius

def __setRadius(self, radius):
self._radius = radius
self._area = pi * radius ** 2

def __getArea(self):
return self._area

def __repr__(self):
return "<Circle r=%0.2f A=%0.2f>" % (self.radius, self.area)

radius = property(__getRadius, __setRadius)
area = property(__getArea)

class CircleB(object):

def __init__(self, radius):
self.radius = radius
self.area = pi * radius ** 2

def setRadius(self, radius):
self.radius = radius
self.area = pi * radius ** 2

def __repr__(self):
return "<Circle r=%0.2f A=%0.2f>" % (self.radius, self.area)

a = CircleA(1.5)
print a
a.radius = 2.0
print a

b = CircleB(1.5)
print b
b.radius = 2.0
print b

----------

http://codepad.org/tpyGNhrZ

I'll give you a hint which one I prefer

cheers
James

 

[Paul Rubin]

import inspect
class ImmutableType(type): ...

Thanks for posting this, I haven't replied because I bookmarked the
post for later study, but that was several days ago, so I just wanted
to say that I'm still looking at it. It's an area of Python that I've
somewhat purposely stayed away from.

But I see no _necessary_ harm on tacking extra attributes to an
existing object - specially if you are going to use them pretty
close to the creation.

I use them, a lot, specially when writing decorators... Many times I
just want to 'mark' the decorated functions so I can inspect those
marks later. I'd rather have a semi-private namespace for each pair
([group of]calling function, object),

Right, thus the usefulness of a static tool that can find all those
places where extra attributes are created. Basically I can see three
approaches to this question:

1) Program in a style where you create and modify attributes all over
the place willy-nilly, with no controls whatsoever, either automatic
or stylistic. From experience (it is a common style in Javascript) I
can say this is a big mess and I think no experienced Python
programmers recommend it.

2) Have mandatory encapsulation like Java's, no way to escape. Bruno
and yourself vehemently object to this, Russ P seems to want it, my
view is that it's not in the Python spirit, so I can accept the
argument that those who really want this are better off choosing
another language.

3) Program in a style where creating new attributes outside the
initializer is normally avoided by convention, but it is possible and
sometimes desirable to break the convention. This seems reasonable
and Pythonic to me. But, any place I break the convention, I should
be able to say "I broke the convention here for such and such a
reason". And (this seems to be a bone of contention) IMO, it would be
good to be able to audit a large codebase with a tool like Pylint, to
automatically find all the places where such breakages occur. Right
now there seems to be no way to do that reliably, short of using
something like your special metaclass instead of the standard one for
just about everything.

Your example about decorators (used on classes that don't expect to be
wrapped by them) is a good one but narrow enough that there could be a
special arrangement for it in a hypothetical auditing tool.

 

[Russ P.]

If properties already
let you change from attribute to method without affecting the caller, why do you
need a property that does nothing?

You don't. As I tried to explain in a earlier post (but one that you
may not have seen yet), that is apparently an implementation detail of
the way Scala implements properties. I probably should have never even
brought it up.

 

[Russ P.]

[removing david's message]

Why did you email your post to me? Did you really think I wanted to
see it in my inbox? I assure you I did not.

Don't get mad at him... and don't take it personal. The reply-to header of this
mailing list (usenet users: for some of us, this is only a mailing list, and
can't hope for more) points to the sender, and not the list. Many believe it is
the right thing to do, but if your mailer doesn't have a 'reply to list'
function, like the one I'm using right now (Horde/Imp), your only option is to
reply-to-all and then delete the original sender. I can't blame anyone for
forgetting about the last part.

You're right. I shouldn't have gotten mad at him. I took his post as a
jab at me, but after thinking about it I realized it probably wasn't.
My apologies.

 

[Paul Rubin]

The failure was because a module tested, QA'd and certified within a
given context (in which it was ok to drop the builtin error handling)
was reused in a context where it was not ok. And the point is exactly
that : no *technology* can solve this kind of problem, because it is a
*human* problem (in that case, not taking time to repass the whole
specs / tests / QA process given context change).

In this case it does nothing at all to support your arguments about
the helpfulness or lack of helpfulness of strong encapsulation. You
may as well say that antibiotics are medically useless because they
won't stop anyone from getting killed by a falling piano.

 

[Paul Rubin]

Take some not-that-trivial projects like Zope/Plone. There are quite a
few lines of code involved, and quite a lot of programmers worked on it.

Zope is about 375 KLOC[1], which I agree is not trivial, but by
today's standards, it's not all that large. Zope also has 275 open
bugs, 6 of which are critical.[2] The Space Shuttle avionics (written
in the 1980's!) are 2 MLOC in which only 3 errors have been found
post-release.[3] I think "large software system" today means 100's of
MLOC. FWIW, Zope has 20x as much code as Django--is that a good
thing!?

[1] http://www.peterbe.com/plog/size-Zope3,Django,TurboGears
[2] https://bugs.launchpad.net/zope3
[3] http://haacked.com/archive/2006/10/20/The_Misuse_of_the_Space_Shuttle_Analogy.aspx#11147

 

[Russ P.]

In this case it does nothing at all to support your arguments about
the helpfulness or lack of helpfulness of strong encapsulation.  You
may as well say that antibiotics are medically useless because they
won't stop anyone from getting killed by a falling piano.

He says that "no *technology* can solve this kind of problem."

First of all, I'm not sure that's true. I think technology *could*
have solved the problem -- e.g., Spark Ada, had it been properly
applied. But that's beside the point. The point is that the problem
had nothing to do with encapsulation. The rocket failed because a
conversion was attempted to a data type that could not hold the
required value. Am I missing something? I don't see what that has to
do with encapsulation.

The logic seems to be as follows:

1. Ada enforces data hiding.
2. Ada was used.
2. A major failure occurred.

Therefore:

Enforced data hiding is useless.

If that reasoning is sound, think about what else is useless.

 

[Luis Zarrabeitia]

Thanks for posting this, I haven't replied because I bookmarked the
post for later study, but that was several days ago, so I just wanted
to say that I'm still looking at it. It's an area of Python that I've
somewhat purposely stayed away from.

Hehe. Keep away - once you get dive into it, it'll be hard go get out. It's
really addictive

I use them, a lot, specially when writing decorators... Many times I
just want to 'mark' the decorated functions so I can inspect those
marks later. I'd rather have a semi-private namespace for each pair
([group of]calling function, object),

Right, thus the usefulness of a static tool that can find all those
places where extra attributes are created. Basically I can see three
approaches to this question:

Btw, this thread inspired me to do another proof-of-concept stuff that I may be
using more seriously. Too bad I don't have anywhere to upload it right now...
Anyway, it looks like this: suppose I need to attach some attributes to the
object "x", but I don't want to do x.attr

import namespaces # my module
class my_private_namespace(namespaces.Namespace): pass

def this_function_stores_the_attr(x):
ns = my_private_namespace(x)
ns.attr = 5 # I'm creating an attribute on the fly
# but at least I'm not touching x.

def this_function_reads_the_attr(x):
ns = my_private_namespace(x)
print ns.attr

1) Program in a style where you create and modify attributes all over
the place willy-nilly, with no controls whatsoever, either automatic
or stylistic. From experience (it is a common style in Javascript) I
can say this is a big mess and I think no experienced Python
programmers recommend it.

Indeed. Though I don't consider myself 'experienced', I certainly don't
recommend it.

2) Have mandatory encapsulation like Java's, no way to escape. Bruno
and yourself vehemently object to this, Russ P seems to want it, my
view is that it's not in the Python spirit, so I can accept the
argument that those who really want this are better off choosing
another language.

Btw, I noticed a funny side effect of my 'namespaces' module (needs a LOT of
work, this was the first time I touched weak references. I should find some
place to upload the module):

=====
import namespaces

class public(namespaces.Namespace): pass
class private(namespaces.Namespace): # the initializer could check if
pass # its called from outside the class. Performance penalty.

class A(object):
"""dummy class A, with one 'private' attribute x, and one
normal 'attribute' x."""
def __init__(self):
self.x = id(self) # normal attr
pub = public(self)
pub.f = self.f
priv = private(self)
priv.x = "this is the private"
def f(self):
print "normal x: ", self.x
priv = private(self)
print "private x: ", priv.x

normal x: 147956684
private x: this is the private
normal x: 147956684
private x: this is the private
======

Of course, without checks in 'private', one could say priv = private(a); priv.x,
but the most interesting part, for me at least, was the ability to define a
'public' interface for a, that contains no attribute directly referencing the
object 'a' with its internals - only the names explicitly declared on the
'interface'. Indirectly... well, p.f.im_self will return back the 'a' object.

Bah, I'm ranting now. I just got carried away with my little experiment. Sorry.
Ah, and yes, metaclasses are dangerously addictive

3) Program in a style where creating new attributes outside the
initializer is normally avoided by convention, but it is possible and
sometimes desirable to break the convention. This seems reasonable
and Pythonic to me. But, any place I break the convention, I should
be able to say "I broke the convention here for such and such a
reason". And (this seems to be a bone of contention) IMO, it would be
good to be able to audit a large codebase with a tool like Pylint, to
automatically find all the places where such breakages occur. Right
now there seems to be no way to do that reliably, short of using
something like your special metaclass instead of the standard one for
just about everything.

Well, I agree. Each and every one of those convention-breakages should be well
documented, and pylint should be able to pick them up. Pylint does that already
(see example below), I don't know if it lets you "document" it explicitly. I
think Eclipse catches some of that stuff. My metaclass... shouldn't be used for
anything serious .

=== breaklint.py === (no, it didn't break pylint)
class A(object):
def __init__(self):
self.a = 5
def f(self):
self.b = 7 # creating outside the init
def g(self):
print self.b # only if f runs first
print self.c # and this one doesn't exist

a = A()
a.c = 8
a.f()
a.g()
a.d = "new var" # outside the class definition
==================
$ pylint breaklint.py
[...]
W: 13: Attribute 'c' defined outside __init__
W: 7:A.f: Attribute 'b' defined outside __init__
W: 16: Attribute 'd' defined outside __init__
[...]
Your code has been rated at -1.54/10

=====

Your example about decorators (used on classes that don't expect to be
wrapped by them) is a good one but narrow enough that there could be a
special arrangement for it in a hypothetical auditing tool.

Indeed. I'll take a look at pylint's config file.

 

[Luis Zarrabeitia]

He says that "no *technology* can solve this kind of problem."

First of all, I'm not sure that's true. I think technology *could*
have solved the problem -- e.g., Spark Ada, had it been properly
applied. But that's beside the point. The point is that the problem
had nothing to do with encapsulation. The rocket failed because a
conversion was attempted to a data type that could not hold the
required value. Am I missing something? I don't see what that has to
do with encapsulation.

The logic seems to be as follows:

1. Ada enforces data hiding.
2. Ada was used.
2. A major failure occurred.

Therefore:

Enforced data hiding is useless.

I don't think that was the logic... At least, it wasn't what I understood from
the example. We were talking at the time (or rather, you both were, as I think I
was still away from the thread) about QA versus static checks (as a superset of
'enforcement of data hiding').

Is not that those checks are useless, is that they make you believe you are
safe. Granted, you may be safe-er, but it may be better to know beforehand "this
is unsafe, I will check every assumption". The arianne 5 example cuts both ways:
messing with "internals" (not that internal in this case, but still) without
fully understanding the implications, and overly trusting the checks put in
place by the language. I don't know spark-ada, but the only thing I can think of
that would have prevented it is not purely technological. It would have been a
better documentation, where that particular assumption was written down. A
language can help you with it, perhaps spark-ada or eiffel or who-knows-what
could've forced the original developers to write that piece of documentation and
make the arianne 5 engineers notice it.

So, Arianne 5's problem had nothing to do with _enforced data hiding_. (Why do
you keep calling it 'encapsulation'?). It was a counterexample for your trust on
compiler checks.

Btw, what do you have against using pylint for detecting those 'access violations'?

If that reasoning is sound, think about what else is useless.

I _hope_ that wasn't Bruno's reasoning. I don't want to switch sides on this
discussion .

Cheers,

 

[Russ P.]

So, Arianne 5's problem had nothing to do with _enforced data hiding_. (Why do
you keep calling it 'encapsulation'?).

I keep calling it encapsulation because that is a widely accepted,
albeit not universal, definition of encapsulation. Google
"encapsulation" and see for yourself. For example, here is what the
Wikipedia page on OOP says:

Encapsulation conceals the functional details of a class from objects
that send messages to it.

Here is what another webpage says:

Definition: In Object Oriented Programming, encapsulation is an
attribute of object design. It means that all of the object's data is
contained and hidden in the object and access to it restricted to
members of that class.

Because those definitions are not universally accepted, however, I
usually say "enforced encapsulation" or "data hiding" to mean what
"encapsulation" means to most OO programmers who use a language other
than Python.

 

[Paul Rubin]

Unless it's changed since I used it, technically, SPADE doesn't allow
or disallow anything.

Right, it's an external tool, like pylint; you can still compile code
that SPADE complains about. Sorry if I wan't more clear. I just
meant SPADE would flag the code.

Since the value appears to come from a sensor, the only way one
could prove that there would be no overflow would be to state it as
a part of the specification of what is read in. If that
specification doesn't match the specification of the actual sensor,
that's nothing to do with the programming language or, for that
matter, the program itself. It's a specification mismatch.

Right, that's the point, the assumption about the sensor reading being
in a certain range would have to stated in the specification rather
than implicit in the code; and as such, the problem would more likely
to have been caught at the systems engineering level.

I was actually at the European Space Agency's Toulouse site the week
after the Ariane 5 incident. I've been at jollier funerals. I can't
help thinking that thinking that the team would have benefited from
reading David Parnas's work on the specification of the A-7E avionics.

Later there was a static analysis of the Ariane code by Cousot et al,
that flagged the error. I haven't yet looked up the paper to see
exactly what it found.

 

[Paul Rubin]

I was actually at the European Space Agency's Toulouse site the week
after the Ariane 5 incident. I've been at jollier funerals. I can't
help thinking that thinking that the team would have benefited from
reading David Parnas's work on the specification of the A-7E avionics.

Thanks, these articles look interesting, though there is a lot to read:

http://www.chacs.itd.nrl.navy.mil/publications/scr-a7e/index.html

Was there something else you were referring to?

 

[Steven D'Aprano]

Having come from all kinda of programming backgrounds and paradigms you
learn to see the value in Python and the kind of simplicity it has to
offer.

Oh yes, it is liberating to say "I don't care if my method crashes
(raises an exception), it's the caller's fault for messing with my class'
internals, and he can deal with it". I'm not being sarcastic by the way.
It really is liberating not to have to deal with unexpected input or
broken pre-conditions. Just let the caller deal with it!

 

[Paul Rubin]

No wonder you can't get Bruno's point. For the second, static checks
to prevent accidents, you have pylint. For the first, not only you
are using the wrong tool, but you are barking at python for not
having it. Assuming that pylint is perfect (big assumption, but it
is up to you to prove where it fails),

Whaat? Assuming a program is perfect unless a failure is proven
is not at all a sane approach to getting reliable software. It is
the person claiming perfection who has to prove the absence of failure.

 

[Luis Zarrabeitia]

(Why do you keep calling it 'encapsulation'?).

I keep calling it encapsulation because that is a widely accepted,
albeit not universal, definition of encapsulation.
[...]

Encapsulation conceals the functional details of a class from objects
that send messages to it.
[..]

Definition: In Object Oriented Programming, encapsulation is an
attribute of object design. It means that all of the object's data is
contained and hidden in the object and access to it restricted to
members of that class.

Ahh, 'concealed', 'contained', 'hidden'. Except the last one, "hidden", python
does the rest... and one could argue the self.__privs get pretty well hidden.

Not 'forbidden', 'restricted', 'enforced'.