Encapsulation unpythonic?

[fsaldan1]

I am new to Python, with experience in Java, C++ and R.

As I understand encapsulation is not a big thing in the Python world. I read that you can put two underscores before the name of a variable within a class declaration but in the many examples of code I looked at this is not widely used. I also read that encapsulation is "unpythonic."

Questions:

1) Is there a good text where I can read about the language philosophy? What practices are "pythonic" or "unpythonic"?

2) If it is in fact true that encapsulation is rarely used, how do I deal with the fact that other programmers can easily alter the values of members of my classes?

Thanks for any insights.

FS

 

[Skip Montanaro]

In Guido's own words: "We're all consenting adults here."

http://importthis.tumblr.com/post/6719643315/public-private-attributes

Skip

 

[David]

1) Is there a good text where I can read about the language philosophy? What practices are "pythonic" or "unpythonic"?

2) If it is in fact true that encapsulation is rarely used, how do I deal with the fact that other programmers can easily alter the values of members of my classes?

Try this:
http://dirtsimple.org/2004/12/python-is-not-java.html

 

[Chris Angelico]

2) If it is in fact true that encapsulation is rarely used, how do I deal with the fact that other programmers can easily alter the values of members of my classes?

Very simply: You accept it. Let them! It's their responsibility.

Python scripts are generally assumed to start at the beginning, go on
till they reach the end, then stop (like the White King's advice to
Alice). The author of the application is assumed to be in command of
everything. If s/he chooses to monkey-patch something, so be it. If
that monkey-patch breaks in the next version, it's the app author's
problem. As a module or class author, you just need to make sure you
don't make crazy changes to the environment, and all will be well.

If you have invariants that you want to maintain, you can simply
document the one official way to mutate your objects ("use the .foo()
method, don't tinker with the members"), and most people will respect
that. But most of the time, that's not even an issue - all you have to
do is tell yourself "It's fine for them to change stuff", and (a) you
save the hassle of preventing them, and (b) you save the hassle of
writing tons of boilerplate to grant specific access.

class Point
def __init__(self,x,y):
self.x,self.y=x,y
def distance(self,other):
return math.sqrt((self.x-other.x)**2+(self.y-other.y)**2)

foo = Point(0,0)
while True:
foo.x+=deltax; foo.y+=deltay
if foo.distance(bar)>50: break

Easy! No getter/setter needed.

ChrisA

 

[Steven D'Aprano]

I am new to Python, with experience in Java, C++ and R.

As I understand encapsulation is not a big thing in the Python world.

Utter nonsense. Whoever told you this doesn't understand what
encapsulation is.

Python encapsulates related code into objects. It encapsulates related
objects into modules. It encapsulates related modules into packages.

I
read that you can put two underscores before the name of a variable
within a class declaration but in the many examples of code I looked at
this is not widely used. I also read that encapsulation is "unpythonic."

That's *data hiding*, not encapsulation.

Very few languages -- possibly none at all -- can hide data from a
sufficiently motivated programmer. Python doesn't even try. "We're all
adults here" is the philosophy, and data hiding is by convention, not
enforced by the compiler.

Single leading underscores are "private". Don't touch them. If you do,
and code breaks, nobody will give you sympathy. You have nobody to blame
but yourself.

Double leading underscores are "private", and also have name-mangling to
try to avoid certain inheritance-related issues. In general, it's more of
a nuisance than anything else, so most people don't bother. Consider
double underscore __names to be for advanced OOP usage, 98% of the time a
single underscore is enough.

Double leading and trailing __names__ are reserved for Python. They're
not necessarily private, but if you're calling them directly, you're
probably doing something wrong. Again, consider them to be advanced usage.

Questions:

1) Is there a good text where I can read about the language philosophy?
What practices are "pythonic" or "unpythonic"?

Good question!

Start at the interactive interpreter:

import this


This is a reasonable description of what it means to be Pythonic:

http://blog.startifact.com/posts/older/what-is-pythonic.html


This is a good pair of resources, comparing the Java and Python
philosophies, and the strengths of each:

http://dirtsimple.org/2004/12/python-is-not-java.html

http://dirtsimple.org/2004/12/java-is-not-python-either.html

Also, it helps to understand that Python is named after Monty Python, not
the snake. It's not necessary to like anarchic British humour, but it
helps to get some of the references. We'll talk about "spam, ham, eggs"
rather than "foo, bar, baz", and the Cheeseshop, and the Spanish
Inquisition, and Norwegian Blue parrots.

But ultimately, writing Pythonic code doesn't come from reading a list of
rules. It comes from becoming comfortable with the language, from
understanding its strengths and weaknesses, from reading lots of people's
code, and writing lots of code, and learning the idioms.

2) If it is in fact true that encapsulation is rarely used,

Not true. It is true that data hiding is really used though, at least in
pure-Python code, except by convention.

(C extensions are much more strict about data hiding, since you can crash
the compiler if you muck about with C-level internals. Exceptions are a
good thing. Segfaults are not.)

how do I
deal with the fact that other programmers can easily alter the values of
members of my classes?

Embrace it! That's a good thing!

In Java or C++ or other languages, other programmers are going to alter
your classes' members anyway. The only difference is that they will spend
hours or days fighting the compiler in order to do so, and eventually end
up with horrible, fragile, non-portable code.

Besides, while it's nearly always a Bad Thing to mess with private
attributes, sometimes it is a really, really Useful Thing to *inspect*
private attributes, for debugging. Python makes that easy.

Treat other programmers as adults, and they in turn will treat you the
same way. If they insist on messing with your private single-underscore
_attributes, you can't stop them, but that's okay, you don't have to be
sympathetic when they shoot their foot off. Just slap them with a large
halibut[1] and laugh.





[1] Another Monty Python reference.

 

[Gary Herron]

I am new to Python, with experience in Java, C++ and R.

As I understand encapsulation is not a big thing in the Python world. I read that you can put two underscores before the name of a variable within a class declaration but in the many examples of code I looked at this is not widely used. I also read that encapsulation is "unpythonic."

Questions:

1) Is there a good text where I can read about the language philosophy? What practices are "pythonic" or "unpythonic"?

2) If it is in fact true that encapsulation is rarely used, how do I deal with the fact that other programmers can easily alter the values of members of my classes?

Thanks for any insights.

FS

You are confusing encapsulation with data hiding!

Encapsulation is very much a part of Python. Every class, module,
indeed every object, encapsulates some kind of behavior.

However, *hiding* the members of a class is not considered Pythonic.
There is no private/public as in C++, however, there are way to achieve
that effect.

Gary Herron

 

[Terry Reedy]

Very simply: You accept it. Let them! It's their responsibility.

When a project has multiple programmers, there is a possibility that
module C could monkeypatch module A in a way that breaks existing user
module B. But it is still the collective responsibility of the
respective users or project manager to assign responsibility for fixing
the conflict.

 

[Chris Angelico]

When a project has multiple programmers, there is a possibility that module
C could monkeypatch module A in a way that breaks existing user module B.
But it is still the collective responsibility of the respective users or
project manager to assign responsibility for fixing the conflict.

Yep. I would say there that the responsibility is with module C's
programmers; they are the ones breaching encapsulation, ergo it's
primarily their responsibility to both test this (against current
usage) and thoroughly document it (against future changes).
Personally, I would like to see a comment in module A that says
something like "NOTE: This {function|attribute|spam} is replaced
externally under [some circumstance]. Changes to its purpose may
affect Module B." to make it clear what's going on. At that point, of
course, it stops being monkeypatching and becomes documented and
official behaviour, so maybe that's not really an argument in this
example.

ChrisA

 

[Joshua Landau]

how do I
deal with the fact that other programmers can easily alter the values of
members of my classes?

...
If they insist on messing with your private single-underscore
_attributes, you can't stop them, but that's okay, you don't have to be
sympathetic when they shoot their foot off. Just slap them with a large
halibut[1] and laugh.

I know I've cropped your points but I just want to mention here that
the only reason to monkey-patch code in these ways where you'd want to
stop them is when the alternative is *worse*. It's like removing railings from
a cliff to stop people hitting the bars.

 

[Steven D'Aprano]

how do I
deal with the fact that other programmers can easily alter the values
of members of my classes?

...
If they insist on messing with your private single-underscore
_attributes, you can't stop them, but that's okay, you don't have to be
sympathetic when they shoot their foot off. Just slap them with a large
halibut[1] and laugh.

I know I've cropped your points but I just want to mention here that the
only reason to monkey-patch code in these ways where you'd want to stop
them is when the alternative is *worse*. It's like removing railings
from a cliff to stop people hitting the bars.

I'm not actually talking about monkey-patching. I'm talking about just
normal inheritance of classes. E.g. If a module has this class:


class Parrot:
def __init__(self):
self._name = "Polly"
def talk(self):
print "%s wants a cracker!" % self._name


I might be tempted to do this:

class MyParrot:
def __init__(self):
super(MyParrot, self).__init__()
self._name = "George"


No monkey-patching involved. But, if the author of Parrot class changes
his implementation and gets rid of "_name", or even makes it public
"name", my subclass will stop working. Sucks to be me.

In this toy example, both parties are at fault: the author of Parrot for
unnecessary data-hiding of something which is so obviously a useful piece
of information and should be part of the public interface, and me for
nevertheless ignoring that warning and using the private attribute in my
own code. More realistic examples may be different.

 

[Dennis Lee Bieber]

I'm not actually talking about monkey-patching. I'm talking about just
normal inheritance of classes. E.g. If a module has this class:


class Parrot:
def __init__(self):
self._name = "Polly"
def talk(self):
print "%s wants a cracker!" % self._name


I might be tempted to do this:

class MyParrot:
def __init__(self):
super(MyParrot, self).__init__()
self._name = "George"

Which will do nothing... you forgot to subclass...

class MyParrot(Parrot):

 

[random832]

In this toy example, both parties are at fault: the author of Parrot for
unnecessary data-hiding of something which is so obviously a useful piece
of information and should be part of the public interface,

It may wish to be notified when its name changes, and so have a name
property. The subclass may want to use a variable called _name for some
other purpose. (maybe "name" isn't the best example).

Examples often look pathological when you simplify out the bit that
makes them make sense.

 

[Steven D'Aprano]

It may wish to be notified when its name changes, and so have a name
property. The subclass may want to use a variable called _name for some
other purpose. (maybe "name" isn't the best example).

Such a "name" property would be a public interface, and so a Good Thing.
However, my toy example was of a case where something *obviously useful*
was being left out of the public interface. If it were a public property,
it wouldn't be the case that it were left out, would it?

Examples often look pathological when you simplify out the bit that
makes them make sense.

Naturally

I did call this a toy example. Nevertheless, in the Real World, data
hiding can sometimes be a bit of a religion. Not all cases where people
try various tricks and hacks to gain access to "private" and "protected"
members are specious.

The whole Java getter and setter philosophy is based on the idea that
everything, even the most innocuous data attribute, ought to be private,
with a computed getter and setter Just In Case some day in the future you
want to wrap access in code. In Python, you can turn an attribute into a
computed property with no change to the public interface. In Java, you
can't.

 

[chaz2cry]

I am new to Python, with experience in Java, C++ and R.



As I understand encapsulation is not a big thing in the Python world. I read that you can put two underscores before the name of a variable within aclass declaration but in the many examples of code I looked at this is notwidely used. I also read that encapsulation is "unpythonic."



Questions:



1) Is there a good text where I can read about the language philosophy? What practices are "pythonic" or "unpythonic"?



2) If it is in fact true that encapsulation is rarely used, how do I dealwith the fact that other programmers can easily alter the values of members of my classes?



Thanks for any insights.



FS

Hi FS,

I'm taking the Python Cert series w/ O'Reilly School of Technology, which Irecommend if you've got a good handle on OO programming. In any event, according to what I've learned, "encapsulation is the idea that the only way to access or change the data inside an object is by calling its methods. This idea has never really gained much ground in the Python world, and it is normally considered acceptable to both read and set an object's attributes from anywhere in a program." Not being an expert OO programmer, I take this at face value.

There are ways to protect class attributes from having their values reset from outside. That is, they can be made "internal use only" and an AttributeError raised when someone tries to change the attribute(s). This involves __setattr__ and checking if the key of the attribute is in a/the list of attributes you've chose to protect. If so, raise AttributeError. Hope thathelps in some small measure.

In the interest of full disclosure, answering a Python question is part of my homework for the O'Reilly Python 4 class I'm taking. Good luck!

 

[Fabrice Pombet]

I am new to Python, with experience in Java, C++ and R.



As I understand encapsulation is not a big thing in the Python world. I read that you can put two underscores before the name of a variable within aclass declaration but in the many examples of code I looked at this is notwidely used. I also read that encapsulation is "unpythonic."



Questions:


2) If it is in fact true that encapsulation is rarely used, how do I dealwith the fact that other programmers can easily alter the values of members of my classes?

Fernando, it is widely accepted that Python pays very little attention to encapsulation as a principle set in stone. Chaz's definition of encapsulation is also mine. Now you need to consider that taking this principle off thehostel of OOP does not mean that you can do whatever you fancy and you can't make anything unsettable.

There are plenty of techniques within Python that allow you to protect yourarguments (in particular, decorators) inside a Class.

Now, lets get to the pretentious philosophical discussion: I guess encapsulation is quite the opposite of, say, dynamic typing, which is arguably corein Python. In practice this allows Python to be less verbose: at the end of the day, if you look back at your previous languages, don't you find thatsome of their compulsory features are usually more of a pain than something useful in practice? And after all, whither encapsulation? Can't we just have objects whose arguments are determined externally if we want to?
And that is the ballgame: as my old tutor says: "the claptrap of setters and getters does not need to be here if it is unnecessary". I would add: "so long as you can have them when you deem it necessary", and Python allows that.

 

[Steven D'Aprano]

Fernando, it is widely accepted that Python pays very little attention
to encapsulation as a principle set in stone.

Widely accepted by whom?

Python code is *full* of encapsulation. Functions, methods, classes,
modules, packages, even local variables, are all mechanisms for
encapsulating code and data. Those who say that Python has little or no
encapsulation are talking rubbish.

Chaz's definition of
encapsulation is also mine.

Who is Chaz, and what definition does he have?

Now you need to consider that taking this
principle off the hostel of OOP does not mean that you can do whatever
you fancy and you can't make anything unsettable.

There are plenty of techniques within Python that allow you to protect
your arguments (in particular, decorators) inside a Class.

And now you are talking about information hiding and protection, which is
not the same of encapsulation, no matter what the academics think.
Sometimes the air gets a bit too thin to breathe way up at the top of
those ivory towers...

Encapsulation is about grouping code that needs to be together together.
In contract, you have programming languages that give you little, or
nothing, in the way of grouping -- everything is one big chunk of code,
with GOTO or GOSUB to jump from place to place.

Functions and procedures are the first, most simple, form of
encapsulation. Classes allow you to encapsulate multiple functions
("methods") together with the data they need to operate on in one chunk.
Even in C++ or Java, you can have classes that provide no information
hiding at all -- just declare everything "public".

On the other hand, non-OOP languages like C can implement information
hiding. In C, you can hide information from other files by declaring them
as "static". Variables declared inside a brace-delimited block only exist
within that block: local variables are hidden. For example:

int foo;
static int bar;


bar is hidden from other files. Likewise, in this function:


int func(void) {
int baz;
...
}


baz is local to func, and invisible to any other function.

So you can have information hiding without classes, and classes without
information hiding. The two concepts are obviously independent, but as
usual, the academics who are in love with OOP like to pretend that
anything that is of any interest whatsoever in computing was invented by
Java and C++.

There are even languages with functions, but no local variables. For
instance, older versions of Forth let you define functions, what Forth
calls "words", but all functions operate on the same global stack.

Python has excellent encapsulation: we can combine code that ought to be
together into a function, related functions into a class, related classes
into a module, and related modules into a package.

Now, lets get to the pretentious philosophical discussion: I guess
encapsulation is quite the opposite of, say, dynamic typing, which is
arguably core in Python.

They are utterly unrelated. Dynamic typing has nothing to do with whether
or not you can encapsulate code into chunks (subroutines, functions,
modules, classes...) or whether you have to write one big amorphous
unstructured program where every chunk of code can reach inside other
chunks of code. Nor does dynamic type have to do with information hiding.
You can have a private member of a class regardless of whether that
member has a single fixed type enforced at compile-time, or a dynamically
typed value enforced at run-time.

 

[Fabrice Pombet]

Widely accepted by whom?

most people(except you, apparently, but I fear that you do not really accept in general)

Python code is *full* of encapsulation. Functions, methods, classes,

modules, packages, even local variables, are all mechanisms for

encapsulating code and data. Those who say that Python has little or no

encapsulation are talking rubbish.




Who is Chaz, and what definition does he have?

See above me (e-mail address removed), the definition of encapsulation from his OST course is fine by y standards, quoting him:
"I'm taking the Python Cert series w/ O'Reilly School of Technology, which I recommend if you've got a good handle on OO programming. In any event, according to what I've learned, "encapsulation is the idea that the only wayto access or change the data inside an object is by calling its methods. This idea has never really gained much ground in the Python world, and it isnormally considered acceptable to both read and set an object's attributesfrom anywhere in a program."

Keep in mind that we are talking about Encapsulation(a general/philosophical principle) as opposed to encapsulating (i.e. setting an argument so that it can only be read/written from within its class/object)
this is a key conceptual point. I agree with you that Python allows you to enforce the encapsulation principle within your code, whenever you want it.But not as a principle that you NEED to respect(as in Java for instance). It is, in my opinion, much better this way.

And now you are talking about information hiding and protection, which is

not the same of encapsulation, no matter what the academics think.

Sometimes the air gets a bit too thin to breathe way up at the top of

those ivory towers...

I am no academic, and I think that's right.

Encapsulation is about grouping code that needs to be together together.

In contract, you have programming languages that give you little, or

nothing, in the way of grouping -- everything is one big chunk of code,

with GOTO or GOSUB to jump from place to place.

I think that I prefer chaz' definition (it is, how could I put it... A tad easier to understand)

Functions and procedures are the first, most simple, form of

encapsulation. Classes allow you to encapsulate multiple functions

("methods") together with the data they need to operate on in one chunk.

Even in C++ or Java, you can have classes that provide no information

hiding at all -- just declare everything "public".



On the other hand, non-OOP languages like C can implement information

hiding. In C, you can hide information from other files by declaring them

as "static". Variables declared inside a brace-delimited block only exist

within that block: local variables are hidden. For example:



int foo;

static int bar;





bar is hidden from other files. Likewise, in this function:





int func(void) {

int baz;

...

}





baz is local to func, and invisible to any other function.



So you can have information hiding without classes, and classes without

information hiding. The two concepts are obviously independent, but as

usual, the academics who are in love with OOP like to pretend that

anything that is of any interest whatsoever in computing was invented by

Java and C++.



There are even languages with functions, but no local variables. For

instance, older versions of Forth let you define functions, what Forth

calls "words", but all functions operate on the same global stack.



Python has excellent encapsulation: we can combine code that ought to be

together into a function, related functions into a class, related classes

into a module, and related modules into a package.










They are utterly unrelated. Dynamic typing has nothing to do with whether

or not you can encapsulate code into chunks (subroutines, functions,

modules, classes...) or whether you have to write one big amorphous

unstructured program where every chunk of code can reach inside other

chunks of code. Nor does dynamic type have to do with information hiding.

You can have a private member of a class regardless of whether that

member has a single fixed type enforced at compile-time, or a dynamically

typed value enforced at run-time.

Steven

well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from outside.. As far as I am concerned this is one hell of an encapsulation violation.... Could you do this -strictly speaking- in Java or C++?

 

[Marco Buttu]

well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from outside.

No, `a` is not an object, so you did not change the type of any object.
`a` is just a name (a label), that initially refers to the tuple (1, 2):
140377464514968

ad after, to another object, of type int:
8752608

The bytecode:
1 0 LOAD_CONST 4 ((1, 2))
3 STORE_NAME 0 (a)
6 LOAD_CONST 5 (5)
9 STORE_NAME 0 (a)
12 LOAD_CONST 3 (None)
15 RETURN_VALUE

Regards, M.

 

[Gary Herron]

... long discussion elided ...
well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from outside. As far as I am concerned this is one hell of an encapsulation violation... Could you do this -strictly speaking- in Java or C++?

Yes, in fact you can do that in C++ and java:

Obj1 a = ...some object...;
{ // new scope...
Obj2 a = ...another object...;
}

On one line, the name 'a' is bound to one object, and later it is bound
to another object. Your Python code is similar, binding the name 'a'
to object (1,2) on one line and the object 5 on the next line. Granted,
Python seems a little freer because, with it's dynamic typing, one
doesn't need to create a new scope to rebind a name, but all languages
with variable names allow some control over binding/rebinding names.

But this has *nothing* at all to do with objects and encapsulation.

Please don't confuse:

the binding of names to objects and

the existence of objects and their encapsulated behavior

They are very different things.

 

[Fabrice Pombet]

On 08/30/2013 11:07 PM, Fabrice Pombet
wrote:


... long discussion elided ...

well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from outside. As far as I am concerned this is one hell of an encapsulation violation.... Could you do this -strictly speaking- in Java or C++?




Yes, in fact you can do that in C++ and java:



Obj1 a = ...some object...;

{ // new scope...

   Obj2 a = ...another object...;

}



On one line, the name 'a' is bound to one object, and later it is
bound to another object.   Your Python code is similar, binding the
name 'a' to object (1,2) on one line and the object 5 on the next
line.  Granted, Python seems a little freer because, with it's
dynamic typing,  one doesn't need to create a new scope to rebind a
name, but all languages with variable names allow some control over
binding/rebinding names.



But this has *nothing* at all to do with objects and encapsulation.



Please don't confuse:

the binding of names to objects and



the existence of objects and their encapsulated behavior


They are very different things.



--
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418

That's interesting, can you do this in C++ or java:

class X():