Encapsulation unpythonic?

[Fabrice Pombet]

That's interesting, can you do this in C++ or java:



class X():

def __init__(self, *arg):
for x in arg:
self.x=x

and then:

a=X("x","y","z")
and then:
a.w="w"
?

I guess my point was dynamic typing and encapsulation go a little in opposite directions in terms of philosophy, and it is therefore clear that Pythonprivileges "dynamic typing" kind of thinking over encapsulation as a philosophical stance. Am I the only one thinking like this?

 

[Steven D'Aprano]

well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from
outside.

Incorrect. You have not changed the type or value of any object. "a" is
not an object, it is a *name*, and while you can change the object bound
to the name, the objects remain unchanged.

When you do this:

x = 23
x = 42

the *object* 23 does not change, only the name binding changes. To do
otherwise would cause all sorts of surprises:

# THIS DOES NOT HAPPEN IN PYTHON
# or any other language, as far as I am aware
x = 23
y = x # y now has the value 23
x = 42 # change the value of the object ### NOT SO! ###
print y
=> prints 42

Name binding (assignment) does not change objects. It changes the link
between a name and the object, but the object remains untouched (unless
it is unbound, and garbage collected). Assignment is not mutation.
Assigning to a name does not modify the object that was previously bound.


But even if you were right about changing the type and value of objects
in place -- Python allows you to mutate lists and dicts in place, and
even mutate the type of some objects, although not built-ins -- your
understanding is still confused. Re-assignment (re-binding) of local
variables is hardly a violation of encapsulation. But if it was, then
Java and C++ have no encapsulation either, because you can re-assign to
local variables inside a function too.

If you want to see a language without encapsulation, you need to look at
something like 1970s-style BASIC, a language where all variables are
global, where there is no support for grouping related code into separate
modules or files, where the closest thing to a function call is GOTO or
GOSUB.

Of course, languages can have *more* or *less* encapsulation than other
languages. C lets you encapsulate related functions into a file, but it
has no way of grouping data and functions together except loosely, in a
file. C++ adds classes, which has more encapsulation since you can group
functions and their data together.

As far as I am concerned this is one hell of an encapsulation
violation... Could you do this -strictly speaking- in Java or C++?

Of course you could. All you need is a way to tell the compiler not to
type-check the variable "a". Or more practically, some way to declare
variable "a" as a reference to an untyped or generic value. C# has the
dynamic keyword for this functionality. I don't know about Java or C++,
but the JVM is certainly capable of implementing dynamic typing as there
are various dynamically-typed languages built on top of the JVM, such as
OpenXION and Cobra.

 

[Ned Batchelder]

Incorrect. You have not changed the type or value of any object. "a" is
not an object, it is a *name*, and while you can change the object bound
to the name, the objects remain unchanged.

When you do this:

x = 23
x = 42

the *object* 23 does not change, only the name binding changes. To do
otherwise would cause all sorts of surprises:

# THIS DOES NOT HAPPEN IN PYTHON
# or any other language, as far as I am aware
x = 23
y = x # y now has the value 23
x = 42 # change the value of the object ### NOT SO! ###
print y
=> prints 42

Name binding (assignment) does not change objects. It changes the link
between a name and the object, but the object remains untouched (unless
it is unbound, and garbage collected). Assignment is not mutation.
Assigning to a name does not modify the object that was previously bound.

I wrote a piece about names and values that might help clarify these
points: Facts and Myths about Names and Values in Python:
http://nedbatchelder.com/text/names.html

--Ned.

 

[Fabrice Pombet]

http://nedbatchelder.com/text/names.html



--Ned.

This is an excellent explanation, thank you. It is mostly of theoretical interest though, and in practice, I still contend that the consequences towards the syntax are (or seem, if you prefer) analogous to those of the lack of encapsulation.

 

[Fabrice Pombet]

Incorrect. You have not changed the type or value of any object. "a" is

not an object, it is a *name*, and while you can change the object bound

to the name, the objects remain unchanged.



When you do this:



x = 23

x = 42



the *object* 23 does not change, only the name binding changes. To do

otherwise would cause all sorts of surprises:



# THIS DOES NOT HAPPEN IN PYTHON

# or any other language, as far as I am aware

x = 23

y = x # y now has the value 23

x = 42 # change the value of the object ### NOT SO! ###

print y

=> prints 42



Name binding (assignment) does not change objects. It changes the link

between a name and the object, but the object remains untouched (unless

it is unbound, and garbage collected). Assignment is not mutation.

Assigning to a name does not modify the object that was previously bound.





But even if you were right about changing the type and value of objects

in place -- Python allows you to mutate lists and dicts in place, and

even mutate the type of some objects, although not built-ins -- your

understanding is still confused. Re-assignment (re-binding) of local

variables is hardly a violation of encapsulation. But if it was, then

Java and C++ have no encapsulation either, because you can re-assign to

local variables inside a function too.



If you want to see a language without encapsulation, you need to look at

something like 1970s-style BASIC, a language where all variables are

global, where there is no support for grouping related code into separate

modules or files, where the closest thing to a function call is GOTO or

GOSUB.



Of course, languages can have *more* or *less* encapsulation than other

languages. C lets you encapsulate related functions into a file, but it

has no way of grouping data and functions together except loosely, in a

file. C++ adds classes, which has more encapsulation since you can group

functions and their data together.











Of course you could. All you need is a way to tell the compiler not to

type-check the variable "a". Or more practically, some way to declare

variable "a" as a reference to an untyped or generic value. C# has the

dynamic keyword for this functionality. I don't know about Java or C++,

but the JVM is certainly capable of implementing dynamic typing as there

are various dynamically-typed languages built on top of the JVM, such as

OpenXION and Cobra.

Steve, I think that your definition of encapsulation is too wide to give a reasonable answer to the question at hand. If I understand you correctly, you are taking encapsulation as a language characteristic, rather than a principle.
Plus, you seem to forget that encapsulation is an OOP principle, and, forgive me if I am wrong, does not apply normally to functions or languages likeC.
Please read Steve Holden's (in chaz') definition, and tell us whether you think that Python enforces strongly this principle, I think that would be a good basis for an agreement. My answer is no, it doesn't, but it allows youto abide by it if you want to. Unlike Java or C++ who would tend to do exactly the contrary (enforces it strictly, and (possibly?) allow you to discard it at times with a few jiffies (or not? I don't know))

 

[Dennis Lee Bieber]

well, look at that:

a=(1,2)
a=2+3 ->a is an object and I have changed its type and value from outside. As far as I am concerned this is one hell of an encapsulation violation... Could you do this -strictly speaking- in Java or C++?

There is where your major misunderstanding is...

"a" is a NAME attached (bound) to an object. In the first statement, the
object is the tuple (1,2). That object was not changed when you execute the
second statement -- which is taking two integer objects and creating a new
integer object having a value of '5', and then attaches the NAME "a" to the
new object. If no other names are bound to the (1,2) object, it will be
garbage collected.

At no time do you change the type/value of the OBJECT.

 

[Tim Delaney]

outside. As far as I am concerned this is one hell of an encapsulation
violation... Could you do this -strictly speaking- in Java or C++?

There is where your major misunderstanding is...

"a" is a NAME attached (bound) to an object. In the first statement, the
object is the tuple (1,2). That object was not changed when you execute the
second statement -- which is taking two integer objects and creating a new
integer object having a value of '5', and then attaches the NAME "a" to the
new object. If no other names are bound to the (1,2) object, it will be
garbage collected.

I'll try another way to explain it, using Java terminology(since Fabrice
appears to be familiar with Java).

Object a = Arrays.asList(1, 2); // a is a reference to the List<Integer>
returned by Arrays.asList
a = Integer.valueOf(2 + 3); // a is now a reference to the Integer
returned by Integer.valueOf

You have not changed the type of 'a' in any way - you have simply changed
what the name 'a' refers to. This is functionally identical to your Python
code above,except that in Python you do not have to downcast the Object
reference 'a' or use reflection to call methods on it or access it's
members (think of it as Python does reflection automatically for you).

Tim Delaney

 

[Steven D'Aprano]

Steve, I think that your definition of encapsulation is too wide to give
a reasonable answer to the question at hand.

My definition of encapsulation is based on the plain language definition.
It is also common in computer programming circles to talk about
encapsulation in this way. OOP designers may (or may not) have been the
first to declare "encapsulation" was a design principle, but programmers
have been using the *principle* of encapsulation since before Grace
Hopper was an admiral.

See, for example, coders talking about encapsulation in C and Powershell:

http://ejrh.wordpress.com/2011/04/29/encapsulation-in-c/

https://www.simple-talk.com/dotnet/.net-tools/further-down-the-rabbit-
hole-powershell-modules-and-encapsulation/

OOP languages give us *more* and *better* ways to encapsulate code and
data, but they did not invent the principle.

If I understand you
correctly, you are taking encapsulation as a language characteristic,
rather than a principle.

No, it is both. The coder may or may not decide to encapsulate code in
subroutines/functions/classes, and the language may or may not allow it.

Languages differ in their ability to allow the programmer to encapsulate.
Some languages, like early BASIC, give you no ability to encapsulate code
or data at all. All variables are global, and there are no functions,
just a big blob of code in a single file. There aren't even data
structures as such, except strings, so you cannot even group related
pieces of data into a struct or record.

Some languages, like Python, give you many ways to encapsulate code and
data: you can group related code in a function, related functions in a
class, related classes in a module, related modules in a package. That's
pretty much exactly the same sort of things that you can do in Java. C++
has an additional "namespace" data structure that Python doesn't have,
but that's just a mechanism for encapsulation.

Encapsulation and information hiding are distinct things -- you can have
one without the other. C, for example, creates a new scope inside for-
loops, so that the for-loop variable is hidden from the rest of the
function. Apart from a pair of braces, there is no encapsulation, but
there is information hiding. Or you could design a language that
encapsulated code into functions and classes, but put all variables in a
single, global, shared namespace (little, or no, information hiding).

It is a myth, and a very obnoxious one, that encapsulation and
information hiding were invented by OOP. What is a function but a way to
hide the implementation of a chunk of code from the caller? What are
local variables but a way to hide variables used by one function from
another? Coders were using information hiding, separation of concerns,
and encapsulation in the 1950s, long before OOP. They just didn't call
them by those names. They just called them "writing good code".

Actually, functions are *not necessarily* a way to hide implementation.
There are languages where you can jump into the middle of a function. So
you can encapsulate a chunk of code into a function, without hiding the
implementation details. Just as you can encapsulate code and data into a
class, without hiding the implementation details, if you declare
everything public.

Plus, you seem to forget that encapsulation is
an OOP principle, and, forgive me if I am wrong, does not apply normally
to functions or languages like C.

I haven't forgotten it, because it isn't true.

One of the most obnoxious and annoying traits of OOP zealots, especially
academics, is that they act as if programming did not exist before Java
and C++, or if you're really lucky, Smalltalk. (Somehow they nearly
always forget about Simula.) Perhaps because OOP was so late to be
invented (structured programming goes back to Fortran in the 1950s,
functional programming to Lisp only a few years after that), and because
it was so heavily hyped as "the solution" to every programming
difficulty, too many people ignore anything outside of the OOP. They
wrongly believe that since Python isn't a "pure" OOP language (according
to some bizarre understanding of "pure" that often considers C++ and Java
pure), Python cannot possibly have "OOP principles" like encapsulation,
information hiding, separation of concerns.

That point of view is sheerest nonsense.

Please read Steve Holden's (in chaz')
definition, and tell us whether you think that Python enforces strongly
this principle, I think that would be a good basis for an agreement.

Are you referring to this quote?

"encapsulation is the idea that the only way to access or change the data
inside an object is by calling its methods."


I disagree with that definition. That's a poor definition, one that has
no relation to the plain English meaning of the word "encapsulation", nor
to how the word is commonly used in the great bulk of programming
circles. By referring to "objects" and "methods" it wrongly assumes that
encapsulation can only apply to OOP.

I quote from The Free On-line Dictionary of Computing:

encapsulation

1. The technique used by layered protocols in which a layer
adds header information to the protocol data unit (PDU) from
the layer above. As an example, in Internet terminology, a
packet would contain a header from the physical layer,
followed by a header from the network layer (IP), followed by
a header from the transport layer (TCP), followed by the
application protocol data.

2. The ability to provide users with a well-defined interface
to a set of functions in a way which hides their internal
workings. In object-oriented programming, the technique of
keeping together data structures and the methods (procedures)
which act on them.


Definition 1 is irrelevant to this discussion, but definition 2 makes it
absolutely clear that it is not just about OOP.

My
answer is no, it doesn't, but it allows you to abide by it if you want
to. Unlike Java or C++ who would tend to do exactly the contrary
(enforces it strictly, and (possibly?) allow you to discard it at times
with a few jiffies (or not? I don't know))

Java and C++ allow you to declare members as public, so it is *not true*
that calling methods is the only way to change members. If you accept
Steve Holden's (wrong) definition above, Java and C++ don't have
encapsulation either.

 

[Chris Angelico]

Java and C++ allow you to declare members as public, so it is *not true*
that calling methods is the only way to change members. If you accept
Steve Holden's (wrong) definition above, Java and C++ don't have
encapsulation either.

That said, though, when you consider the language ecosystem rather
than just the language, there is a strong tendency for Java and C++
code to wrap everything up with functions (no public data members),
whereas Python code is far more likely to have external code directly
access data inside an object. You usually will find Java code calling
methods to change members, whereas that's done in Python only when
there's a need for it.

ChrisA

 

[Fabrice Pombet]

That said, though, when you consider the language ecosystem rather

than just the language, there is a strong tendency for Java and C++

code to wrap everything up with functions (no public data members),

whereas Python code is far more likely to have external code directly

access data inside an object. You usually will find Java code calling

methods to change members, whereas that's done in Python only when

there's a need for it.

Yep, this is precisely my point, if you take encapsulation as a philosophical principle, Java and C++ would tend to be abiding by it, as a "default" setting that you can at times change, whereas python would tend to be the contrary. In other words, you can set some encapsulation if and when you wantto, but you can leave your code without it when it's not needed/inconvenient. So I guess that we are actually all agreeing on this one.

 

[Ethan Furman]

So I guess that we are actually all agreeing on this one.

No, we are not.

"encapsulation" != "inaccessible except by getters/setters"

 

[Roy Smith]

No, we are not.

"encapsulation" != "inaccessible except by getters/setters"

Nothing is accessible in Python except via getters and setters. The
only difference between Python and, say, C++ in this regard is that the
Python compiler writes them for you most of the time and doesn't make
you put ()'s at the end of the name

 

[Ethan Furman]

Nothing is accessible in Python except via getters and setters. The
only difference between Python and, say, C++ in this regard is that the
Python compiler writes them for you most of the time and doesn't make
you put ()'s at the end of the name

class Javaesque:

__value = None

def get_value(self):
return self.__value

def set_value(self, new_value):
validate(new_value)
self.__value = new_value


class ProtectedPython:

_value = None

@property
def value(self):
return self._value

@value.setter
def value(self, new_value)
validate(new_value)
self._value = new_value

class PlainPython:

value = None


In the Javaesque class we see the unPythonic way of using getters/setters; in the ProtectedPython* class we see the
pythonic way of providing getters/setters**; in the PlainPython class we have the standard, unprotected, direct access
to the class attribute.

No where in PlainPython is a getter/setter defined, nor does Python define one for us behind our backs.

If you have evidence to the contrary I'd like to see it.


* Not the best name, but oh well.
** In Python, using @property makes getter/setter usage look just like normal attribute usage, which is cool.

 

[Tim Delaney]

class PlainPython:

value = None


In the Javaesque class we see the unPythonic way of using getters/setters;
in the ProtectedPython* class we see the pythonic way of providing
getters/setters**; in the PlainPython class we have the standard,
unprotected, direct access to the class attribute.

No where in PlainPython is a getter/setter defined, nor does Python define
one for us behind our backs.

If you have evidence to the contrary I'd like to see it.

I think Roy is referring to the fact that attribute access is implemented
via __getattr__ / __getattribute__ / __setattr__ / __delattr__. From one
point of view, he's absolutely correct - nearly all attributes are accessed
via getters/setters in Python.

Tim Delaney

 

[Roy Smith]

I think Roy is referring to the fact that attribute access is implemented
via __getattr__ / __getattribute__ / __setattr__ / __delattr__. From one
point of view, he's absolutely correct - nearly all attributes are accessed
via getters/setters in Python.

Tim Delaney

Thank you.

 

[Ethan Furman]

I think Roy is referring to the fact that attribute access is implemented via __getattr__ / __getattribute__ /
__setattr__ / __delattr__. From one point of view, he's absolutely correct - nearly all attributes are accessed via
getters/setters in Python.

Seems to me there is a difference between an underlying generic protocol for data manipulation and "Python writing them
[getters/setters] for you".

 

[Roy Smith]

One of the more annoying traits of humanity is that whatever context we
first encounter a term is disproportionately privileged, causing us to
irrationally dismiss better (more useful, more reasonable, more
pedagogically appropriate, more historically correct, etc.) definitions.

Known in the education world as "The law of primacy".

 

[Roy Smith]

I think Roy is referring to the fact that attribute access is implemented
via __getattr__ / __getattribute__ /
__setattr__ / __delattr__. From one point of view, he's absolutely correct
- nearly all attributes are accessed via
getters/setters in Python.

Seems to me there is a difference between an underlying generic protocol for
data manipulation and "Python writing them
[getters/setters] for you".

Why? When I write "foo.bar", a bunch of generic code gets run which
figures out what value to return. If I don't like the generic behavior,
I can write my own __getattrr__(), etc, and make it do whatever I want.

How is that any different from, in C++, if you don't write a default
constructor, the compiler will write one for you. If you don't like the
generic behavior you get from that, you can write your own and make it
do whatever you want.

 

[Steven D'Aprano]

Nothing is accessible in Python except via getters and setters. The
only difference between Python and, say, C++ in this regard is that the
Python compiler writes them for you most of the time and doesn't make
you put ()'s at the end of the name

Very clever! Pedantic, and an unusual look at what's going on under the
hood!

I wanted to say it was *not quite correct*, because you can read or write
directly to the instance dict:

instance.__dict__['name'] = 42


If I understand Python's internals correctly, __dict__ is a slot, and so
bypasses the usual getattr machinary. But even if so, __dict__['name']
uses the dictionary __get/setitem__ method, so it's still a getter/setter
under the hood.

In any case, even if you are *technically* correct that Python has
getters and setters under the hood, that's not quite what the discussion
here is about. But I'm sure you realise that

 

[Steven D'Aprano]

Roy Smith wrote:

Nothing is accessible in Python except via getters and setters. The
only difference between Python and, say, C++ in this regard is that
the Python compiler writes them for you most of the time and doesn't
make you put ()'s at the end of the name

I think Roy is referring to the fact that attribute access is
implemented via __getattr__ / __getattribute__ /
__setattr__ / __delattr__. From one point of view, he's absolutely
correct - nearly all attributes are accessed via getters/setters in
Python.

Seems to me there is a difference between an underlying generic
protocol for data manipulation and "Python writing them
[getters/setters] for you".

Why? When I write "foo.bar", a bunch of generic code gets run which
figures out what value to return. If I don't like the generic behavior,
I can write my own __getattrr__(), etc, and make it do whatever I want.

How is that any different from, in C++, if you don't write a default
constructor, the compiler will write one for you. If you don't like the
generic behavior you get from that, you can write your own and make it
do whatever you want.

And fundamentally, all programming is flipping bits, therefore all
languages are exactly the same, right?


I can't speak for C++, but comparing Java and Python there are
differences:


- Java the language makes it a pain to change your mind and convert a
public attribute to a computed attribute. Since the pain of changing your
mind far outweighs the pain of writing trivial getters/setters up front,
it is good defensive practice to make attribute access via getters just
in case. Python makes it trivial to change your mind, and so YAGNI rules
and you shouldn't write getters unless you actually need them.

- Perhaps because the focus in Java is on massive projects with large
numbers of barely adequate coders, Java tries to protect the average
coder from shooting themselves in the foot. Consequently, Java encourages
a philosophy of "default deny" when it comes to attribute access: don't
give your caller access to anything except the absolute minimum you know
they need. In Python, the focus tends to be more about smaller projects
with small teams of better than average coders, and a philosophy of
"we're all adults here". Consequently, Python code tends towards
"everything not explicitly prohibited is permitted".

- Similarly, while Java the language doesn't force you to *explicitly*
declare members as public (if you don't declare members private, they are
public), it strongly encourages you to think about information hiding and
explicitly mark members as public. Python does not. There is no way to
explicitly mark attributes as public.

- The Java compiler enforces public/private, while Python treats it as a
convention. (At least for pure-Python code.)


But notice that they are differences of degree, not kind. Java encourages
information hiding in classes, but does not prohibit you from making
members public, and using reflection you can break into classes and get
access to anything; Python allows information hiding, but trusts the
programmer to honour "private" names, and reflection is such a
fundamental part of Python that we don't even call it that.