Making immutable instances

[Paul Rubin]

No I don't believe that. If an object is immutable, then
obj.serialize() should return the same string every time. If you can
add attributes then the serialization output will become different.

 

[Giovanni Bajo]

"Giovanni Bajo" <[email protected]> writes:

[pay attention to the quoting, I didn't write that ]

No I don't believe that. If an object is immutable, then
obj.serialize() should return the same string every time. If you can
add attributes then the serialization output will become different.

I guess it might be argued that the method serialize() could return whatever
value it returned before, and that the added attribute could be "optional" (eg.
think of it as a cache that can be recomputed from the immutable attributes at
any time).

It's kind of a grey area. Surely, I would *not* mind if language support for
immutability prohibited this

 

[Giovanni Bajo]

Not *my* arguments, certainly. Not unless you're seriously
misinterpreting them.

Sorry then, I probably am. There must be a misunderstanding somewhere.

What is your position about frozenset? By my understanding of your arguments,
it is a hand-cuffed version of set, which just prevents bugs that could still
be caught by testing. The same applies for the arbitrary restriction of not
allowing sets to be key dictionaries (with their hash value being their id).

 

[Mike Meyer]

I always thought that adding an attribute was just as bad as changing the
attributes, for an immutable object. But I now see your point (eventually!),
they are pretty different issues.

But, whatever attribute you add to the instance, it should *also* be immutable
(so that, in other words, wouldn't be so different than carrying around a tuple
with the original instance and the added attribute). This said, I think I
devise that a language support for enforcing immutability could allow adding
attributes to instance -- as long as those attributes then become immutable as
well.

That's cool, but only because immutability is such a slippery concept
in Python. We both agree that tuplee are immutable, right? So consider
this:

So I'm perfectly willing to let your class declare that my attributes
be immutable, just so long as you mean the same thing by that as you
mean when you refer to tuples as immutable.

Yes. But I'm familiar with different object semantics, and I found the
immutable objects to be a pretty good replacement of value semantics, and to
implement a kind-of pass-by-value convention in a language which only has
references.

Like I said, I don't have a problem with immutable objects per
se. It's taking away my ability to extend the objects in all the ways
that Python allows that bothers me.

Personally, I haven't found much use for immutable *objects*, as for
immutable *attributes*. Python allows the latter. In Python, you
create an immutable object by creating an object with no mutable
attributes.

I don't think identity is important for immutable objects (as I wrote
elsewhere), so I don't think adding language constucts for this would prove
useful. Instead, immutable objects *are* common, and we still miss a way to
mark them as such.

I think identity is important from an implementation point of
view. Implementation is important - so having constructs that let
you deal with this is useful. Python has those constructs.

What Python doesn't have in general is the ability to "mark"
objects. You have the ability to create attributes that are
immutable. You can create immutable objects by creating objects which
have no mutable attributes. What more do you need?

<mike

 

[Mike Meyer]

No I don't believe that. If an object is immutable, then
obj.serialize() should return the same string every time. If you can
add attributes then the serialization output will become different.

There isn't a standard serialize method in Python, so I don't know how
you want to define it. I can think of perfectly reasonable definitions
of serialize where obj.serialize() won't always return the same string
on an immutable object, even if you don't allow adding attributes.

Personally, I'd argue that serialize shouldn't return the extra
attribute. If you want to add an attribute that gets serialized, you
need to subclass the immutable class, add the attribute there, and
extend serialize to include it. Whether or not that's reasonable
depends on how you want to define serialize.

<mike

 

[Mike Meyer]

Sorry then, I probably am. There must be a misunderstanding somewhere.

I'm seeing posts from you attribute to "Mike Meyer" (me) and "Mike" -
some of which definitely weren't me. So maybe it wasn't mine.

What is your position about frozenset? By my understanding of your arguments,
it is a hand-cuffed version of set, which just prevents bugs that could still
be caught by testing.

I have exactly the same problem with frozensets as I do with sets - I
can't add attributes to either one. That frozenset can't be changed
doesn't bother me particularly.

The same applies for the arbitrary restriction of not allowing sets
to be key dictionaries (with their hash value being their id).

I can't parse that. What do you mean by "key dictionaries"?

<mike

 

[Paul Rubin]

There isn't a standard serialize method in Python, so I don't know how
you want to define it.

Well, consider pickle, for example.

I can think of perfectly reasonable definitions
of serialize where obj.serialize() won't always return the same string
on an immutable object, even if you don't allow adding attributes.

Fair enough. How's this:

a = ImmutableObject()
b = deepcopy(a)
assert a == b # a and b start out equal
.... do stuff ....
# since a and b are immutable, they should still be equal
# no matter what has happened above
assert a == b

If you've added attributes to a but not to b, they should compare
unequal, breaking immutability.

 

[bonono]

Fair enough. How's this:

a = ImmutableObject()
b = deepcopy(a)
assert a == b # a and b start out equal
.... do stuff ....
# since a and b are immutable, they should still be equal
# no matter what has happened above
assert a == b

If you've added attributes to a but not to b, they should compare
unequal, breaking immutability.

How about this ?

a=(1,[])
b=copy.deepcopy(a)
assert a==b # True
a[1].append(1)
assert a!=b

 

[Mike Meyer]

Fair enough. How's this:

a = ImmutableObject()
b = deepcopy(a)
assert a == b # a and b start out equal
.... do stuff ....
# since a and b are immutable, they should still be equal
# no matter what has happened above
assert a == b

If you've added attributes to a but not to b, they should compare
unequal, breaking immutability.

Why should they compare unequal just because you add an attribute?
Nothing says all attributes have to be involved in an equality
comparison. In fact, Python classes by default ignore all attributes
when doing an equality comparison. In order to compare attributes, you
have to provide an __eq__ method (or __cmp__, but we'll ignore
that). It can't mention your new attribute, because it doesn't exist
unless you add it. So your final assertion will be true even after
adding a new attribute.

Of course, there's a fundamental flaw in your definition of
"immutable", in that there are immutable objects - tuples - for which
the condition t1 == t2 is *not* a constant. Tuples can hold mutable
objects, meaning you can change those. Doing so will make a tuple not
compare equal to a copy of the state prior to changing the contents of
the tuple.

<mike

 

[Steven D'Aprano]

Suppose I have the following code.

from module import __take_care__

__private_detail__ = ...

I now have two variable that are flaged the same way, but they are not.

No, you have two names written using a poor naming convention.

__name__ should be used only for Python's special methods.

__take_care__ is a private variable from an other module which I should
use with extreme care not to break the other package.

Python doesn't do any special treatment of __name or __name__ from
modules. The only information hiding techniques Python enforces are that
module._name (single leading underscore) is not imported by "from module
import *", and class.__name (double leading underscore) is mangled to
class._Class__name.

Both techniques are done in order to let the module creator protect users
from *accidentally* shooting themselves in the foot, while still allowing
the module users to *deliberately* shoot themselves in the foot if that is
what they need or want to do. Deliberately breaking these information
hiding techniques are features, not bugs.

__private_detail__ on the other hand is just keeping private data for my
own module, which I should care about as just any other variable in my
module.

If you care about keeping __private_detail__ no more than you care about
public_detail, what makes one private and one public?

It are other modules that should take special care if they
should choose to import this variable.

I'm not sure what the difference is. If there is a difference, why are you
using the same naming convention for different sorts of names ("private
module variable" and "private module data"). If there is no difference, I
don't understand the point of your example.

 

[Steven D'Aprano]

In Python, I can even fix
it so *your* code uses my wrapped version:

import Finney
class Addable(Finnney.Immutable): pass
Finney.Immutable = Addable

Which means that from now on *your* code that tries to create
Immutables will actually get Addables. The inability to do this in B&D
languages is - well, painfull. That Python doesns't require the
boilerplate in a good thing.

The ability to shoot yourself in the foot *if you want to* is a feature,
not a bug. But the ability to shoot yourself in the foot *accidentally* is
not a feature.

I am happy that whatever restrictions the class designer builds on their
class is easy to bypass, because the class designer is not omniscient
and their "private data" may be just the thing I need to solve a
problem some day. But I should have to *think about it* before messing
about with it.

Python's consenting adults philosophy allows the class designer some
limited ability to force the class user to think about it before messing
about with private variables. I think Ben's immutable class falls into
that same category.

 

[Mike Meyer]

Python's consenting adults philosophy allows the class designer some
limited ability to force the class user to think about it before messing
about with private variables. I think Ben's immutable class falls into
that same category.

If you read the whole thread, you'll see that I don't have a problem
with messing with private variables. It's disallowing the addition of
new variables that I found objectionable.

<mike

 

[Steven D'Aprano]

If you read the whole thread, you'll see that I don't have a problem
with messing with private variables. It's disallowing the addition of
new variables that I found objectionable.

But you've just demonstrated that those who really want to shoot
themselves in the foot, er, add new attributes (not variables, let's get
the terminology right) can do so by sub-classing and rebinding. The fact
that Ben's proposal would prevent you from adding attributes is precisely
analogous to the name mangling that Python does: if you want to work
around it, it is easy to do so, but you have to make a conscious effort to
do so it.

"Hmmm, the class designer didn't want me adding attributes to instances...
maybe he had a good reason for that..."

 

[Mike Meyer]

"Hmmm, the class designer didn't want me adding attributes to instances...
maybe he had a good reason for that..."

When it was suggested that a facility for doing this be added to the
language, I asked for a use case for it. Nobodies come up with a
reason for placing such restriction on the client yet. If you've got a
use case, I'd be interested in hearing it.

<mike

 

[Steven D'Aprano]

When it was suggested that a facility for doing this be added to the
language, I asked for a use case for it. Nobodies come up with a
reason for placing such restriction on the client yet.

Oh, I think it is terribly unfair to call them nobodies, even if you don't
like their use-cases *grin*

If you've got a use case, I'd be interested in hearing it.

frozenset perhaps? If it were needed once, it could be needed again.

The obvious case would be for a class where distinct instances that
compare equal but not identical map to the same value in a dict.

In any case, I'm not the one claiming that I need custom immutable
classes. I'm just suggesting that there is nothing non-Pythonic about
them. If Ben thinks he needs them, I'm sure he has put *far* more thought
into it than I have. I know Ben in RL, and he is not someone to make snap
judgements about turning Python into Some Other Language Just Because.

 

[Mike Meyer]

frozenset perhaps? If it were needed once, it could be needed again.

That's not a use case, that's an example. And not a very good one, as
it's not at all clear that the restriction is intentional in that
case. After all, the same restriction applies to every builtin type,
including the mutable version of frozenset.

The obvious case would be for a class where distinct instances that
compare equal but not identical map to the same value in a dict.

How does the ability to add attributes to the instances of the class
change that behavior?

In any case, I'm not the one claiming that I need custom immutable
classes. I'm just suggesting that there is nothing non-Pythonic about
them. If Ben thinks he needs them, I'm sure he has put *far* more thought
into it than I have. I know Ben in RL, and he is not someone to make snap
judgements about turning Python into Some Other Language Just Because.

I claim that the dynamic natture of Python - which is exemplified by
things like duck typing and the ability to add attributes to nearly
everything on the fly - is a fundamental part of what makes Python
Python. The best reason Ben could come up with is that it makes
finding bugs a bit easier. But so do type declarations, static
namespaces, private and protected attributes, and a slew of similar
B&D features that are pretty much anathema to dynamic languages. This
feature fits *very* well in languages that have those features, and
poorly in languages that reject them, which includes Python.

Of course, that a feature has a lot in common with features from
un-Pythonic languages doesn't make it ipso facto unPythonic. After
all, practicality beats purity. So what's the practical application
for such a feature? What's the use case?

<mike

 

[Steven D'Aprano]

That's not a use case, that's an example.

Fine. "I want to use a set as a dictionary key, but dict keys must be
immutable."

And not a very good one, as
it's not at all clear that the restriction is intentional in that
case. After all, the same restriction applies to every builtin type,
including the mutable version of frozenset.

Every builtin type, including mutable sets, is immutable???

I think we're talking at cross purposes. I'm talking about immutable
instances. What are you talking about?

How does the ability to add attributes to the instances of the class
change that behavior?

Er, who said it did?

I claim that the dynamic natture of Python - which is exemplified by
things like duck typing and the ability to add attributes to nearly
everything on the fly - is a fundamental part of what makes Python
Python.

I can't disagree with you there.

The best reason Ben could come up with is that it makes
finding bugs a bit easier.

Are you sure that was Ben? Maybe I missed it.

But so do type declarations, static
namespaces, private and protected attributes, and a slew of similar
B&D features that are pretty much anathema to dynamic languages. This
feature fits *very* well in languages that have those features, and
poorly in languages that reject them, which includes Python.

Perhaps. But I don't object to *mild* B&D, the sort you can get out of
relatively easily. You know, handcuffs, maybe a few ropes, but not
lock-them-up-in-the-oubliette-and-throw-away-the-key *wink*

All joking aside, I think having immutable custom classes, with or without
restricting attribute creation, is no worse than (say) Python's name
mangling.

Of course, that a feature has a lot in common with features from
un-Pythonic languages doesn't make it ipso facto unPythonic. After
all, practicality beats purity. So what's the practical application
for such a feature? What's the use case?

Class instances match by identity when used as keys, not equality. That's
over-strict: why should Parrot("Norwegian Blue") map to a different item
from Parrot("Norwegian" + " Blue") just because the first instance has a
different memory location to the second?

Please note, I do not expect -- and would not want -- the default
behaviour to change. Most class instances presumably should be mutable,
and therefore mapping by ID is the right behaviour.

But some class instances shouldn't be mutable, and should match as keys by
equality not identity: we would expect Fraction(num=1, den=2) to match
Fraction(num=2, den=4) as a key.

 

[Mike Meyer]

Fine. "I want to use a set as a dictionary key, but dict keys must be
immutable."

Except that dict keys don't have to be immutable. The docs say (or
imply) that, but they're wrong. We're discussing new wording in
another thread.

Every builtin type, including mutable sets, is immutable???

No. I think you missed the entire second half the thread.

I think we're talking at cross purposes. I'm talking about immutable
instances. What are you talking about?

I'm talking about being able to add attributes to an instance. You
can't add an attribute to *any* builtin type. This is an
implementation detail. I consider it to be a wart in the language, but
an acceptable one, because the costs of fixing it are much worse than
the costs of working around it, and practicality beats purity.

Er, who said it did?

You did. We're looking for a use case for adding the ability to mark a
class as having instances that you can't add an attribute to. You
suggested this as such a use case.

Are you sure that was Ben? Maybe I missed it.

No, I'm no sure it was Ben - I didn't go chase down the
reference. It's the best use case *from anyone* so far, though.

All joking aside, I think having immutable custom classes, with or without
restricting attribute creation, is no worse than (say) Python's name
mangling.

There are well-defined facilities for creating read-only
attributes. Is there a difference between an immutable object you can
add attributes to, and an object that has nothing but read-only
attributes?

Class instances match by identity when used as keys, not equality. That's
over-strict: why should Parrot("Norwegian Blue") map to a different item
from Parrot("Norwegian" + " Blue") just because the first instance has a
different memory location to the second?

Uh - HTH did we get here?

Please note, I do not expect -- and would not want -- the default
behaviour to change. Most class instances presumably should be mutable,
and therefore mapping by ID is the right behaviour.

The default behavior is right for in many cases. For those that it's
not right, you can fix it. Proper definition of the Parrot class will
insure that Parrot("Norwegian Blue") == Parrot("Norwegian" + " Blue")
always evaluates to True.

But some class instances shouldn't be mutable, and should match as keys by
equality not identity: we would expect Fraction(num=1, den=2) to match
Fraction(num=2, den=4) as a key.

And you can do that with Python as it exists today (and one of these
days I'll get back to my Rational class that does that...). There's no
need for any changes to the language to deal with this case.

<mike

 

[Antoon Pardon]

No, you have two names written using a poor naming convention.

Well if it is a poor naming convention, why react to me, and not to
Mike who was defending this poor naming convention?

__name__ should be used only for Python's special methods.


Python doesn't do any special treatment of __name or __name__ from
modules. The only information hiding techniques Python enforces are that
module._name (single leading underscore) is not imported by "from module
import *", and class.__name (double leading underscore) is mangled to
class._Class__name.

We were not talkin about special treatment by python. We were talking
about conventions to communicate purpose to other readers of the
software.

I'm not sure what the difference is. If there is a difference, why are you
using the same naming convention for different sorts of names ("private
module variable" and "private module data"). If there is no difference, I
don't understand the point of your example.

Well it seems you didn't seem to understand the point of my answer.
Maybe you should first reread the article I responded too.

 

[Paul Rubin]

When it was suggested that a facility for doing this be added to the
language, I asked for a use case for it. Nobodies come up with a
reason for placing such restriction on the client yet. If you've got a
use case, I'd be interested in hearing it.

I see it the other way; almost all the time in my own code I try to
initialize all of any instance's attributes in an __init__ method.
Dynamically adding attributes is useful sometimes, but done
willy-nilly leads to spaghetti code. In the cases where it's useful,
I'd rather see it done through a special __setattr__ method, perhaps
inherited from a mixin:

class Frob(Whatsit, DynamicAttributeMixin):
...

However, that's not likely to happen.