Tuples and immutability

[Mark H. Harris]

How would you propose doing that? Bear in mind that while Python
knows that tuples specifically are immutable, it doesn't generally
know whether a type is immutable.

I was going to bed, and then thought of the solution. Allow the action.

Hold the error pending until the action either succeeds or fails for the "item," in this case a list, and then post the error for the tuple depending.... this would make the whole mess consistent, and even compatible with the idea of dynamic name and type binding...

So, in other words, the action should be permitted, and actually was allowed, and the message is incorrectly reporting something that for this particular "item" is NOT true. The error message (and timing of same) is the bug (philosophically).

Under the covers it may be (and probably is) more complicated.

Good night everyone.

Cheers

 

[Chris Angelico]

This is what I mean... the error message is telling the user that it cannot do what he has requested, and yet IT DID. You have one of two scenarios: 1) the message is arbitrarily lying and it really can assign an immutable's item... (and it did!) or 2) It really should NOT assign an immutables item (the message is truth) but the action was "allowed" anyway despite the protocol and accepted policy... in which case the two scenarios add up to a giant logical inconsistency... QED a bug.

It did not assign anything. The error was raised because the tuple
rejects item assignment. It's that simple.

The object was mutated before the assignment was attempted.

ChrisA

 

[Mark Lawrence]

hi Ian, consider the problem... its a "cake" and "eat it too" scenario... you can't have your cake and eat it too....

This is what I mean... the error message is telling the user that it cannot do what he has requested, and yet IT DID. You have one of two scenarios: 1) the message is arbitrarily lying and it really can assign an immutable's item... (and it did!) or 2) It really should NOT assign an immutables item (the message is truth) but the action was "allowed" anyway despite the protocol and accepted policy... in which case the two scenarios add up to a giant logical inconsistency... QED a bug.

There really is no way to get around this from the average user's perspective. And therefore, this is going to come up again and again and again... because coders don't like unexpected logical inconsistencies.

It is not helpful either to explain it away by knowing how the policy works under the covers... the average user of the python language should not have to understand the policy of the underlying substructure...

1) either they can't assign the list item of an immutable tuple type (and the action is a flaw, say bug

of

2) they really CAN set the immutable's list item type (which it did!) and the message is bogus.


Its one way or the other.... we can't have our cake and eat it too... this is (one way or the other) a bug.

IMHO

would you please be kind enough to use plain English... your use of
sentences like this... is starting to annoy me...

you're also using google groups... it doesn't wrap paragraphs
correctly... please read and action this
https://wiki.python.org/moin/GoogleGroupsPython... it does wrap
paragraphs correctly... it also prevents us seeing double line spacing...

do you get my drift???

 

[Ned Batchelder]

One very common example of tuples containing lists is when lists are
passed to any function that accepts *args, because the extra arguments
are passed in a tuple. A similarly common example is when returning
multiple objects from a function, and one of them happens to be a
list, because again they are returned in a tuple.

def f(*args):
print(args)
return (args[1:]

result = f(1, 2, 3, [4, 5]) (1, 2, 3, [4, 5])
print(result)

(2, 3, [4, 5])

I agree Ian... good points all. ... again, I'm not arguing with anyone... just saying that an error (whatever we mean by that) should not half-way-fail.... we are only pointing out the problem... we have not idea what the solution is yet.

Intuitively everyone can see that there is a problem here... the debate cannot be answered either because of the inherent design of python (almost all of which we love). So, as they say, what is a mother to do? ... I mean, some people's kids...

I don't know how I propose to handle the problem... I think the first step is getting everyone to agree that there IS a problem... then debate how to tackle the solution proposals.

marcus

Everyone can agree that it is not great to raise an exception and also
leave the list modified. But I very much doubt that anything will be
done to change the situation. All of the solutions are too extreme, and
bring their own infelicities, and the actual problems caused by this
scenario are vanishingly small.

BTW: I also am mystified why you uses ellipses to end your sentences,
surely one period would be enough?

 

[Oscar Benjamin]

Where is `.__iadd__()` called outside of `list += X`?

Numpy uses it for in-place operations with numpy arrays:

import numpy
a = numpy.arange(10)
a array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
a[::2] += 10
a

array([10, 1, 12, 3, 14, 5, 16, 7, 18, 9])

It makes sense for any mutable type that supports +. The stdlib
doesn't have many of these. The obvious one is list but there's also
MutableString (in 2.x):

'qweasd'

If the only difference from `.extend()` is that it returns `self`, but the list was
already modified anyway, why bother with reassignment?

I don't know of any situation where __iadd__ is defined but doesn't
return self and I can't think of a use case apart from operator abuse.

I had thought that the other difference was that .extend would accept
any iterable but it seems += does also. Perhaps that was changed at
some point.

l = [1, 2, 3]
l + (1, 2, 3)

Traceback (most recent call last):
[1, 2, 3, 1, 2, 3]


Oscar

 

[Mark H. Harris]

you're also using google groups... it doesn't wrap paragraphs

correctly... please read and action this

https://wiki.python.org/moin/GoogleGroupsPython... it does wrap

paragraphs correctly... it also prevents us seeing double line spacing....



do you get my drift???

hi Mark, yes, I understand. I am using google groups (for the moment) and it does not wrap correctly, at least not on my end. It also adds additional carrots > that make replying difficult.

The elipses thing is a bad habit which I 'picked up' in academia lately. We use elipses frequently in papers (also this---which also annoys people) and there is no problem. In a writing context the elipses seem to function for some people (intentionally) as 'um'. When humans speak they use the word 'um' to pause between though context and its usually unintentional at the surface. Often when I write the elipses (...) show up for the same reason. I really don't know that I'm doing it... it just happens. So, I am working on it. No, you're not the first to mention it. Its one of my many flaws.

Thanks for keeping me accountable.

Enjoy the day!

marcus

 

[Mark H. Harris]

How would you propose doing that? Bear in mind that while Python
knows that tuples specifically are immutable, it doesn't generally
know whether a type is immutable.

hi Ian, I thought of something else after I slept on it, so to speak. Consider this:

s=(2,3,[42,43,44],7)
s[2] [42, 43, 44]

s[2] is a list. We should be able to change s[2] even though its within a tuple, like the:

[42, 43, 44]

s[2].append(45)
s[2] [42, 43, 44, 45] <===== no error

or like this:

s[2] [42, 43, 44, 45]
s[2]=s[2]+[46]

Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
s[2]=s[2]+[46] <==========error, but look below======
TypeError: 'tuple' object does not support item assignment

s[2] [42, 43, 44, 45] <=========also no change to list !!

The point I'm trying to make with this post is that s[2]+=[46] and s[2]=s[2]+[46] are handled inconsistently. In each case (whether the listis part of a tuple or not) s[2] is a valid list and should be mutable. In every case, in fact, it is. In each case, in fact, the change might occur. But, the change occurs in the first case (append) without an error. The change occurs in the next case too (+=) but with an error not relatedto the tuple. The change does not occur in the third case (s=s+ with anerror) but the list is not changed !

We all know this is not good. We also all know that its because of implementation details the user/coder should not need to know. QED: a bug.

Having slept on it, I am remembering my Zen of Python: Errors should neverpass silently, unless explicitly silenced. It looks to me like (+=) in this case is not being handled correctly under the covers. At any rate the three scenarios of trying to change a mutable list as an immutable tupleitem should be handled consistently. Either we can change the tuple item (if its a list) or we can't. Also, if we cannot then the expected error should be consistent.

Wow. I think I got that whole thing out without one elipses.

Cheers

 

[Eric Jacoboni]

Le 01/03/2014 22:21, Mark H. Harris a écrit :

The point I'm trying to make with this post is that s[2]+=[46] and
s[2]=s[2]+[46] are handled inconsistently.

For my own, the fact that, in Python, a_liste += e_elt gives a different
result than a_list = a_list + e_elt is a big source of trouble...
I don't really find a good reason to justify that : a_list += "spam"
gives a valid result, when a_list = a_list + "spam" is not allowed. Why
the former is like a_list.extend() in the first place ? (well, it's not
exactly an extend() as extend returns None, not +=).

And if both operations were equivalent, i think my initial question
about tuples will be clear and the behavio(u)r would me more consistent
for me :

tu = (10, [10, 30])
tu[1] = tu[1] + [20] <-- Big mistake!
TypeError: 'tuple' object does not support item assignment <-- ok
tu
(10, [10, 30]) <-- ok too...

In fact, i think i'm gonna forget += on lists

 

[Mark H. Harris]

hi Ian, I thought of something else after I slept on it, so to speak.

Consider this:

s=(2,3,[42,43,44],7)
s[2] [42, 43, 44]

s[2] is a list. We should be able to change s[2] even though its within a tuple,
like this:

[42, 43, 44]

s[2].append(45)
s[2] [42, 43, 44, 45] <===== no error

or like this:

s[2] [42, 43, 44, 45]
s[2]=s[2]+[46]

Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
s[2]=s[2]+[46] <==========error, but look below======
TypeError: 'tuple' object does not support item assignment

s[2] [42, 43, 44, 45] <=========also no change to list !!

The point I'm trying to make with this post is that s[2]+=[46]
and s[2]=s[2]+[46] are handled inconsistently. In each case
(whether the list is part of a tuple or not) s[2] is a valid
list and should be mutable. In every case, in fact, it is.
In each case, in fact, the change might occur. But, the change
occurs in the first case (append) without an error. The change
occurs in the next case too (+=) but with an error not related
to the tuple. The change does not occur in the third case
(s=s+ with an error) but the list is not changed !

We all know this is not good. We also all know that its because
of implementation details the user/coder should not need to know.

QED: a bug.

Having slept on it, I am remembering my Zen of Python: Errors
should never pass silently, unless explicitly silenced. It looks
to me like (+=) in this case is not being handled correctly
under the covers. At any rate the three scenarios of trying to
change a mutable list as an immutable tuple item should be handled
consistently.
Either we can change the tuple item (if its a list) or we can't.
Also, if we cannot then the expected error should
be consistent.

Cheers


Hi , I'm experimenting a bit with TextWrangler and gg to see if I can
at least solve my double spacing and wrap problems with cut
a paste for the moment.

Looks ok?

Thanks

 

[Mark H. Harris]

In fact, i think i'm gonna forget += on lists

hi Eric, well, that might be extreme, but you certainly want to avoid
trying to change an immutable. Something you might want to consider
is trying something like creating a new immutable. In the example we've
been looking at I think I would change the tuple list item with an append();
however, you might try building a new tuple instead of trying to change
an existing tuple item.
I must admit that I would never have found this bug myself, because I don't
try to change tuples (at all). Anyway, I'm glad you found it -- we all learn
and that's the good news.

Cheers

 

[Ian Kelly]

In fact, i think i'm gonna forget += on lists

Well, do what you want, but I think you're taking the wrong lesson
from this. Don't forget about using += on lists. Instead, forget
about using assignments, augmented or otherwise, on tuple elements.
Would you expect this to work?

tup = (1, 2, 3)
tup[1] += 42

I'm guessing probably not. So I have a hard time understanding why it
should be expected to work any differently when the tuple element
happens to be a list. The fact that the list is actually modified I
think is a distraction from the main point: the error indicates that
something illegal happened, not that nothing has changed. Assignments
just are not allowed on tuple elements, and it doesn't matter what
type they happen to be.

 

[Eric Jacoboni]

Le 02/03/2014 13:32, Ian Kelly a écrit :

Well, do what you want, but I think you're taking the wrong lesson
from this. Don't forget about using += on lists. Instead, forget
about using assignments, augmented or otherwise, on tuple elements.
Would you expect this to work?

Well, the thing about += on lists that makes me forget it, like i said
in my previous post, is that its behaviour is not consistent with +.

Don't get me wrong: i don't expect that modifying a tuple element works.
That's exactly my point: my initial question was, why it "half works :
it should not work at all". I was thinking that += on lists worked like
update() or extend() : modifying lists in place... It was my mistake

So, yes, i still don't get the point using a += operation, which is not
even consistent with the + operation (see my exemple on "spam" in my
previous post). The + operator to return the modified list and the
update() or extend() methods to do in place replacements are well enough
for my present needs. Maybe, in the future, i will find a use case of +=
for lists which is not covered by others methods, though... Who knows? I
don't doubt that Python designers have made this choice and this
behavior for a good reason: i've just not get it yet.

 

[Mark Lawrence]

Le 02/03/2014 13:32, Ian Kelly a écrit :

Well, the thing about += on lists that makes me forget it, like i said
in my previous post, is that its behaviour is not consistent with +.

The behaviour is consistent except when you try to modify a tuple.

Don't get me wrong: i don't expect that modifying a tuple element works.
That's exactly my point: my initial question was, why it "half works :
it should not work at all". I was thinking that += on lists worked like
update() or extend() : modifying lists in place... It was my mistake

I'd like to see you get update to work on a list. This shows how
confused you are over this issue.

So, yes, i still don't get the point using a += operation, which is not
even consistent with the + operation (see my exemple on "spam" in my
previous post). The + operator to return the modified list and the
update() or extend() methods to do in place replacements are well enough
for my present needs. Maybe, in the future, i will find a use case of +=
for lists which is not covered by others methods, though... Who knows? I
don't doubt that Python designers have made this choice and this
behavior for a good reason: i've just not get it yet.

All of this comes about because of the known issue with tuples. Take
them out of the equation and the behaviour of =, +=, extend and append
is consistent.

 

[Eric Jacoboni]

Le 02/03/2014 15:05, Mark Lawrence a écrit :

The behaviour is consistent except when you try to modify a tuple.

Not in my opinion...

li = [10, 30]
li = li + "spam" --> TypeError: can only concatenate list (not "str")
li += "spam" --> Ok

So, not, that's not what i call consistent.

And it's not related to tuples...

I'd like to see you get update to work on a list. This shows how
confused you are over this issue.

Well, sorry, i wanted to write .append()

 

[albert visser]

Le 02/03/2014 15:05, Mark Lawrence a écrit :

The behaviour is consistent except when you try to modify a tuple.

Not in my opinion...

li = [10, 30]
li = li + "spam" --> TypeError: can only concatenate list (not "str")
li += "spam" --> Ok

possibly because you expect += to take "spam" as a string, but have you
looked at the result?

In [1]: mylist = ['1', '2']

In [2]: mylist += 'spam'

In [3]: mylist
Out[3]: ['1', '2', 's', 'p', 'a', 'm']

consequently, try adding something that can not be interpreted as a
sequence:

In [4]: mylist += 3
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-4-782b544a29d1> in <module>()
----> 1 mylist += 3

TypeError: 'int' object is not iterable


--
Vriendelijke groeten / Kind regards,

Albert Visser

Using Opera's mail client: http://www.opera.com/mail/

 

[Chris Angelico]

Is there any reason why tuples need to throw an exception on assigning to
the element if the old value and new value are the same object?

It'd be easy enough to implement your own tuple subclass that behaves
that way. Try it! See how many situations it actually helps.

ChrisA

 

[Peter Otten]

It'd be easy enough to implement your own tuple subclass that behaves
that way. Try it! See how many situations it actually helps.

.... def __setitem__(self, index, value):
.... if value is not self[index]:
.... raise TypeError("{} is not {}".format(value,
self[index]))
........ T()[0] = k
....
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "<stdin>", line 4, in __setitem__
TypeError: 257 is not 257

I'm not sure "help" is the right word here

 

[Chris Angelico]

TypeError: 257 is not 257

I'm not sure "help" is the right word here

It doesn't help with non-small integers, yes, but the original case
was a list. Personally, I don't think there are many situations that
would benefit from it, plus it'd be confusing ("I can use += with a
list but not a number, why not?!").

ChrisA

 

[Roy Smith]

Is there any reason why tuples need to throw an exception on assigning to
the element if the old value and new value are the same object?

If I say:

a = ("spam", [10, 30], "eggs")

then

a[0] = a[0]

won't actually mutate the object. So tuples could let that silently pass.

But, why would you want them to? What a way to introduce bugs which are
difficult to test for.

 

[Ian Kelly]

I would think it would be better if the exception was thrown before the
assignment to the list took place
simply seeing that a modification action was being applied to a tupple
should be enough.
this would alert the programmer to the fact that he was trying something
that may have undesired consequences

Then the behavior of tuples would be inconsistent with other immutable
types. This can't be applied generally, because the Python
interpreter doesn't generally know whether a given type is supposed to
be immutable or not.