Bug or feature?

[Alexey Nezhdanov]

Hello. Found a strange python behaivoir while writing some object. I've
stripped out all other stuff so the problem is stright here:
=================================
#!/usr/bin/python

class Object:
def __init__(self,val):
self.val=val

def Sub(self,delta):
self.val-=delta
return delta

c=Object(1)

### case 1 ###
d=c.Sub(1)
c.val+=d
print 'case A:',c.val

### case 2 ###
c.val+=c.Sub(1)
print 'case B:',c.val
=================================
Case 1 and case 2 prints out different calues (1 and 2 respectively).
Do not know - if this is a bug or feature of python.

 

[James Henderson]

Hello. Found a strange python behaivoir while writing some object. I've
stripped out all other stuff so the problem is stright here:
=================================
#!/usr/bin/python

class Object:
def __init__(self,val):
self.val=val

def Sub(self,delta):
self.val-=delta
return delta

c=Object(1)

### case 1 ###
d=c.Sub(1)
c.val+=d
print 'case A:',c.val

### case 2 ###
c.val+=c.Sub(1)
print 'case B:',c.val
=================================
Case 1 and case 2 prints out different calues (1 and 2 respectively).
Do not know - if this is a bug or feature of python.

This is the expected behaviour.

The terms on the RHS (c.val and c.Sub(1) in case 2) are evaluated left to
right before being added together. If you'd reversed the terms of case 2 and
written:

c.val = c.Sub(1) + c.val

c.val would end up as 1.

James

 

[Dennis Lee Bieber]

Alexey Nezhdanov fed this fish to the penguins on Wednesday 14 January
2004 21:40 pm:

Case 1 and case 2 prints out different calues (1 and 2 respectively).
Do not know - if this is a bug or feature of python.

It's a problem with using what I would consider a side effect... And
not understanding that "variables" are not memory addresses.

Observe -- I've explicitly made all "var" accesses methods of the
class, which means the equations have to be written in long form:

class O:
def __init__(self, ival):
self._val = ival
print "__init__", self._val

def Sub(self, delta):
self._val -= delta
print "Sub", self._val
return delta

def GetVal(self):
print "GetVal", self._val
return self._val

def SetVal(self, nval):
self._val = nval
print "SetVal", self._val

c = O(1)

print c.GetVal()
print

d = c.Sub(1)
c.SetVal(c.GetVal() + d)

print c.GetVal()
print

c.SetVal(c.GetVal() + c.Sub(1))

print c.GetVal()
print

c.SetVal(c.Sub(1) + c.GetVal())

print c.GetVal()
print

And here is the result of the run:

[wulfraed@beastie wulfraed]$ python t.py
__init__ 1
GetVal 1
1

Sub 0
GetVal 0
SetVal 1
GetVal 1
1

GetVal 1
Sub 0
SetVal 2
GetVal 2
2

Sub 1
GetVal 1
SetVal 2
GetVal 2
2

Note how the order of the Sub() and GetVal() calls affects the result.

--

 

[Terry Reedy]

Alexey Nezhdanov fed this fish to the penguins on Wednesday 14 January
2004 21:40 pm:

It's a problem with using what I would consider a side effect... And
not understanding that "variables" are not memory addresses.

In particular, the problem with both modifying in place (the side effect)
and returning a value. I see a little better Guido's reason for having
list modification methods return None. While chaining is not too
problematical, object returns would allow list expressions mixing implicit
and overt effects with possible puzzles similar to the one presented in
this thread.

Terry J. Reedy

 

[Dennis Lee Bieber]

Terry Reedy fed this fish to the penguins on Thursday 15 January 2004
15:39 pm:

and returning a value. I see a little better Guido's reason for
having
list modification methods return None. While chaining is not too
problematical, object returns would allow list expressions mixing
implicit and overt effects with possible puzzles similar to the one
presented in this thread.

Now that you mention it... Yeah... Can you imagine trying to figure
out what something like

alist = ['j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a']
res = alist[:5] + alist.sort() + alist[5:]

was supposed to return?

jihgfabcdefghijedcba
or
jihgfabcdefghijabcde
or
abcdeabcdefghijabcde
or
abcdeabcdefghijfghij

The only "known" is that the middle is supposed to be abcdefghij, but
the left and right ends depend solely on order of evaluation...

--

 

[Alexey Nezhdanov]

In particular, the problem with both modifying in place (the side effect)
and returning a value. I see a little better Guido's reason for having
list modification methods return None. While chaining is not too
problematical, object returns would allow list expressions mixing implicit
and overt effects with possible puzzles similar to the one presented in
this thread.

Terry J. Reedy

So since the word "problem" is so popular in this tred I have an another
question:
Does anybody agree that this is a python problem and not a programmer
problem?

Let me explain.
I DO know that this behaivoir of python is matches the DOCS and all
declared official language laws.
I DO know why it is happening and how to fix my program to solve the
problem (in fact I already knowed it when written my first post here).
I DO know that fixing on python level it will rise particular
incompartibility with previous versions of python.
BUT
I know the case where such python behaivoir causes hard-to-catch problems.
I do not know any case (anybody knows? please give example) where such
"feature" may be useful.

So I proposing to make a change into python and reverse the order of
calculations.
Just for sure:
=== Order used now ===
## a+=a.Sub(b)
1. evaluate value of 'a'
2. evaluate value of 'a.Sub(b)'
3. evaluate the sum
4. store the result into 'a'
=== Proposed order ===
## a+=a.Sub(b)
1. evaluate value of 'a.Sub(b)'
2. evaluate value of 'a'
3. evaluate the sum
4. store the result into 'a'

(same about -=, *=,... of course)

 

[Jp Calderone]

Terry Reedy fed this fish to the penguins on Thursday 15 January 2004
15:39 pm:

and returning a value. I see a little better Guido's reason for having
list modification methods return None. While chaining is not too
problematical, object returns would allow list expressions mixing
implicit and overt effects with possible puzzles similar to the one
presented in this thread.

Now that you mention it... Yeah... Can you imagine trying to figure
out what something like

alist = ['j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a']
res = alist[:5] + alist.sort() + alist[5:]

was supposed to return?

jihgfabcdefghijedcba
or
jihgfabcdefghijabcde
or
abcdeabcdefghijabcde
or
abcdeabcdefghijfghij

The only "known" is that the middle is supposed to be abcdefghij, but
the left and right ends depend solely on order of evaluation...

http://python.org/doc/current/ref/evalorder.html

Doesn't seem to be a problem, after all.

Jp

 

[Skip Montanaro]

>> Now that you mention it... Yeah... Can you imagine trying to figure
>> out what something like
>>
>> alist = ['j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a']
>> res = alist[:5] + alist.sort() + alist[5:]
>>
>> was supposed to return?

...

That's easy. You get a TypeError exception, because alist.sort() returns
None.

Like Jp said:

Jp> Doesn't seem to be a problem, after all.

Skip

 

[James Henderson]

On Friday 16 January 2004 2:44 pm, Alexey Nezhdanov wrote:

I know the case where such python behaivoir causes hard-to-catch problems.
I do not know any case (anybody knows? please give example) where such
"feature" may be useful.

So I proposing to make a change into python and reverse the order of
calculations.
Just for sure:
=== Order used now ===
## a+=a.Sub(b)
1. evaluate value of 'a'
2. evaluate value of 'a.Sub(b)'
3. evaluate the sum
4. store the result into 'a'
=== Proposed order ===
## a+=a.Sub(b)
1. evaluate value of 'a.Sub(b)'
2. evaluate value of 'a'
3. evaluate the sum
4. store the result into 'a'

-1
In any language that uses such an operator I would expect:

a += b

to be synonymous with:

a = a + b

You are suggesting that it should mean:

a = b + a

It is not like Python to attach its own idiosyncratic semantics to operators
already well known from other languages. I think this would catch a lot of
people out.

Of course, it's probably best to avoid writing code where the order makes a
difference.

James

 

[Dennis Lee Bieber]

Skip Montanaro fed this fish to the penguins on Friday 16 January 2004
07:44 am:

That's easy. You get a TypeError exception, because alist.sort()
returns None.

That is the behavior in the real Python... The example was meant to
apply to a hypothetical version where operations like [].sort()
modified in place /and/ returned the modified list for chained
operations.

The example was meant to illustrate the perils of side-effects in
functions. A casual newcomer to programming could be perplexed by the
left-to-right evaluation if they were expecting to just have a sorted
copy embedded in the middle, as the side-effect would have changed the
value of the right-end expression.

--

 

[Dennis Lee Bieber]

Alexey Nezhdanov fed this fish to the penguins on Friday 16 January
2004 06:44 am:

So I proposing to make a change into python and reverse the order of
calculations.
Just for sure:
=== Order used now ===
## a+=a.Sub(b)
1. evaluate value of 'a'
2. evaluate value of 'a.Sub(b)'
3. evaluate the sumf
4. store the result into 'a'
=== Proposed order ===
## a+=a.Sub(b)
1. evaluate value of 'a.Sub(b)'
2. evaluate value of 'a'
3. evaluate the sum
4. store the result into 'a'

(same about -=, *=,... of course)

But, since Python is dynamically typed, reversing the evaluation order
could have major effects on other operations... The "problem" is not
the order of evaluation, but that you have a "hidden" modification to a
term occurring.
.... a.insert(0,3)
.... return a
........ a = [3] + list(a)
.... return a
....

l = [1,5,7]
l += nose(l)
l [1, 5, 7, 3, 1, 5, 7]
l = [1,5,7]
l += se(l)
l [3, 1, 5, 7, 3, 1, 5, 7]
l = [1,5,7]

A very long time ago, some FORTRAN compilers had "non-constant
constants"...

subroutine Modify(a, b)
a = a + b
return
end

program demo
call Modify(1, 3)
print 1
stop
end

On these machines, calling Modify would result in the "constant" 1
taking the value of /4/, and the print statement would print 4.
Nowaday's it is normal to place all such constants in a read-only block
of memory, generating a trap when a modification is attempted. Other
languages have explicit usage qualifications allowing a compiler to
determine validity.

None of which can be done in Python if one passes in a mutable object.

--

 

[Jp Calderone]

On Friday 16 January 2004 2:44 pm, Alexey Nezhdanov wrote:


-1
In any language that uses such an operator I would expect:

a += b

to be synonymous with:

a = a + b

Of course, it should be noted that, in Python, "a += b" is only sometimes
synonymous with "a = a + b". The rest of the time, it's hard to say what it
is synonymous with

You are suggesting that it should mean:

a = b + a

It is not like Python to attach its own idiosyncratic semantics to operators
already well known from other languages. I think this would catch a lot of
people out.

Definitely. Consider tuple += tuple. It would not only be idiosyncratic,
but outright surprising and irrational if using += on (1, 2) and (3, 4)
resulted in (3, 4, 1, 2).

Of course, it's probably best to avoid writing code where the order makes a
difference.

Amen. Write simple code and all those who follow in your footsteps will
thank you.

Jp

 

[Terry Reedy]

http://python.org/doc/current/ref/evalorder.html
Doesn't seem to be a problem, after all.

Not for you, but how about newbie and others who don't read or ignore,
misunderstand, or reject what they read? Lets try this example based on
OP's real examples. Again, hypothetical, assuming list mutators were to
return list.

l3 = range(3)
l3+l3.reverse()
# I believe that this wouldbe [1,2,3,3,2,1]
l3 = range(3)
tem = l3.reverse()
l3 + tem
# I believe that this would instead be [3,2,1,3,2,1]

I am sure that were such things legal, there would be numerous posts
complaining about 'unexpected' outcomes. Hence my comment about their not
being legal being good.

OP believe that getting a different answer when inserting a tempory in the
midst of a side-effect dependent expression is such a problem that we
should break code to 'fix' it. I obviously disagree.

Terry J. Reedy

 

[Terry Reedy]

But, since Python is dynamically typed, reversing the evaluation order
could have major effects on other operations... The "problem" is not
the order of evaluation, but that you have a "hidden" modification to a
term occurring.
... a.insert(0,3)
... return a
...... a = [3] + list(a)
... return a
...

l = [1,5,7]
l += nose(l)
l [1, 5, 7, 3, 1, 5, 7]
l = [1,5,7]
l += se(l)
l

[3, 1, 5, 7, 3, 1, 5, 7]

Lovely example, no hypotheticals needed.

For those who did not get it, the key is that Python expressions evaluate
to objects, and that the + operator does not reach into the objects to pull
out values for the new object until it has (references to) both objects,
which in the second case, are both the same.

Terry J. Reedy

 

[James Henderson]

Of course, it should be noted that, in Python, "a += b" is only sometimes
synonymous with "a = a + b". The rest of the time, it's hard to say what
it is synonymous with

What do you have in mind? J.

 

[Skip Montanaro]

James> What do you have in mind? J.

For immutable objects, += works as you'd expect: return a new object,
leaving the old object unchanged. That's not the case for mutable objects,
to wit:

>>> foo = [1]
>>> bar = foo
>>> foo += [2]

The object referenced by foo is modified in-place...

>>> bar [1, 2]
>>> foo = foo + [2]

Here foo is bound to a new object, leaving the old object (still referenced
by bar) unchanged.

>>> foo [1, 2, 2]
>>> bar

[1, 2]

Skip

 

[James Henderson]

James> What do you have in mind? J.

For immutable objects, += works as you'd expect: return a new object,
leaving the old object unchanged. That's not the case for mutable objects,

to wit:

foo = [1]
bar = foo
foo += [2]

The object referenced by foo is modified in-place...

[1, 2]

foo = foo + [2]

Here foo is bound to a new object, leaving the old object (still referenced
by bar) unchanged.

[1, 2, 2]

[1, 2]

Skip

Thanks for pointing that out. It's also the case the += may be more
efficient, although that doesn't undermine my original point about surface
behaviour.

For any third party interested the details are at:

http://www.python.org/doc/current/ref/augassign.html

Back to the original topic, I have to agree with Robert Brewer that Alexey
Nezhdanov's proposal was capable of several other interpretations than a
switch to right-to-left evaluation.

James