Reference or Value?

T

Torsten Mohr

Hi,

how is the rule in Python, if i pass objects to a function, when is this
done by reference and when is it by value?

def f1(a):
a = 7

b = 3
f1(b)
print b
=> 3

Integers are obviously passed by value, lists and dicts by reference.

Is there a general rule? Some common formulation?


Thanks for any hints,
Torsten.
 
M

MRAB

Torsten said:
Hi,

how is the rule in Python, if i pass objects to a function, when is this
done by reference and when is it by value?

def f1(a):
a = 7

b = 3
f1(b)
print b
=> 3

Integers are obviously passed by value, lists and dicts by reference.

Is there a general rule? Some common formulation?
They are all passed the same way:

def f2(a):
a = [7]

b = [3]
f2(b)
print b
=> [3]

It's just that lists and dicts are containers whose contents you can
change (they are mutable), but integers aren't containers (they are
immutable).

Tuples are also containers, but you can't change their contents (they
are immutable).
 
A

Andrew Koenig

as far as i understand things, the best model is:

1 - everything is an object
2 - everything is passed by reference
3 - some objects are immutable
4 - some (immutable?) objects are cached/reused by the system

0 - Assignment rebinds the reference on its left-hand side; it does not
change the object to which that reference refers.

Example:

x = 42
y = x

Now x and y are bound to the same object

x = x + 1

This statement computes the value of x + 1, which is a new object with value
43. It then rebinds x to refer to this object, so x and y now refer to
different objects. Therefore:

def f(a):
a = a + 1
x = 42
f(x)

This example behaves analogously to the previous one: The assignment a = a +
1 binds a to a new object, so it does not affect the object to which x is
bound.

z = [3]
y = z
z[0] = z[0] + 1

The assignment rebinds z[0] to refer to a new object that has the value 4.
This rebinding does not affect the object formerly bound to z[0]. It does,
however, affect the value of the object to which z is bound, because it
changes the value of its list element. By analogy:

def g(b):
b[0] = b[0] + 1
w = [42]
g(w)

Now w[0] will be 43.
 
S

Steven D'Aprano

Hi,

how is the rule in Python, if i pass objects to a function, when is this
done by reference and when is it by value?

Never, and never.

Integers are obviously passed by value, lists and dicts by reference.

Your error is assuming that pass-by-value and pass-by-reference are the
only two possibilities, consequently you are misinterpreting what you see.

Python uses the same evaluation strategy regardless of the type of object
being passed. Sometimes that strategy looks something like pass-by-value,
and sometimes it looks something like pass-by-reference, but in fact it
always behaves consistently no matter what object you use.

Is there a general rule? Some common formulation?


http://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_sharing
http://effbot.org/zone/call-by-object.htm
 
S

Steven D'Aprano

as far as i understand things, the best model is:

1 - everything is an object
2 - everything is passed by reference

Except that is wrong. If it were true, you could do this:

def swap(x, y):
y, x = x, y

a = 1
b = 2
swap(a, b)
assert a == 2 and b == 1


but you can't, it does not work. Ergo, parameter passing in Python does
not have the same semantics as languages that use pass-by-reference, such
as Pascal and Basic. That means that even if you can justify the claim
"Python is pass-by-reference" by some technical argument (and I don't
believe you can), it is misleading to make that claim without further
qualifications.

3 - some objects are immutable

All objects are passed the same way, regardless of whether they are
mutable or immutable.

4 - some (immutable?) objects are cached/reused by the system

That's irrelevant. Caching affects the creation of new objects, but has
no effect on argument passing.
 
D

Denis Kasak

Except that is wrong. If it were true, you could do this:

def swap(x, y):
y, x = x, y

a = 1
b = 2
swap(a, b)
assert a == 2 and b == 1


but you can't, it does not work. Ergo, parameter passing in Python does
not have the same semantics as languages that use pass-by-reference, such
as Pascal and Basic. That means that even if you can justify the claim
"Python is pass-by-reference" by some technical argument (and I don't
believe you can), it is misleading to make that claim without further
qualifications.

You could, however, argue that the swap function doesn't work as
expected (e.g. from a Pascal or a C++ POV) simply because the
underlying objects aren't mutable. The objects *do* get passed by
reference; the function doesn't receive a new copy of the object and
it can examine the original object's ID. The actual culprit is not the
way objects are passed but the assignment operator, since it works by
rebinding names (as Andrew Koenig explained) and not by changing the
object itself. If the swap() function could somehow access the
underlying integer object and modify it, swapping of values would
indeed occur because the function *did* get references to the objects
passed to it.

That said, it's a rather convoluted way of explaining what happens and
calling it pass-by-object feels much better. :)
 
A

afriere

Hi,

how is the rule in Python, if i pass objects to a function, when is this
done by reference and when is it by value?

def f1(a):
    a = 7

b = 3
f1(b)
print b
=> 3

Integers are obviously passed by value, lists and dicts by reference.

Is there a general rule?  Some common formulation?

Thanks for any hints,
Torsten.

Let's add some comments and a little code
print a, id(a)
#print the object passed and it's id
a = 7
#reassign 'a' to point to a different object
print a, id(a)
#print the new object and it's id
1 161120104
7 161120032
#no surprises there, the new object has a different id.
id(b) 161120104
#no surprises ther, b (unlike the name internal to the function
#hasn't been reassigned, so it still points where it always has
#which is of course ..
b 1
z = []
#name 'z' refers to an empty list
id(z) 162432780
f1(z)
[] 162432780
7 161120032
 
G

Gabriel Genellina

You could, however, argue that the swap function doesn't work as
expected (e.g. from a Pascal or a C++ POV) simply because the
underlying objects aren't mutable.

That's irrelevant - mutable and immutable objects are passed exactly the
same way.
The objects *do* get passed by
reference; the function doesn't receive a new copy of the object and
it can examine the original object's ID. The actual culprit is not the
way objects are passed but the assignment operator, since it works by
rebinding names (as Andrew Koenig explained) and not by changing the
object itself.

There is *no* difference between "the way objects are passed" and "the
assignment opera[tion]": both work exactly the same way.

Python execution model is based on namespaces (a collection of
name->object pairs, usually implemented as dictionaries). An assignment x
= y means:

- evaluate the right hand side and obtain a value (an object, always). In
this case, it is the object referenced by the name "y"
- make the name "x" refer to such object. That's all.

In short, assignment rebinds a name in a namespace.
When a function call is made, a new namespace is created. The names come
from the function parameters (the names that were used to define the
function in the def ... line) and the objects come from the actual
arguments (the objects you pass inside the (...) when doing the call). The
function code is then executed using this namespace as its locals().

def foo(a, b):
print a, b

foo(4, ['hello', 'world!'])

The call is equivalent to:
ns = {}
ns['a'], ns['b'] = 4, ['hello', 'world!']
execute foo code using ns as locals

except the right hand side is evaluated on the *calling* namespace, and
the left hand assignments are done on the new namespace. So a call binds
many names in a new namespace: the same as assignment.

No copies, no pointers, nothing: only names and objects are manipulated.
If the swap() function could somehow access the
underlying integer object and modify it, swapping of values would
indeed occur because the function *did* get references to the objects
passed to it.

There is a difference between modify the value of an object, and modify
the caller's namespace. Usually "pass by reference" means that the
*caller* may see a different thing after the call - this is not true in
Python, there is no way the called code could alter the caller namespace
(well, not just due to the call operation alone...)
That said, it's a rather convoluted way of explaining what happens and
calling it pass-by-object feels much better. :)

And is more correct...
 
S

Steve Holden

Denis said:
You could, however, argue that the swap function doesn't work as
expected (e.g. from a Pascal or a C++ POV) simply because the
underlying objects aren't mutable. The objects *do* get passed by
reference; the function doesn't receive a new copy of the object and
it can examine the original object's ID. The actual culprit is not the
way objects are passed but the assignment operator, since it works by
rebinding names (as Andrew Koenig explained) and not by changing the
object itself. If the swap() function could somehow access the
underlying integer object and modify it, swapping of values would
indeed occur because the function *did* get references to the objects
passed to it.

That said, it's a rather convoluted way of explaining what happens and
calling it pass-by-object feels much better. :)
The underlying point is that there is no way to rebind names in the
caller's namespace by manipulations of parameters passed as arguments to
a function.

regards
Steve
 
S

Steve Holden

Denis said:
I assure you I am not confused about Python's object model / calling
system. I was arguing, from a purely theoretical standpoint, that the
same system Python uses could be described in terms of
call-by-reference with some additional constraints. I am by no means
arguing that this is a good way of explaining it or trying to explain
it to someone in terms of call-by-reference. I just don't think it's
"plain simple wrong", just confusing and suboptimal.
Well, what's the benefit of discussing such a description, then?
Regulars on c.l.py see this subject arising so regularly it's a bit like
"there's nothing new under the sun". Honestly, it's been discussed /ad
nauseam/ lately.

regards
Steve
 
D

Denis Kasak

Well, what's the benefit of discussing such a description, then?
Regulars on c.l.py see this subject arising so regularly it's a bit like
"there's nothing new under the sun". Honestly, it's been discussed /ad
nauseam/ lately.

Someone said upthread that Andrew Cooke was completely wrong and I
contested that. :)
I apologise for not being current with the latest discussions.
 
S

Steve Holden

Denis said:
Someone said upthread that Andrew Cooke was completely wrong and I
contested that. :)
I apologise for not being current with the latest discussions.
That's OK. If you haven't seen this stuff before it's new to you, I guess.

regards
Steve
 
G

Gabriel Genellina

I don't think they were saying that mutable and immutable objects
were handled differently. They are different in terms of what a
function can do to them (mutate them or no). Thus, if you can't
mutate them, then the caller can't see any changes in the objects by
actions in the function so the swap won't work: it doesn't mutate the
objects at all. In the case of a list or a dict, then one can mutate
them, and the changes are seen in the caller. In both cases, the
object itself is passed the same way.

Ok, in any case, making such claims without explanation or restricting the
sense, only confuses the matter (which is innecesarily confusing
already...)
 
S

Steven D'Aprano

Steven said:
Except that is wrong. If it were true, you could do this:
[pointer swapping]

i was thinking of how the stack is used; i would have called what you
are talking about "pointer semantics". however, on reading around a
little, it seems that i'm in a very small minority (which is a pity,
because if you restrict the meaning to how values are handled on the
stack then you get a lot closer to having just values and references,
rather than the whole pile of different terms that are apparently in
use).

sorry for the confusion,

Why is it a pity to have a whole pile of different terms to describe a
whole lot of different behaviours? I would suggest the real confusion
would be if we had two terms to describe a dozen different parameter-
passing conventions.
 
G

Gabriel Genellina

Steven said:
Steven D'Aprano wrote:
On Sun, 22 Feb 2009 13:37:27 -0300, andrew cooke wrote:

as far as i understand things, the best model is:

1 - everything is an object
2 - everything is passed by reference

Except that is wrong. If it were true, you could do this:
[pointer swapping]

i was thinking of how the stack is used; i would have called what you
are talking about "pointer semantics". however, on reading around a
little, it seems that i'm in a very small minority (which is a pity,
because if you restrict the meaning to how values are handled on the
stack then you get a lot closer to having just values and references,
rather than the whole pile of different terms that are apparently in
use).

sorry for the confusion,

Why is it a pity to have a whole pile of different terms to describe a
whole lot of different behaviours? I would suggest the real confusion
would be if we had two terms to describe a dozen different parameter-
passing conventions.

because:

(1) it's often useful to explain things as (simpler) orthogonal
components
rather than globbing everything under one term; that lets you more easily
carry across knowledge from previous experience.

(2) while using a term that has (according to wikipedia) "widespread
usage
in the Python community" helps you save time and avoid confusion, it
doesn't necessarily explain best to someone from outside that community
what is happening. in this particular case you are getting dangerously
close to having a term that is used only for one language and which,
therefore, is at risk of meaning little more that "the way python does
it"
(which is, i think you'll agree, more tautological than helpful).

But -paraphrasing A.E.- we should explain it in the simplest terms, but
not simpler.

Some people think that there exist only two possible ways to pass an
argument: by value or by reference. The way Python does it doesn't have
the same properties that pass-by-value nor pass-by-reference have in other
languages, so both names are inapropiate. You appear to favor the "by
reference" name. A couple months ago some other guy were strongly
defending the "by value" name -- he even set up a website supporting his
idea.

The fact is, if you say "it's by value" you'll disappoint some
expectations from some users, and if you say "it's by reference" you'll
disappoint some other expectations. So you must say something different
("it's by reference, but the references cannot be modified (!), and...",
"it's by value, but the values are references (!!), and...", or whatever
description you choose). Given that, it's better to use a different name:
you're going to explain the concept anyway.

And Python isn't the first language using this convention; if you look in
the FAQ or search recent threads in this group you'll find references
going back to the '70s.
 
S

Steven D'Aprano

andrew said:
Steven said:
Steven D'Aprano wrote:
On Sun, 22 Feb 2009 13:37:27 -0300, andrew cooke wrote:

as far as i understand things, the best model is:

1 - everything is an object
2 - everything is passed by reference

Except that is wrong. If it were true, you could do this:
[pointer swapping]

i was thinking of how the stack is used; i would have called what you
are talking about "pointer semantics". however, on reading around a
little, it seems that i'm in a very small minority (which is a pity,
because if you restrict the meaning to how values are handled on the
stack then you get a lot closer to having just values and references,
rather than the whole pile of different terms that are apparently in
use).

sorry for the confusion,

Why is it a pity to have a whole pile of different terms to describe a
whole lot of different behaviours? I would suggest the real confusion
would be if we had two terms to describe a dozen different parameter-
passing conventions.

because:

(1) it's often useful to explain things as (simpler) orthogonal components
rather than globbing everything under one term;

"One term"?

If you look at this Wikipedia page:

http://en.wikipedia.org/wiki/Evaluation_strategy

you should be able to count no fewer than thirteen different terms for
different evaluation strategies, and "call-by-value" itself is described as
a family of strategies. Who is gobbing everything under one term? Certainly
not me. If anything, your preferred tactic is to gob everything under one
of two terms, call-by-value and call-by-reference. That's not my preferred
tactic.

that lets you more easily
carry across knowledge from previous experience.

That's only useful if that previous experience *clarifies* rather than
*confuses*. This very thread demonstrates that people's prior knowledge of
call-by-whatever doesn't clarify Python's behaviour.

(2) while using a term that has (according to wikipedia) "widespread usage
in the Python community" helps you save time and avoid confusion, it
doesn't necessarily explain best to someone from outside that community
what is happening.

And you think call-by-reference does?

Python demonstrably does not behave like call-by-reference in other
languages. I do not agree that it is wise to hammer the round peg of
Python's actual behaviour into the square peg of newbies' concept of
call-by-reference.

in this particular case you are getting dangerously
close to having a term that is used only for one language

So what? What if Python was the only language that used the term? That
doesn't make it inappropriate. If the users of other languages that use
call-by-object semantics (Java, RealBasic, etc) choose to prefer a
misleading name over a descriptive name, that's no reason for Python users
to follow down their path.

and which,
therefore, is at risk of meaning little more that "the way python does it"
(which is, i think you'll agree, more tautological than helpful).

No at all. Do you imagine that call-by-name means "the way Algol does it"?
Algol is the only major language that uses call-by-name, it may even be the
only language of any size, but that doesn't mean that call-by-name can't be
defined and described independently of Algol. There's even a test,
Knuth's "Man Or Boy Test", for testing how well a language implements
call-by-name semantics.
 
T

Tim Rowe

2009/2/24 Steven D'Aprano said:
If you look at this Wikipedia page:

http://en.wikipedia.org/wiki/Evaluation_strategy

you should be able to count no fewer than thirteen different terms for
different evaluation strategies, and "call-by-value" itself is described as
a family of strategies.

And that call by sharing is a member of that family even though the
behaviour is (as I understand it) exactly what you would get if you
pass an object by ^reference^ in C++. Wikipedia can be a double-edged
sword ;-).

I think the problem is that "Call by value" and "Call by reference"
are terms that pre-date object orientation, and don't cope well with
obect-oriented languages in general, not just with Python. That Java
programmers get by thinking the mechanism is called "call by value",
C++ programmers get by thinking the same mechanism is called "call by
reference" and Python programmers get by thinking the same mechanism
is called "call by sharing" suggests that the terms are no longer
helpful. Programmers in one language simply can't know what the
terminology means when applied to another language without learning
the behaviour in that other language before learning the terminology.

That means that Torsten's original question probably wasn't the one he
wanted to ask, and wasn't a useful one. The useful question is "what
is Python's parameter passing *behaviour^",
with a possible supplementary "what do they call that behaviour *in
the Python community*", recognising that if they asked "what is Java's
parameter passing *behaviour^",
with the supplementary "what do they call that behaviour *in the Java
community*" they would get much the same answer for the first question
(immutability is handled differently, I think, but it's there) but a
totally different answer to the second.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,209
Messages
2,571,088
Members
47,687
Latest member
IngridXxj

Latest Threads

Top