Pass-by-reference : Could a C#-like approach work in Python?

[Stephen Horne]

No, you *would* know. The above call would *not* be able to
rebind any of thisArgument or thatArgument.

Thanks for the support.

(For the record, I do not support Stephen's proposal, nor any
other kinds of call-by-reference in Python. For one, I don't
think it meshes well with the rest of Python, and moreover, I
have yet to encounter a real-world example where this would
actually make code more readable - and I have been using Python
for eight years now, and Lisp (which has similar call semantics)
for a few years before that.)

Yes, I do agree with this now - it was a poorly thought out idea which
reveals nothing so much as the fact that I've got too comfortable with
some other languages ways of working.

 

[Stephen Horne]

... and that is the whole point of my example: You can easily bypass the
rebinding in a manner that is hard to find (if fun() and nested() contained
some real code)

Oops - for some reason I read "cause" as "catch". Still, you made your
point even though I didn't realise you intended to make it - and it
was a very compelling point.

Actually, the 'builder' solution to the issue that I mentioned before
isn't as good as I initially thought. Here is how I wrote it...

: builder = Builder ()
:
: builder.Do_Setup_Stuff_1 ()
: builder.Do_Setup_Stuff_2 ()
: builder.Do_Setup_Stuff_3 ()
:
: container.Add (builder.Build ())
:
: builder.Do_Setup_Stuff_3_Slightly_Differently ()
:
: container.Add (builder.Build ())

The problem here is that it would be too easy for the caller to forget
to use the Build method - depending on what the 'container' actually
does, that may mean the builder gets used instead of the object.

Easily fixed, though - the builder and container are most likely part
of the same set of functionality, and therefore written to work
together. So the Add function should take the Builder object as its
parameter and call Build method itself. If it is given anything other
than a builder, the method won't exist and will fail immediately.

 

[Steve Holden]

"Stephen Horne", using a ridiculous bogus email address, wrote...

Obviously, your example could easily be rewritten to

def inc(p):
return p + 1

x = inc(x)

and a similar function with two by-ref parameters would still be readable:

x, y = inc2(x, y)

So here's my question. How many functions are there in your *real* code that
could benefit from, say three or more by-ref parameters?
Hint: for me it comes close to 0

Its not an issue of whether the same effect can be achieved in Python.
It's more an issue of whether the method used expresses the
programmers intentions properly.

If a function is defined as...

def Inc (p) :
return p + 1

then that to me expresses an intention which is different to that
expressed by...

def Inc (ref p) :
p += 1

The latter case expresses the intention to in-place rebinding of the
parameter. In doing so, it better expresses the purpose of some kinds
of functions (though not this noddy example, obviously).

This is all very fuzzy, so let's examine an example which is based on
an issue in some real Python code (though this is simplified to make
the point). The problem is to insert an object into some container -
appending to a list will do here. However, the object being inserted
will typically be quite large, so we want to avoid copying it if
possible.

No problem there, it seems...

class c_Container :
f_List = []

def Add (self, p_Object) :
f_List.append (p_Object)

The problem with this, however, is the risk of accidental
side-effects. If the caller goes on to modify the object that he
passed as a parameter, that will change the content of the container.

I don't see why this is any different, or worse, than having a function
modify a single reference. You are setting your calling code up to fail due
to the invisible side-effects of the function call. This is a Bad Idea.

[...]

And that is the big question.

C# 'needs' ref parameters because it has no convenient way to return
multiple values from a function. Python does not. The question is
really whether this argument about expressing intentions and evading
errors is compelling or not.

You have still to convince me that there would be any gain whatsoever. Think
of me as the forces of reaction ;-)

regards

 

[Steve Holden]

--
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/


First of all, if you insist on using an invalid address, might I suggest
that "(e-mail address removed)" would be less visually annoying, with the additional
merit of actually meaning something.

1. I pointed out that this is an advantage of the C# system, which I
was proposing to imitate.

: The C# solution uses a modifier keyword, but that keyword must be
: specified in both the function definition *and* the call. For
: instance...

...

So, if I can summarise. You see a feature in C# that represents perhaps one
of the less-well-thought-out aspects of the language. You then ask how to
graft it into Python because "Python should be more like C#"?

: The rationale for this in C#, I assume, is mostly about clarity and
: maintainability - the caller can't be surprised when value type
: parameters are changed in a function as he had to explicitly allow it.

I agree the requirement for the "ref" keyword in the call lends some
clarity. Not sure I'd agree about maintainability, though. And it can only
be a matter of time before people start suggesting that the ref mechanism be
used because there's a two-hundred nanosecond time saving ...

2. You quoted the example I labelled as "(roughly) what I'd like to
be able to do" and ignored the one where I adopted the C#
approach...

: >>> def Inc(ref p) :
: ... p += 1
: ...
: >>> x = 1
: >>> Inc(ref x)
: >>> x
: 2

So your criticism is invalid simply because the whole point of my post
was to suggest a scheme where "modifiability should be evident to the
caller".

But given Python's ability to have a function return a tuple to an unpacking
assignment (which has the real advantage that the intention to modify the
value of the left-hand tuple elements is explicit) I really can't see why
you want to complicate things further. "Explicit is better than implicit".

And if in some future version of Python the suggestion I made was
implemented, when you see f(x) you will still know that x cannot be
rebound - but when you see f(ref x) you will know that x may well be
rebound.

OK, remind me why this is better than

x = Inc(x)

or

x, y = DoubleInc(x, y)

I wouldn't want this. To implement it, either a single compiled
version of a function would need to figure out at run time whether the
parameters where by reference or not, or several versions of each
function would have to be compiled into the .pyc file.

Also, I don't really see the benefit. If a function has by-reference
parameters then rebinding those parameters is likely a big part of the
purpose of the function. Supplying a parameter which cannot be rebound
is likely an error caused by someone forgetting to type the 'ref', in
which case they'd equally likely appreciate an error message as
opposed to wierd, hard-to-trace logic errors.

So you introduce a "ref" keyword so that a dangerous and unneccessary
mechanism can be "safely" introduced?

still-not-convinced-ly y'rs - steve

 

[Stephen Horne]

First of all, if you insist on using an invalid address, might I suggest
that "(e-mail address removed)" would be less visually annoying, with the additional
merit of actually meaning something.

I used to have something like that, only my newreader won't allow a
completely invalid addresss (it insists on seeing '.co.uk' and some
other formatting stuff). It turned out that my obviously invalid
domain was actually valid and owned by someone else. Actually, it was
pointed out to me by someone who disagreed with a point I was making.

Basically, even if a spamtrap domain is actually a real domain by pure
fluke, it will still recieve my spam - as said the lecture.

If dollar signs are valid in a domain name (which I assume they are as
my newsreader allows them) then I can imagine a valid domain
consisting of one, two, three or whatever of them - but not a huge run
of them. That is the reason for my choice. By preference I would miss
that line out of the header completely, but it's not an option.

This first comment of yours raises the suspicion that, in disagreeing
with my idea, you feel you have to attack everything you can about me
and about the idea. Let's see how that suspicion pans out...

So, if I can summarise. You see a feature in C# that represents perhaps one
of the less-well-thought-out aspects of the language. You then ask how to
graft it into Python because "Python should be more like C#"?

Oh dear.

In C#, 'ref' parameters are not IMO a "less-well-thought-out" aspect
of the language. They are, IMO, a very well thought out feature in the
context of that language. Making the 'ref' explicit in the call is
also a novel approach which I don't think I've seen in any other
language, which is why I decided to raise the subject.

On thinking it through, it simply isn't necessary in Python - but that
doesn't mean you have to make it out to be a universally bad idea or
go through the old 'stop trying to turn Python into X' garbage.

Actually I'm sick to death of the old 'your just trying to turn Python
into X' argument whenever I suggest that *maybe* Python might learn a
trick from another language. I have been accused of trying to turn
Python into C, C++, Java, Pascal, Ada and several others in the past.
None of these accusations is true. They are tired and pathetic. They
say more about the accuser than the accused.

Believe it or not, languages other than Python have good features.
Believe it or not, Python has become a better language by adopting
features from other languages AS WELL AS by adopting ideas of its own.

Sometimes a feature that works well in one language can work well in
another.

Python does not need a 'not invented here' mentality.

I agree the requirement for the "ref" keyword in the call lends some
clarity. Not sure I'd agree about maintainability, though.

Maintainability is mostly helped by explicitness. The maintainer can
see that the parameter might be rebound simply by looking at the call.
I can't imagine any scenario where improved clarity doesn't lead to
improved maintainability.

And it can only
be a matter of time before people start suggesting that the ref mechanism be
used because there's a two-hundred nanosecond time saving ...

Garbage. Neither Python nor C# are meant to be execution speed record
holders. I did address a performance issue, but only because (1) that
performance issue was raised by the 'against' side in a previous
discussion of pass-by-reference parameters and (2) there is already
some sensitivity to the idea of Python becoming slower than it already
is (IIRC some people still use Python 1.5 in part because it is a
little faster than Python 2.x).

It would be hard to predict the relative speeds of a this idea against
the tuple-returning method. Each has its own overheads - in one case
creating and unpacking the tuple, in the other case checking that the
right parameters are marked as 'ref' or not. All I can say for sure is
that the performance of conventional function calls would not change.

But then my argument was never about performance. It appears to me
that you invented that issue purely so you could counter it - but
countering an argument that the other side never made is a very weak
victory.

But given Python's ability to have a function return a tuple to an unpacking
assignment (which has the real advantage that the intention to modify the
value of the left-hand tuple elements is explicit) I really can't see why
you want to complicate things further.

It is possible for a person to propose an idea for discussion without
having though it through fully - that may even be the point of
proposing it for discussion. If discussion could never lead to new
ideas and changed opinions there'd really be no point.

In the context of Python the idea turns out to serve no compelling
purpose. I already admitted that in message ID
(e-mail address removed). I was even showing signs
of heading that way when I said...

: C# 'needs' ref parameters because it has no convenient way to return
: multiple values from a function. Python does not. The question is
: really whether this argument about expressing intentions and evading
: errors is compelling or not.

all the way back in message ID
(e-mail address removed)

You don't have to claim that the feature is "one of the
less-well-thought-out aspects of" C# to do that - C# is not Python.
You also don't need to accuse me of trying to "graft it into Python
because "Python should be more like C#"".

Basically, you don't have to turn this into a crusade just because I
thought an idea from another language might be worth some thought and
discussion. Python is not the one true faith - it is a programming
language. Like all good programming languages it has sometimes adopted
good ideas from other programming languages. This idea is not destined
to be one of them, and that's fine - it was a bad idea, but that
doesn't make it a heresy. Discovering whether an idea is bad is one of
the advantages of discussion.

"Explicit is better than implicit".

This has nothing to do with it - the 'ref' notation *is* explicit.

OK, remind me why this is better than

x = Inc(x)

or

x, y = DoubleInc(x, y)

The idea I had in mind is that sometimes it is a good thing to
restrict what the caller can do in order to prevent errors. The
return-value method may hypothetically, in some cases, allow excessive
flexibility. If the intention of the function is to replace a value
then literally doing that in the function itself - rather than leaving
it the the caller to decide whether to replace it or whether to keep
both - may be a good idea.

At present Python allows this if the value is a component of a mutable
object passed in as a paramater, but not if the value was passed in
directly as the parameter itself.

As it turns out, I can't think of a compelling example where this idea
would be beneficial and that logic above almost certainly doesn't
connect to the practicalities of real world programming - I've already
said that - but it wasn't immediately obvious that it would work out
like that when I started the thread.

So you introduce a "ref" keyword so that a dangerous and unneccessary
mechanism can be "safely" introduced?

Not at all.

The dangerous thing in JCMs example is the overloading, not the
call-by-reference.

Pass-by-reference parameters are not dangerous. Making the
pass-by-reference explicit in the call is also not dangerous.

I am a little suspicious of the C# system of using 'ref' and 'out' to
resolve overloading, but Python doesn't do overloading at all. That
doesn't mean overloading has to be a bad idea and neither does it mean
that Python is a bad language. Like any language feature overloading
has a good and a bad side to it, and like any good language Pythons
features are selected to work well in the context of the language as a
whole. Different languages simply have different strengths and
weaknesses.

But overloading is a separate issue to pass-by-reference - take a look
at standard Pascal for instance, which has pass-by-reference but no
overloading.

Pass-by-reference is unnecessary in Python, yes. The theoretical
argument does not connect to practical reality. But as I already said,
it was not immediately obvious that it would work out that way.

 

[Bengt Richter]

I have in the past suggested that Python functions could use some kind
of modifier token for parameters to indicate pass-by-reference, so
that immutable items in particular could be modified (or more
accurately replaced) by the function, with that modification affecting
what the caller sees. In case I'm explaining this badly, here is how
Python currently works...

... p += 1
...
1

... and here is (roughly) what I'd like to be able to do...

... p += 1
...
2

ref p logically implies that Inc will receive somehow both the information
*behind* the name 'p' -- i.e., the caller's name for the value that the local
parameter name 'p' is temporarily bound to -- *and* the specific namespace of the caller.

The implication is that there is a name that should be unchanged, whatever it is, but
rebound in its namespace to an updated value. We can't mess with the actual value at
the other end of the binding, since that could be shared and/or immutable.

You could do this very explicitly, e.g., explicitly pass both name space and name string,
with an ordinary dict playing the namespace role:

>>> ns = {'x':1}
>>> ns['x'] 1
>>> def incr(namespace, name):

... namespace[name] += 1
...

>>> incr(ns, 'x')
>>> ns['x']

2

or, using the __dict__ of an object instance
11

obviously, we could write incr to accept an instance and a name and do
inst.__dict__[name]+=1 instead of nsdict[name]+=1, but the point is
we are somehow being explicit in specifying both namespace and
the name to be rebound. Implementing ref would imply automating
something similar hidden in the compiled code, IWT. But it gets
messy fast, I think. what if you write a =['foo',[23]]; Inc(a[1][0]) ?
Should that 23 be rebound to a 24? You can't always analyze the expression
statically, so you can't even presume that Inc will be the same global binding
by the time a[1] has been evaluated. So you'd have to look ahead to see if Inc
should be passed the address of the hidden pointer to 0 involved in the [0] or
the pointer to 0 itself. But then you'd have to figure out whether <whatever-a[1]-is>[0]
can be used that way. Should any mutable object that supports __getitem__ and __setitem__
be able to produce the target address for its __setitem__ for conditional passing
to a function like Inc? ISTM a ref thing would ripple pretty far into the binding
methodology of Python.

Regards,
Bengt Richter

 

[Terry Reedy]

Actually I'm sick to death of the old 'your just trying to turn Python
into X' argument whenever I suggest that *maybe* Python might learn a
trick from another language. [etc]

This seems to be a predictible c.l.py response to proposals. I got
similar accusations several years ago, which made me similarly sick,
when I proposed the addition of list.pop() (which Guido did, in more
generalized form, about a year later).

My main objection was/is that expressed by Michael Chermside today:
this would complexify a simplicity that I consider a good feature of
Python. Reading Bengt Richter's lastest response, also today, I
realize that 'ref x' would have to have much the same effect of Lisp's
FEXPR defun keyword -- of automatically quoting rather than evaluating
args -- although on just one rather than all. Perhaps you should be
accused of 'trying to Lispify Python' -- or perhaps not ;-).

Terry J. Reedy

 

[Bengt Richter]

Actually I'm sick to death of the old 'your just trying to turn Python
into X' argument whenever I suggest that *maybe* Python might learn a
trick from another language. [etc]

This seems to be a predictible c.l.py response to proposals. I got
similar accusations several years ago, which made me similarly sick,
when I proposed the addition of list.pop() (which Guido did, in more
generalized form, about a year later).

My main objection was/is that expressed by Michael Chermside today:
this would complexify a simplicity that I consider a good feature of
Python. Reading Bengt Richter's lastest response, also today, I
realize that 'ref x' would have to have much the same effect of Lisp's
FEXPR defun keyword -- of automatically quoting rather than evaluating
args -- although on just one rather than all. Perhaps you should be
accused of 'trying to Lispify Python' -- or perhaps not ;-).

If we had a wrapper for rebinding targets, then maybe such a wrapper could
be passed to functions and used in a uniform way. E.g., faking it with real python:
... def __init__(self, thing, key):
... self.thing=thing
... self.key=key
... def _getv(self):
... try: return self.thing[self.key]
... except: pass
... try: return getattr(self.thing, self.key)
... except: pass
... raise KeyError,'%r is not key or attribute of %r'%(self.thing, self.key)
... def _setv(self, v):
... try: self.thing[self.key]=v
... except:
... try: setattr(self.thing, self.key, v)
... except: raise KeyError,'%r is not key or attribute of %r'%(self.thing, self.ke
... value = property(_getv,_setv)
...

>>> a= ['foo', [23]]

Oops, I should have called this Inc ;-)
... aw.value = aw.value + 1
...

>>> aaw = GAW(a[1], 0)
>>> aaw.value 23
>>> foo(aaw)
>>> a ['foo', [24]]
>>> aaw.value 24
>>> z=100
>>> foo(GAW(globals(),'z'))
>>> z

101

Maybe something could be done in C and with compiler support to implement safe access
via deferral of the final access tail term of a target expression, using an object with an
apparent value property that works more or less as above -- but so you could spell it
e.g., awrap(z) or awrap(obj.x) or awrap(a[1][0]) by some magic ;-)

The compiler would have to know about awrap, so it could separate the final access mechanism
in the expression and create the code to create the appropriate access object...
(a lot of hand waving, but maybe not impossible?)

Regards,
Bengt Richter

 

[Stephen Horne]

My main objection was/is that expressed by Michael Chermside today:
this would complexify a simplicity that I consider a good feature of
Python. Reading Bengt Richter's lastest response, also today, I
realize that 'ref x' would have to have much the same effect of Lisp's
FEXPR defun keyword -- of automatically quoting rather than evaluating
args -- although on just one rather than all. Perhaps you should be
accused of 'trying to Lispify Python' -- or perhaps not ;-).

I didn't know Lisp could do that. I really have to read up on Common
Lisp one of these days. And figure out Smalltalk properly. And Oberon.
And - well, the list goes on. Theres never enough time, is there :-(

 

[Terry Reedy]

I didn't know Lisp could do that.

Based on my reading of Winston&Horn's LISP (1st ed, 1981, MacLisp
based), auto quoting is the key to how Lisp conveniently turns
statements into syntactic functions. Example:

Python statement: a=b # a is *not* evaluated but used literally
Python function: set('a',b) # requires explicit quote to not eval a
where, for instance, def set(nam,val): globals()[nam]=val

Lisp s-expression 1: (set 'a b) # call is like Python function
Lisp s-expression 2: (setq a b) # where setq is an FEXPR-type function
that evaluates b but uses a literally, like Python statement

Autoquoting is also used for macro manipulation of code.

I have to read up on Common Lisp one of these days.

LISP, 3rd ed, 1989, still in print, uses standard Common Lisp.
Chapter 12 Macros and Backquote might be of interest to you.

Terry J. Reedy

 

[Paul Foley]

I didn't know Lisp could do that. I really have to read up on Common
Lisp one of these days.

Terry said "FEXPR", which tells you he's about 30 years out of date
wrt Lisp Any vaguely modern Lisp has macros instead of FEXPRs.



It's not difficult to implement what you're asking for in Lisp:

(shadow 'defun)

(defmacro defun (name lambda-list &body body)
(let ((args '()) (mlet '()))
(dolist (arg lambda-list)
(if (and (consp arg) (eql (first arg) 'ref))
(let ((tmp (gensym)))
(push tmp args)
(push `(,(second arg) (deref ,tmp)) mlet))
(push arg args)))
`(cl:defun ,name ,(nreverse args)
(symbol-macrolet ,mlet
,@body))))

(defstruct (reference constructor %make-reference (reader writer))
conc-name ref%))
(reader #'identity :type function)
(writer #'identity :type function))

(defmacro ref (obj)
(multiple-value-bind (vars vals args writer reader)
(get-setf-expansion obj)
`(let ,(mapcar #'list vars vals)
(%make-reference (lambda () ,reader) (lambda ,args ,writer)))))

(defun deref (x) (funcall (ref%reader x)))
(defun (setf deref) (v x) (funcall (ref%writer x) v))


E.g.,

USER> (defun foo (x (ref y) z (ref w))
(setq y (+ x y))
(setq w (+ y z w)))
FOO
USER> (let ((x 2) (y 4)) (values (foo 1 (ref x) 3 (ref y)) x y))
10 ; value returned by foo
3 ; value of x after call
10 ; value of y after call

[In fact, you can now use REF to make references to any "place", not
just simple variables: a particular element of an array, an instance
slot accessor, etc., etc.]

--
....Please don't assume Lisp is only useful for Animation and Graphics,
AI, Bioinformatics, B2B and E-Commerce, Data Mining, EDA/Semiconductor
applications, Expert Systems, Finance, Intelligent Agents, Knowledge
Management, Mechanical CAD, Modeling and Simulation, Natural Language,
Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web
Authoring just because these are the only things they happened to
list. -- Kent Pitman

(setq reply-to
(concatenate 'string "Paul Foley " "<mycroft" '(#\@) "actrix.gen.nz>"))

 

[Stephen Horne]

My poster child here would be the idiom

number, digit = divmod (number, base)

Well since I think of this as...

quotient, remainder = divmod (number, base)

I would never have thought of replacing it with...

digit = refdivmod (ref number, base)

....since I don't see this as having any reason to want to replace any
of its inputs rather than providing distinct outputs and leaving the
caller to decide what to do with that information.

But then that's kind of the point, as I haven't found any case where
there's a good reason for using a 'ref' parameter.


divmod is a fine example of what Python does well by having a
convenient return-multiple-values mechanism.

At the risk being accused of trying to turn Python into Haskell,
though, I'd quite like some equivalent of Haskells 'where' local
definitions to make the best use of multiple return values. A quick
imaginary example might be...

money = ( (pounds*100 + pence)
where detail1, detail2, pounds, pence
= file.readrecord () )

Of course the alternative is obvious...

detail1, detail2, pounds, pence = file.readrecord ()
money = pounds*100 + pence

But an in-expression version can be used in places where this cannot -
in a lambda, for instance.

This would likely not be accepted, though. Allowing more things to be
done in an expression context isn't compelling to everyone - see the
rejection of PEP308 for an example. And as Python is an imperative
language, I should probably see the functional features that it has as
a bonus.

 

[Terry Reedy]

Terry said "FEXPR", which tells you he's about 30 years out of date
wrt Lisp

Actually, less than 22 years (LISP was (C)ed in 1981). I have no idea
when the last (MAC)LISP definition using FEXPR was written, if indeed
no one is still using it on a retro machine. I'd bet less than 15
years ago. No need to exaggerate my newly acquired (I just read the
book) and freely admitted (in the followup post) out-of-dateness;-).

Actually, I do not agree idea that acquiring new knowledge of old
things makes one 'out of date' with respect to that subject.

Any vaguely modern Lisp has macros instead of FEXPRs.

Even back then, Winston and Horn presented MACRO as a alternate
keyword, with effects similar to, but more complex and powerful than
FEXPR. But I believe that FEXPR is/was more like what Horne was
proposing for Python, and hence more appropriate,

But back to the semantics of 'function' calls. In all languages I
have used, a function call meant/means one thing: evaluate the
argument expressions according to the rules of the language and pass
the results to the function body. I believe Horne was proposing that
there be some mechanism added to function declaration syntax that
would change the usual argument handling without having to explicitly
change argument expressions. He happened to have gotten that idea
from C#. The point of my FEXPR-containing paragraph is that C# might
well have gotten the idea from Lisp, where it appeared decades ago.
(And hence the joke about Lispifying rather than C#ifying Python.)

Terry J. Reedy

 

[Hung Jung Lu]

I have in the past suggested that Python functions could use some kind
of modifier token for parameters to indicate pass-by-reference,

... p += 1
...

You could do this very explicitly, e.g., explicitly pass both name space and name string,
with an ordinary dict playing the namespace role:

ns = {'x':1}
ns['x'] 1
def incr(namespace, name):

... namespace[name] += 1
...

incr(ns, 'x')
ns['x']

2

Yes, the key is in the namespace.

The pass-by-reference idea has been mentioned at a frequency of about
once a month. Assumming around a decade of Python language, that means
this issue has been discussed for over 100 times. Yet, Python has not
been modified for this purpose. I think everyone can draw their own
conclusion.

Python assigment is just different from assignments in C/C++/Java/C#.
Python assigment is based on "name binding". The usage of namespace
dictionary and name binding is the heart and soul of Python. As a
poster said once upon a time, you need to "reset your brain" in order
to understand it. I know it is true, since I myself had to reset my
brain (I came from C++) before I understood Python.

Despite the apparent similarity in syntax (e.g: x = 3), Python just
does not work like C/C++/Java/C#. When a Python programmer sees the
statement

def f(x):
...

he/she immediately thinks about a name "x" being created in the local
namespace dictionary, which is then bound to the object/value passed
in the argument list. This "Pythonic" way of thinking is completely
foreign to C/C++/Java/C#. I don't need to repeat all these points, as
they have been discussed for over 100 times by other people.

================================

What I'd like to say is though, there are existing ways to deal with
the "pass-by-reference" needs. Otherwise Python wouldn't be as widely
used and praised. Instead of asking for the impossible (i.e.: changing
the Python syntax), why not try to find out what people actually do?

To start with, if you ever want to pass something by reference, you
are not thinking the Pythonic way. I have NEVER, repeat, NEVER, needed
to think on passing something by reference in Python, despite that I
pass things by reference in C++ all the time.

Methods to deal with "pass-by-reference":

(1) The most primitive way is to use a list:

x = [3,]
f(x)

However, my personal experience is that I have NEVER needed to write a
program this way.

(2) Use the return value:

def f(x):
y = x + 1
return y
....
x = f(x)

If multiple values are needed, return a tuple:

def f(x, y):
z = x + 1
w = y + 1
return z, w
....
x, y = f(x, y)

This approach is the most used method.

(3) The above method has the problem that if you want to add one more
variable to the argument list, or return one more value, you need to
modify the code at several spots. The solution is to pass a
"workspace" request data structure. Or, as it is very popular
nowadays, to pass a request and a response object:

def f(request):
request.x += 1

def f(request, response):
response.x = request.x + 1

With these simplified function "prototype headers", you won't need to
change the headers nor return statements as your program becomes more
complex.

regards,

Hung Jung

 

[Stephen Horne]

Python assigment is just different from assignments in C/C++/Java/C#.
Python assigment is based on "name binding".
....

Despite the apparent similarity in syntax (e.g: x = 3), Python just
does not work like C/C++/Java/C#. When a Python programmer sees the
statement

Actually, you could say the same about assignments in Java/C# compared
with C/C++. In C# that would only be for 'reference' objects, but IIRC
in Java it's for all objects.

That is, in C#, if you write...

String x = "Hello";
String y = x;

....both x and y are bound to the same object. Exactly as they would in
Python. Or Perl.

My thinking at the start of this thread was that if its valid to
modify a part of a (mutable) parameter, maybe it should be valid to
modify the whole thing - ie to replace it (meaning rebinding the
reference to the object - the callers variable/whatever).

C# allows this, but more likely the intention is to support
call-by-reference with 'value' types (unboxed basic types such as
integers and chars, and structs). As C# is statically typed and able
to determine the call signatures of functions at compile time the
issues are different to those in Python, but they are equally
different to those in C/C++.

I'm not a big fan of Pythons assignment semantics, but I do understand
them and have been working with them about 6 or 7 years now - about a
year or two longer than I have worked with C++. I have used a lot of
languages in the roughly 18 years since I first started programming. I
have also always had an interest in learning new languages, and in my
experience pretty much every language has something to teach in terms
of new approaches and new ways of thinking.

You say I should not be "asking for the impossible (i.e.: changing
the Python syntax)". I say that Python isn't so stagnant as you seem
to think. It has adopted new ideas, and continues to do so. Python has
adopted techniques from languages as different as Haskell and Icon -
languages that bear essentially no comparison to the ABC that Python
was originally based on.

I am not fixated on one way of working. That includes my not being
fixated on the way Python works now. Which is quite a bit different to
the way it worked in MacPython 1.4 when I first used it.

It isn't a matter of "resetting your brain". It's a matter of being
open to new ideas, even if they were not invented here. Sometimes
those ideas turn out to be bad - but you never find the good ones if
you're closed to them.

Yes - call-by-reference has been discussed before. But the idea of
making it explicit in both the call and the definition has not, and
addresses a number of the past objections. It still turned out to be a
bad idea, but thats the nature of 90% of ideas.

BTW - if you'd read the rest of the thread, you'd know that I
suggested a use case, examined it critically and took on board other
peoples criticisms, and decided the use case was better handled
another way using existing Python features. The alternate solution
that I suggested and that I am still happiest with is basically the
Gamma, Helm, etc design pattern called 'Builder'. Those design
patterns were not aimed at Python - the book illustrates them with
UML, C++ and smalltalk examples, but so far as I've seen the Java
community probably has most design pattern advocates. No language is
an island.

So while Python is still my preferred language whenever I have a
choice, I don't think it is perfect - I'm still expecting it to
improve a little from time to time. Several of the current PEPs look
quite interesting.

 

[Michael Geary]

Python assigment is just different from assignments in C/C++/Java/C#.
Python assigment is based on "name binding". The usage of namespace
dictionary and name binding is the heart and soul of Python. As a
poster said once upon a time, you need to "reset your brain" in order
to understand it. I know it is true, since I myself had to reset my
brain (I came from C++) before I understood Python.

Despite the apparent similarity in syntax (e.g: x = 3), Python just
does not work like C/C++/Java/C#. When a Python programmer sees the
statement

def f(x):
...

he/she immediately thinks about a name "x" being created in the local
namespace dictionary, which is then bound to the object/value passed
in the argument list. This "Pythonic" way of thinking is completely
foreign to C/C++/Java/C#. I don't need to repeat all these points, as
they have been discussed for over 100 times by other people.

Interestingly enough, Python's name binding feels right at home to a
JavaScript programmer. Many of the issues here are the same in both
languages. I don't see Python and JavaScript compared often, but to me
Python resembles JavaScript a lot more than it resembles any of those other
languages.

That's a compliment--I like JavaScript very much. While working on the
multimedia extensions for Acrobat 6.0 last year, I often had a choice
between coding something in JavaScript or C++, and I picked JavaScript
whenever I could. (If you have Acrobat 6.0 or the "big" version of Adobe
Reader 6.0, you'll find my code in media.js in your Acrobat installation...)

-Mike

 

[Donn Cave]

Quoth Stephen Horne <$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk>:
....
| I'm not a big fan of Pythons assignment semantics, but I do understand
| them and have been working with them about 6 or 7 years now - about a
| year or two longer than I have worked with C++.

I wonder if this is a some kind of key observation. Python's
assignment semantics are so much what Python is about, that if
you aren't happy with them, there is something basically wrong.

Hung Jung Lu put it all better than I could in the post you're
responding to. If you want to work with what Python gives you,
its assignment semantics are really just what you need. There
are people who do find it just what they need, and if you're
really interested in this issue, maybe the challenge for you
is to understand why they're happy when you're not. It could
be, among various possible reasons, that there is something
a bit off in your understanding -- not so much that you're
really wrong about it, but maybe you make it harder than it
ought to be, like the difference between thinking in a spoken
language, vs. translating your thoughts into it from your native
language. In fact that was certainly the case a couple months
ago, from your posts in another similar thread.

Or you could look elsewhere. I personally am interested in some
things that I can't and never will get from Python, and that means
checking out some other languages.

Donn Cave, (e-mail address removed)

 

[Stephen Horne]

Quoth Stephen Horne <$$$$$$$$$$$$$$$$$@$$$$$$$$$$$$$$$$$$$$.co.uk>:
...
| I'm not a big fan of Pythons assignment semantics, but I do understand
| them and have been working with them about 6 or 7 years now - about a
| year or two longer than I have worked with C++.

I wonder if this is a some kind of key observation. Python's
assignment semantics are so much what Python is about, that if
you aren't happy with them, there is something basically wrong.
....

It could
be, among various possible reasons, that there is something
a bit off in your understanding -- not so much that you're
really wrong about it, but maybe you make it harder than it
ought to be, like the difference between thinking in a spoken
language, vs. translating your thoughts into it from your native
language.

My first instinct is to say "why do you have to insist that I'm
wrongthinking", but in fact this is a pretty good summary of my
position. Python's assignment semantics just don't quite seem natural
to me, so that while I can describe them perfectly accurately right
now I know that if I go more than a couple of months without using
Python I may well end up misunderstanding them yet again as I have in
the past - see (e-mail address removed) for an
example.

But I don't go much for cutting my nose off to spite my face. This is
one feature, and the fact that getting used to it when I come back to
Python is too quick to cause much pain.

I tend to get into this thing where I see the assignment/passing of
immutable objects as different to the mutable object case. This is
because I find copy semantics more natural, and because in the
immutable case Pythons semantics co-incindentally give the same result
as copy semantics. Therefore, for immutable objects, I don't get
forced back to reality as soon as I use Python.

That said, after some of the messages in the thread branch that post
above prompted, and after having gone through the cycle, I may
possibly have got the reality fixed in my mind now. And given that C#
really does have two sets of assignment semantics - I may be doomed
:-(

Sharing objects by reference is now much more mainstream than it once
was. Python, Perl, Java and C# all do it. It has always been the norm
in some areas of course,

Thinking about it, there's more than just performance concerns -
garbage collection is much easier if objects are not directly 'owned'
by any piece of code (e.g. if objects don't get automatically
discarded as the stack unwindeds when a function returns, but can be
preserved if any references survive).

Whether I like it or not, I have to put up with it - it's not going
away.