Python Args By Reference

[Joseph Garvin]

Hello all, I was wondering if there was any way to pass arguments
(integer and such) by reference (address of), rather than by value.

Many thanks in advance.

-Wes

All mutable types in python are passed by reference automatically.

 

[ncf]

Hello all, I was wondering if there was any way to pass arguments
(integer and such) by reference (address of), rather than by value.

Many thanks in advance.

-Wes

 

[Grant Edwards]

Hello all, I was wondering if there was any way to pass arguments
(integer and such) by reference (address of), rather than by value.

Yes. All arguments are passed by reference. This must be in
the FAQ somewhere...

Sure enough. 30 seconds of browsing the FAQ yielded this:

http://www.python.org/doc/faq/progr...tion-with-output-parameters-call-by-reference

 

[Dan Bishop]

All mutable types in python are passed by reference automatically.

More accurately:

(1) All function arguments are passed by value, but
(2) All "values" are pointers.

Statement (2) also explains why

a = [1, 2, 4]
b = a
b[1] = 0
print a

[1, 0, 4]

behaves the way it does.

And there's no difference between mutable and immutable types. It's
just that with immutable types, it doesn't matter how they're passed.

 

[Grant Edwards]

All mutable types in python are passed by reference automatically.

So are immutable ones. It just doesn't matter in that case.

 

[ncf]

Ok, I'm relatively new to python (< 1 year experiance), yet I have had
much experiance with other languages.

I never really looked that deeply in the FAQ -- temporary lapse of
stupidity(?). Thanks for the link, the concept seems to help.

The two issues I am having in grasping all of this are as follows:
1) Being new to Python, terms like mutable and immutable. Although I
have probably dealt with them in other languages, the terms by
themselves are a little confusing, but managable overall, so this issue
isn't so big.

2) LARGELY, my issue is as demonstrated by the following code. I was
trying to accomplish an effect similar to what is possible in C.
(Trying to make a pure-python FIPS-180-2 compliant implementation of
the SHA-256 algorithm for more Python practice and to put into some
code for a *trial* secure protocol.)
Example C Code:
#define P(a,b,c,d,e,f,g,h,x,K) \
{ \
temp1 = h + S3(e) + F1(e,f,g) + K + x; \
temp2 = S2(a) + F0(a,b,c); \
d += temp1; h = temp1 + temp2; \
}

Python Code:
def P(a,b,c,d,e,f,g,h,x,K):
temp1 = h + S3(e) + F1(e,f,g) + K + x
temp2 = S2(a) + F0(a,b,c)
d += temp1; h = temp1 + temp2

The reason why it'd be a pain to implement this by any of the methods
provided in the Python FAQs is that SHA-256 rotates the variable order
in the calls. Example code:
P( A, B, C, D, E, F, G, H, W[ 0], 0x428A2F98 );
P( H, A, B, C, D, E, F, G, W[ 1], 0x71374491 );
P( G, H, A, B, C, D, E, F, W[ 2], 0xB5C0FBCF );
P( F, G, H, A, B, C, D, E, W[ 3], 0xE9B5DBA5 );
P( E, F, G, H, A, B, C, D, W[ 4], 0x3956C25B );
P( D, E, F, G, H, A, B, C, W[ 5], 0x59F111F1 );
P( C, D, E, F, G, H, A, B, W[ 6], 0x923F82A4 );
P( B, C, D, E, F, G, H, A, W[ 7], 0xAB1C5ED5 );
P( A, B, C, D, E, F, G, H, W[ 8], 0xD807AA98 );

Since I cannot simply do it the way I had originally seen it, would
there be an alternative method to proforming the operations that I am
missing?

Once again, many thanks for your time.
-Wes

 

[Paul Rubin]

Since I cannot simply do it the way I had originally seen it, would
there be an alternative method to proforming the operations that I am
missing?

Use an array instead of all those named variables.

 

[ncf]

As I fail to see how an array could be used in this (my own
stupidity?), would you have any such example? For reference, I'm trying
to translate this: http://www.cr0.net:8040/code/crypto/sha256/ (Inside
sha256_process).

Once again, thanks for the patience, I'm still picking up on all the
little tid-bits.

-Wes

 

[Paul Rubin]

As I fail to see how an array could be used in this (my own
stupidity?), would you have any such example?

How's this (untested):

state = [A,B,C,D,E,F,G,H]
magic = [0x428A2F98, 0x71374491, 0xB5C0FBCF, 0xE9B5DBA5,
0x3956C25B, 0x59F111F1, 0x923F82A4, 0xAB1C5ED5, 0xD807AA98 ]

def P(state, i, magic):
a,b,c,d,e,f,g,h = state[i:] + state[:i]
temp1 = h + S3(e) + F1(e,f,g) + K + x
temp2 = S2(a) + F0(a,b,c)
# d += temp1; h = temp1 + temp2
state[(i+3)%8] += temp1; state[(i+7)%8] = temp1 + temp2

for i in range(9):
P(state, i, W, magic)

Actually this isn't so good. You have to be careful about arithmetic
overflow, i.e. do all the additions mod 2**32, so you don't get Python
longs. I'll leave that as an exercise.

 

[Paul Rubin]

state = [A,B,C,D,E,F,G,H]
magic = [0x428A2F98, 0x71374491, 0xB5C0FBCF, 0xE9B5DBA5,
0x3956C25B, 0x59F111F1, 0x923F82A4, 0xAB1C5ED5, 0xD807AA98 ]

def P(state, i, magic):
a,b,c,d,e,f,g,h = state[i:] + state[:i]
temp1 = h + S3(e) + F1(e,f,g) + K + x

Woops, K is supposed to say magic.

 

[Dan Bishop]

As I fail to see how an array could be used in this (my own
stupidity?), would you have any such example? For reference, I'm trying
to translate this: http://www.cr0.net:8040/code/crypto/sha256/ (Inside
sha256_process).

Once again, thanks for the patience, I'm still picking up on all the
little tid-bits.

def P(arr, x, k):
a, b, c, d, e, f, g, h = arr
temp1 = arr[7] + S3(arr[4]) + F1(*arr[4:7]) + k + x
temp2 = S2(arr[0]) + F0(*arr[:3])
arr[3] += temp1
arr[7] = temp1 + temp2

K = [0x428A2F98, 0x71374491, 0xB5C0FBCF,
0xE9B5DBA5, 0x3956C25B, 0x59F111F1,
0x923F82A4, 0xAB1C5ED5, 0xD807AA98]

....

for i in xrange(9):
P(arr, w, K)
if i != 8: # don't rotate the last time
arr = [arr[-1]] + arr[:-1] # rotate arr


The last line moves the last element of arr to the beginning (so the
positions are correct for the call to P).

 

[Donn Cave]

Quoth "ncf" <[email protected]>:
....
| The two issues I am having in grasping all of this are as follows:
| 1) Being new to Python, terms like mutable and immutable. Although I
| have probably dealt with them in other languages, the terms by
| themselves are a little confusing, but managable overall, so this issue
| isn't so big.

It's often introduced in a confusing way. We just say it's mutable
if it has one or more functions that modify its contents. There
isn't any other distinction, any other difference than that.

| 2) LARGELY, my issue is as demonstrated by the following code. I was
| trying to accomplish an effect similar to what is possible in C.
| (Trying to make a pure-python FIPS-180-2 compliant implementation of
| the SHA-256 algorithm for more Python practice and to put into some
| code for a *trial* secure protocol.)
| Example C Code:
| #define P(a,b,c,d,e,f,g,h,x,K) \
| { \
| temp1 = h + S3(e) + F1(e,f,g) + K + x; \
| temp2 = S2(a) + F0(a,b,c); \
| d += temp1; h = temp1 + temp2; \
| }
|
| Python Code:
| def P(a,b,c,d,e,f,g,h,x,K):
| temp1 = h + S3(e) + F1(e,f,g) + K + x
| temp2 = S2(a) + F0(a,b,c)
| d += temp1; h = temp1 + temp2
|
| The reason why it'd be a pain to implement this by any of the methods
| provided in the Python FAQs is that SHA-256 rotates the variable order
| in the calls. Example code:
| P( A, B, C, D, E, F, G, H, W[ 0], 0x428A2F98 );
| P( H, A, B, C, D, E, F, G, W[ 1], 0x71374491 );
| P( G, H, A, B, C, D, E, F, W[ 2], 0xB5C0FBCF );
| P( F, G, H, A, B, C, D, E, W[ 3], 0xE9B5DBA5 );
| P( E, F, G, H, A, B, C, D, W[ 4], 0x3956C25B );
| P( D, E, F, G, H, A, B, C, W[ 5], 0x59F111F1 );
| P( C, D, E, F, G, H, A, B, W[ 6], 0x923F82A4 );
| P( B, C, D, E, F, G, H, A, W[ 7], 0xAB1C5ED5 );
| P( A, B, C, D, E, F, G, H, W[ 8], 0xD807AA98 );
|
| Since I cannot simply do it the way I had originally seen it, would
| there be an alternative method to proforming the operations that I am
| missing?

I could be missing something, but I think you just have to suck it up
and do it one of the other two ways:

1. The more or less functional way: the function doesn't
have any side effects. It just returns the values it
computes, and it's for the caller to bind them to variables.

D, H = P(A, B, C, D, E, F, G, H, W[0], 0x428A2F98)
C, G = (H, A, B, C, D, E, F, G, W[1], 0x71374491)
...

2. The OOP way: each object has the appropriate function
that modifies its content value. That looks like a
big waste of time to me, but I mention it for conceptual
completeness.

Donn Cave, (e-mail address removed)

 

[Roy Smith]

The two issues I am having in grasping all of this are as follows:
1) Being new to Python, terms like mutable and immutable. Although I
have probably dealt with them in other languages, the terms by
themselves are a little confusing, but managable overall, so this issue
isn't so big.

If you know C++, imagine a class where every method is declared const.
That's essentially what Python's immutable means. Once you've created an
object, you can never modify it (other than to destroy it).

The most common immutable objects you'll see are strings and tuples, and
the main reason they're immutable is to allow them to be dict keys.

2) LARGELY, my issue is as demonstrated by the following code. I was
trying to accomplish an effect similar to what is possible in C.
(Trying to make a pure-python FIPS-180-2 compliant implementation of
the SHA-256 algorithm for more Python practice and to put into some
code for a *trial* secure protocol.)
Example C Code:
#define P(a,b,c,d,e,f,g,h,x,K) \
{ \
temp1 = h + S3(e) + F1(e,f,g) + K + x; \
temp2 = S2(a) + F0(a,b,c); \
d += temp1; h = temp1 + temp2; \
}

Python Code:
def P(a,b,c,d,e,f,g,h,x,K):
temp1 = h + S3(e) + F1(e,f,g) + K + x
temp2 = S2(a) + F0(a,b,c)
d += temp1; h = temp1 + temp2

Perhaps something like this:

def P(a,b,c,d,e,f,g,h,x,K):
temp1 = h + S3(e) + F1(e,f,g) + K + x
temp2 = S2(a) + F0(a,b,c)
return (d+temp1, temp1+temp2)

then you could call it as:

D, H = P( A, B, C, D, E, F, G, H, W[ 0], 0x428A2F98 )
D, H = P( H, A, B, C, D, E, F, G, W[ 1], 0x71374491 )
D, H = P( G, H, A, B, C, D, E, F, W[ 2], 0xB5C0FBCF )
D, H = P( F, G, H, A, B, C, D, E, W[ 3], 0xE9B5DBA5 )
D, H = P( E, F, G, H, A, B, C, D, W[ 4], 0x3956C25B )
D, H = P( D, E, F, G, H, A, B, C, W[ 5], 0x59F111F1 )
D, H = P( C, D, E, F, G, H, A, B, W[ 6], 0x923F82A4 )
D, H = P( B, C, D, E, F, G, H, A, W[ 7], 0xAB1C5ED5 )
D, H = P( A, B, C, D, E, F, G, H, W[ 8], 0xD807AA98 )

 

[Peter Hansen]

The most common immutable objects you'll see are strings and tuples, and
the main reason they're immutable is to allow them to be dict keys.

And ints. Strings, tuples and ints are the *three* most common
immutable objects you'll see...

-Peter

 

[Grant Edwards]

The most common immutable objects you'll see are strings and tuples,

And integers. They're immutable, aren't they? At least the
small ones. And floats.

 

[Grant Edwards]

And ints. Strings, tuples and ints are the *three* most
common immutable objects you'll see...

Right. In Python, strings tuples and ints are the five most
common immutable objects you'll see.

 

[ncf]

Thanks to everyone for your assistance. I shall reread this a couple
times and then try to make something that works.

Many thanks and have a GREAT day.
-Wes

 

[Terry Hancock]

And ints. Strings, tuples and ints are the *three* most common
immutable objects you'll see...

And floats. Floats, strings, tuples, and ints art the *four* most common
immutable objects you'll see...

Time to break out the comfy chair. ;-)

 

[Fredrik Lundh]

Yes. All arguments are passed by reference. This must be in
the FAQ somewhere...

hopefully not, because that saying that "all arguments are passed
by reference" is extremely confusing for people who've learned about
"call by reference" in school.

as has discussed many times on this list, using "call by object"
(or perhaps "call by object reference" or even "call by sharing")
is better, both for CS heads and for ordinary people.

see e.g.

http://mail.python.org/pipermail/python-list/2003-May/163312.html
http://mail.python.org/pipermail/python-list/2003-May/163028.html
http://mail.python.org/pipermail/python-list/2003-May/163078.html

from a thread where someone claimed that Python used "call by value"
(which is true, as long as you're free to use your own definition of the
word "value" ;-)

</F>

 

[Terry Reedy]

Example C Code:
#define P(a,b,c,d,e,f,g,h,x,K) \
{ \
temp1 = h + S3(e) + F1(e,f,g) + K + x; \
temp2 = S2(a) + F0(a,b,c); \
d += temp1; h = temp1 + temp2; \
}

Python Code:
def P(a,b,c,d,e,f,g,h,x,K):
temp1 = h + S3(e) + F1(e,f,g) + K + x
temp2 = S2(a) + F0(a,b,c)
d += temp1; h = temp1 + temp2

Your problem is that you are replacing a C macro, which is inlined and
affects the original 'namespace' in which it appears, and which does not
have a direct Python equivalent, with a Python function (admittedly
slower), which performs its calculation in a separate namespace. So you
have to pass the results back to the calling namespace. Simple enough:
make the last line of P

return d+temp1, temp1+temp2

and the call

a4,a8 = P(a1, a2, a3, a4, a5, a6, a7, a8, a9, a10)

The stuff about mutability and calling conventions is important to learn
eventually but seems beside the point for your immediate application.

Terry J. Reedy