Tuple assignment and generators?

[Tim Chase]

Just as a pedantic exercise to try and understand Python a
bit better, I decided to try to make a generator or class
that would allow me to unpack an arbitrary number of
calculatible values. In this case, just zeros (though I
just to prove whatever ends up working, having a counting
generator would be nice). The target syntax would be
something like

where "zeros()" returns an appropriately sized tuple/list of
zeros.

I've tried a bit of googling, but all my attempts have just
ended up pointing to pages that blithly describe tuple
assignment, not the details of what methods are called on an
object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to
unpack" result.

As a second attempt, I tried a couple of attempts at classes
(I started with the following example class, only derived
from "object" rather than "list", but it didn't have any
better luck):
.... def __getitem__(self,i):
.... return 0
....Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack


It looks like I need to have a pre-defined length, but I'm
having trouble figuring out what sorts of things need to be
overridden. It seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the
interpreter into really believing that the object has the
desired elements is escaping me. Alternatively if there was
a "doYouHaveThisManyElements" pseudo-function that was
called, I could lie and always return true.

Any hints on what I'm missing?

Thanks,

-tkc

 

[Diez B. Roggisch]

Just as a pedantic exercise to try and understand Python a
bit better, I decided to try to make a generator or class
that would allow me to unpack an arbitrary number of
calculatible values. In this case, just zeros (though I
just to prove whatever ends up working, having a counting
generator would be nice). The target syntax would be
something like

<snip/>

By using this:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/284742


I came up with a small decorator doing that:

import inspect, dis

def expecting():
"""Return how many values the caller is expecting"""
f = inspect.currentframe()
f = f.f_back.f_back
c = f.f_code
i = f.f_lasti
bytecode = c.co_code
instruction = ord(bytecode[i+3])
if instruction == dis.opmap['UNPACK_SEQUENCE']:
howmany = ord(bytecode[i+4])
return howmany
elif instruction == dis.opmap['POP_TOP']:
return 0
return 1

def variably_unpack(f):
def d(*args, **kwargs):
r = f(*args, **kwargs)
exp = expecting()
if exp < 2:
return exp
return (r.next() for i in xrange(exp))
return d

@variably_unpack
def test():
def gen():
i = 0
while True:
yield i
i += 1
return gen()



a, b, c = test()

print a,b,c

a, b, c, d = test()

print a,b,c, d



Diez

 

[Larry Bates]

Just as a pedantic exercise to try and understand Python a bit better, I
decided to try to make a generator or class that would allow me to
unpack an arbitrary number of calculatible values. In this case, just
zeros (though I just to prove whatever ends up working, having a
counting generator would be nice). The target syntax would be something
like


where "zeros()" returns an appropriately sized tuple/list of zeros.

I've tried a bit of googling, but all my attempts have just ended up
pointing to pages that blithly describe tuple assignment, not the
details of what methods are called on an object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to unpack" result.

As a second attempt, I tried a couple of attempts at classes (I started
with the following example class, only derived from "object" rather than
"list", but it didn't have any better luck):

... def __getitem__(self,i):
... return 0
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack


It looks like I need to have a pre-defined length, but I'm having
trouble figuring out what sorts of things need to be overridden. It
seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the interpreter into
really believing that the object has the desired elements is escaping
me. Alternatively if there was a "doYouHaveThisManyElements"
pseudo-function that was called, I could lie and always return true.

Any hints on what I'm missing?

Thanks,

-tkc

While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]

of if you like:

def zeros(num):
return num*[0]

a,b,c = zeros(3)
q,r,s,t,u,v = zeros(6)

I think the reason I don't every use anything like this is that
you don't need to initialize variables in Python to zero and I would
probably use a list instead of individual variables like q,r,s,t,u,v.

-Larry Bates

 

[Ant]

I don't think he was explicitly wanting to initialize things to zero,
but rather unpack an arbitrary sequence into a tuple (could perhaps be
the fibonnacci sequence for example).

How about:
.... for i in range(count):
.... yield 0
....0

 

[Just]

While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]

This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just

 

[Larry Bates]

While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]

This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just

You must be careful with this as they all point to
exactly the same object. Example:

Notice that all of them point to exactly the same object,
not 6 copies of zero which is "probably" what the poster
was thinking.

Most of the time when I see this, it is because people are
thinking of variables having values which is mostly a
carry-over from old Fortran/Cobol/Basic programming ideas.
In python variables are pointers to objects. Objects could
be values, but they are not placeholders where you store
stuff.

I read on this list (quite frequently) that people
think they are getting 6 separate variables each with
a zero stored in them. They are not. They are getting
six pointers that all point to an integer zero (IMHO it
would be a rather odd application for a programmer
to want this). Here is where multiple assignments
causes problems for beginners:

a=[]
b=c=a
a.append(6)
b

[6]

What?? 'b' should contain an empty list, right? Nope.
a, b, and c all point to the SAME list just like the
poster's q, r, s, t, u, v all point to the SAME zero.

What they meant to write was:

c=a[:] # Shallow copy of list
b=a[:]

My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.

-Larry Bates

 

[Tim Chase]

I don't think he was explicitly wanting to initialize

things to zero, but rather unpack an arbitrary sequence
into a tuple (could perhaps be the fibonnacci sequence
for example).

Ant is correct here...Fibonnaci, digits of pi, the constant
42, an incrementing counter, a series of squares, whatever.
That was my original intent in the matter. Zeros just
happened to be a nice sort of place to start my exercise.

How about:



... for i in range(count):
... yield 0

One of the things I was trying to avoid was having to know
(and maintain) the count of items initialized. In the
theoretical world of my example, one could do something like

def counter():
counter = 0
while 1:
yield counter
counter += 1

and then initialize several variables, such as
(0,1,2,3,4,5,6)

It's similar to C's auto-numbering of enum values...If I
want to add another entry to a C enum, I just put it there,
and let the language take care of matters. With most of the
provided solutions, I also have to increment the count each
time I add an item to the list.

Diez provided an elegant solution with a decorator
(employing an incredibly ugly sort of hack involving
sniffing the opcode stack behind the scenes) that does what
I was looking for.

I was hoping that there was just some __foo__ property I was
missing that would have been called in the process of tuple
unpacking that would allow for a more elegant solution such
as a generator (or generator method on some object) rather
than stooping to disassembling opcodes.

Ah well.

-tkc

 

[jemfinch]

While I have never needed anything like this in my 5 years of Python
programming, here is a way:

a,b,c = 3*[0]
q,r,s,t,u,v = 6*[0]

This is (IMO) fairly idiomatic:

a = b = c = 0
q = r = s = t = u = v = 0

Just

You must be careful with this as they all point to
exactly the same object. Example:

Notice that all of them point to exactly the same object,
not 6 copies of zero which is "probably" what the poster
was thinking.

Numbers are immutable. They're never copied. Zero, in particular, is
the same variable all throughout a Python interpreter.

Most of the time when I see this, it is because people are
thinking of variables having values which is mostly a
carry-over from old Fortran/Cobol/Basic programming ideas.

Most of the time when I see it, it's written by someone who's used
Python for quite some time. It's a standard Python idiom. You'll find
it all over the standard library. It's not a carry-over from
Fortran/Cobol/Basic at all.

In python variables are pointers to objects. Objects could
be values, but they are not placeholders where you store
stuff.

And all immutable objects are indistinguishable from values. Immutable
objects include ints, longs, strings, unicode objects, tuples,
frozensets, and perhaps some others that I'm forgetting.

I read on this list (quite frequently) that people
think they are getting 6 separate variables each with
a zero stored in them.

That's because they are. They're getting 6 different pointers
(bindings) to zero. If you change one, the others remain pointed at
(bound to) zero.

They are not. They are getting
six pointers that all point to an integer zero

Six *different* pointers. Six *different* bindings.

(IMHO it
would be a rather odd application for a programmer
to want this).

No, it wouldn't be. It's exactly how a person works with immutable
(value) objects.

Here is where multiple assignments
causes problems for beginners:

a=[]
b=c=a
a.append(6)
b

[6]

Yes, that does sometimes trouble beginners to programming. But so do
regular expressions. Programmers ought not to restrict themselves by
what beginners will have no difficulty learning.

What?? 'b' should contain an empty list, right? Nope.
a, b, and c all point to the SAME list just like the
poster's q, r, s, t, u, v all point to the SAME zero.

There is only one zero in Python! It can never change!

What they meant to write was:

c=a[:] # Shallow copy of list
b=a[:]

My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.

Your rule should only be followed by people who obstinately refuse
either to understand the way variable bindings work in Python, or by
people who refuse to know or care whether a given kind of object is
immutable. And that group of people doesn't seem to include any of the
good Python programmers I know.

Jeremy

 

[jemfinch]

Zero, in particular, is the same variable all throughout a Python interpreter.

For the sake of accuracy let me note that I ought to have said, "is the
same *value* all throughout a Python interpreter."

Jeremy

 

[Ben Finney]

There is only one zero in Python! It can never change!

+0.5 QOTW

 

[Carl Banks]

You must be careful with this as they all point to
exactly the same object. Example:

3301924

But:
134536636


[snip]
My rule, don't do it unless you know exactly why you
want to do it. It will trip you up at some point and
be VERY hard to find.

It's ok to do it with constant objects, really.


Carl Banks

 

[Michele Simionato]

It is okay with constant object, really.
No:

135104688

It worked with the number 0 because of an implementation accident,
in general Python can use different ids for constant objects that are
equals in the == sense.

Michele Simionato

 

[vdrab]

Wow, so, to see if I understand correctly:

what the...?
does anybody else get mighty uncomfortable about this?
s.

 

[Erik Max Francis]

what the...?
does anybody else get mighty uncomfortable about this?

No. Why should you ever care about whether two integers representing
values are the same object? Your tests should be with `==`, not `is`.

 

[Diez B. Roggisch]

Wow, so, to see if I understand correctly:


what the...?
does anybody else get mighty uncomfortable about this?

#include <stdio.h>

int main(int argc, char **argv) {
int a = 1;
int b = 1;
printf("a == b: %i\n", a == b);
printf("&a == &b: %i\n", &a == &b);
return 0;
}

droggisch@ganesha:/tmp$ ./test
a == b: 1
&a == &b: 0


Feeling the same might uncomfortableness? Get used to it: object identity
and two objects being part of an equality-relation are two different
beasts. It can get even worse: I can define an object (in C++ as well as in
python) that is not even equal to itself. Not that I felt the need for that
so far....

So: don't use object identity where you want equality. In all languages.

Diez

 

[robert]

Just as a pedantic exercise to try and understand Python a bit better, I
decided to try to make a generator or class that would allow me to
unpack an arbitrary number of calculatible values. In this case, just
zeros (though I just to prove whatever ends up working, having a
counting generator would be nice). The target syntax would be something
like


where "zeros()" returns an appropriately sized tuple/list of zeros.

I've tried a bit of googling, but all my attempts have just ended up
pointing to pages that blithly describe tuple assignment, not the
details of what methods are called on an object in the process.

My first thought was to get it to use a generator:

def zeros():
while 1: yield 0

However, I get back a "ValueError: too many values to unpack" result.

As a second attempt, I tried a couple of attempts at classes (I started
with the following example class, only derived from "object" rather than
"list", but it didn't have any better luck):

... def __getitem__(self,i):
... return 0
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ValueError: need more than 0 values to unpack


It looks like I need to have a pre-defined length, but I'm having
trouble figuring out what sorts of things need to be overridden. It
seems like I sorta need a

def __len__(self):
return INFINITY

so it doesn't choke on it. However, how to dupe the interpreter into
really believing that the object has the desired elements is escaping
me. Alternatively if there was a "doYouHaveThisManyElements"
pseudo-function that was called, I could lie and always return true.

Any hints on what I'm missing?

Think the use case, you present, is (pythonically) ill as such. if you
want to fill literal variable's do it like: a=b=c=...=0. Otherwise
you'd handle variable lists zeros(7)

Yet there is a real freqent need for adding transparent extra/decorating
return values in cases, where you don't want to break the simple return
scheme (in maybe big existing code). For such use cases check out this
"RICHVALUE" approach with _named_ extra values - no puzzles about
non-matching tuple assignments:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/496676 :

....
def f():
return RICHVALUE(7, extra='hello')

ret=f()
print ret, ret+1, ret.extra
....


As I have this need very very often I'd like to see an even more natural
builtin method for enriching return values (both positional and by name)
in Python3000 - maybe by making calling and returning almost symetric!
Yet, this ideas maybe strange ? ...

def f_P3K(*args,**kwargs):
xreturn((7,8), 3, extra=5, *args, **kwargs) # first is main

v = f_P3K() # (7,8)
v,x = f_P3K() # (7,8),3
v,x,*pret,**kwret= f_P3K()
extra=kwret.get('extra',-1)


-robert

 

[vdrab]

No. Why should you ever care about whether two integers representing

values are the same object? Your tests should be with `==`, not `is`.

Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...
In fact, I think my code contains things like "if len(arg) is 0:" and
so on, and I feel I should be able to do so given the way python treats
(claims to treat?) constant objects, even if I don't care whether the
values actually represent the same object.
s.

 

[vdrab]

beasts. It can get even worse: I can define an object (in C++ as well as in

python) that is not even equal to itself. Not that I felt the need for that
so far....

hehe... now you've picked my curiosity... how?

ps.
def __eq__(self, other): return False
does not count !

 

[Diez B. Roggisch]

hehe... now you've picked my curiosity... how?

ps.
def __eq__(self, other): return False
does not count !

Sure it does! The rich comparison methods can very well be used to define
whatever semantics you want. Now I agree that

def __eq__(self, other):
if self is other:
return False
... # do something more sane


doesn't make much sense usually. Or maybe even never. Still, it illustrates
the point: object identity is not to be confused with two objects being
part of an equality relation.

Diez

 

[Fredrik Lundh]

Given this though, what other such beauties are lurking in the
interpreter, under the name of 'implementation accidents'? One of the
things that drew me to python is the claimed consistency and
orthogonality of both language and implementation, not sacrificing
clarity for performance, minimizing ad-hoc design hacks and weird
gotcha's, etc...

so anything you don't understand, and cannot be bothered to look up in
the documentation, just has to be an inconsistent ad-hoc weird-gotcha
design ?

I think we can all safely *plonk* you know.

</F>