Generator

[R. David Murray]

Can you explain me this behaviour:

s = [1,2,3,4,5]
g = (x for x in s)
next(g) 1
s [1, 2, 3, 4, 5]
del s[0]
s [2, 3, 4, 5]
next(g) 3

Why next(g) doesn't give me 2?

Think of it this way: the generator is exactly equivalent to
the following generator function:

def g(s):
for x in s:
yield x

Now, if you look at the documentation for the 'for' statement, there is
a big "warning" box that talks about what happens when you mutate an
object that is being looped over:

There is a subtlety when the sequence is being modified by the loop (this
can only occur for mutable sequences, i.e. lists). An internal counter is
used to keep track of which item is used next, and this is incremented on
each iteration. When this counter has reached the length of the sequence
the loop terminates. This means that if the suite deletes the current (or
a previous) item from the sequence, the next item will be skipped (since
it gets the index of the current item which has already been treated).
Likewise, if the suite inserts an item in the sequence before the current
item, the current item will be treated again the next time through the
loop.

As you can see, your case is covered explicitly there.

If you want next(g) to yield 3, you'd have to do something like:

g = (x for x in s[:])

where s[:] makes a copy of s that is then iterated over.

 

[R. David Murray]

Il Sun, 22 Mar 2009 16:52:02 +0000, R. David Murray ha scritto:

Can you explain me this behaviour:

s = [1,2,3,4,5]
g = (x for x in s)
next(g)
1
s
[1, 2, 3, 4, 5]
del s[0]
s
[2, 3, 4, 5]
next(g)
3


Why next(g) doesn't give me 2?

Think of it this way: the generator is exactly equivalent to the
following generator function:

def g(s):
for x in s:
yield x

Now, if you look at the documentation for the 'for' statement, there is
a big "warning" box that talks about what happens when you mutate an
object that is being looped over:

There is a subtlety when the sequence is being modified by the loop
(this can only occur for mutable sequences, i.e. lists). An
internal counter is used to keep track of which item is used next,
and this is incremented on each iteration. When this counter has
reached the length of the sequence the loop terminates. This means
that if the suite deletes the current (or a previous) item from the
sequence, the next item will be skipped (since it gets the index of
the current item which has already been treated). Likewise, if the
suite inserts an item in the sequence before the current item, the
current item will be treated again the next time through the loop.

As you can see, your case is covered explicitly there.

If you want next(g) to yield 3, you'd have to do something like:

g = (x for x in s[:])

where s[:] makes a copy of s that is then iterated over.

Ok, thanks. Yes, I had the idea that a counter was used in order to
explain my problem. Now I know that my intuition was correct. Thanks.

By the way, it's not the 'for' loop that maintains the counter. It's the
code that implements the iteration protocol for the thing being looped
over. So theoretically you could make a subclass of list that would
somehow handle items being deleted or added in a sensible fashion.
But I doubt it is worth doing that, since the implementation would be
would be pretty idiosyncratic to the use-case.