should I transfer 'iterators' between functions?

[seaspeak]

take the following as an example, which could work well.
But my concern is, will list 'l' be deconstructed after function return? and then iterator point to nowhere?

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return iter(l)
def main():
for i in test():
print(i)

 

[Chris Angelico]

take the following as an example, which could work well.
But my concern is, will list 'l' be deconstructed after function return? and then iterator point to nowhere?

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return iter(l)
def main():
for i in test():
print(i)

Perfectly safe. Python guarantees that nothing can ever point to
"nowhere"; everything that might be looking for something else will
hold a reference to it, so the thing referred to will hang around.

ChrisA

 

[Steven D'Aprano]

take the following as an example, which could work well. But my concern
is, will list 'l' be deconstructed after function return? and then
iterator point to nowhere?

That would be a pretty awful bug for Python, since it would like lead to
a core dump. But no, it works fine, as you can see by trying it at the
interactive interpreter:

py> def test():
.... L = [1, 2, 4, 8, 16]
.... return iter(L)
....
py>
py> for i in test():
.... print(i)
....
1
2
4
8
16


In fact, we don't even need a function to see the same effect:


py> L = [1, 2, 4, 8, 16]
py> it = iter(L)
py> del L
py> L = "something else"
py> for i in it:
.... print(i)
....
1
2
4
8
16


Python keeps track of when objects are in use, and does not destroy them
so long as it is in use. In the meantime, you can exit the function,
unbind (delete) the variable, re-assign to something else, whatever, and
you should never get a core dump.

(I've only ever seen a single core dump in Python in 15+ years.)

 

[Ned Batchelder]

take the following as an example, which could work well.
But my concern is, will list 'l' be deconstructed after function return? and then iterator point to nowhere?

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return iter(l)
def main():
for i in test():
print(i)

The two things to understand here are names, and values. When you leave
the function test, the name l goes away, because it is a locally scoped
name. It referred to a value, your list. But values have reference
counts, and are not reclaimed until their reference count drops to zero.

Your function is returning a value, a listiterator, and that
listiterator refers to the list, so the list won't be reclaimed.

Names have scopes but no types. Values have types, but no scope. They
live as long as they are referred to by something.

This is covered in more detail here:
http://nedbatchelder.com/text/names.html

 

[Ned Batchelder]

take the following as an example, which could work well.
But my concern is, will list 'l' be deconstructed after function return? and then iterator point to nowhere?

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return iter(l)
def main():
for i in test():
print(i)

One more thing: there's no need to call iter() explicitly here. Much
more common than returning an iterator from a function is to return an
iterable. Your code will work exactly the same if you just remove the
iter() call:

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return l
def main():
for i in test():
print(i)

 

[Peter Otten]

take the following as an example, which could work well.
But my concern is, will list 'l' be deconstructed after function return?
and then iterator point to nowhere?

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return iter(l)
def main():
for i in test():
print(i)

One more thing: there's no need to call iter() explicitly here. Much
more common than returning an iterator from a function is to return an
iterable. Your code will work exactly the same if you just remove the
iter() call:

def test():
l = [1, 2, 3, 4, 5, 6, 7, 8]
return l
def main():
for i in test():
print(i)

For that specific client code, yes. In the general case the difference
matters. Iteration over an iterable starts at the beginning while iteration
over an iterator starts at the current item:
.... items = test()
.... print(list(zip(items, items)))
....

def test(): return range(6) # an iterable ....
main() [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5)]
def test(): return iter(range(6)) # an iterator ....
main()

[(0, 1), (2, 3), (4, 5)]