efficiency of range() and xrange() in for loops

[Felipe Almeida Lessa]

Em Qua, 2006-04-05 às 19:15 -0700, Alex Martelli escreveu:

Then, the speediest approach may be completely different:

import itertools

for i in itertools.repeat(None, N):
...


Remember, when you're thinking "blazing speed", think itertools.
Hmmm...

setup='import itertools').repeat(3, 1000))
5.6378099918365479
setup='import itertools').repeat(3, 1000))
0.0065851211547851562

It *is* faster, but by a small margin. Considering that it is IMHO less
readable, I'd use it just for *very* big loops, or in functions in
hotspots.

Anyway, it's always good to remember about itertools, it's a great
module that some people don't even know.

Cheers,

 

[Azolex]

I guess I should laugh. :-/


When you read my original articles, did *you* think I was proposing that
range() be changed to always return an iterator? I thought what I wrote
was pretty clear...

I just re-read your original post, and in fact this appears to have been
your drift with "is there any chance this optimization could be added
?". Anyway that wasn't really controversial (since indeed, as was noted
by John Salerno who first replied to you, it is slated for py3k - and
with good reason).

 

[Carl Banks]

I wondered if the Python
compiler could, as a special case, turn:

for i in range(n)

into compiled code equivalent to

for i in itr

where "itr" is a lightweight iterator that returns the same values as
iter(range(n)).

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time. There might even be a good reason to do so.... Point is,
optimizing the call to range can break compatibility even in the for
loop.


Carl Banks

 

[Fredrik Lundh]

Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object.

so you're saying that the value of i isn't an integer object?

if you don't need the index value at all, itertools.repeat is sometimes
faster than xrange.

for i in range(10**6):
pass

and produced code equivalent to this:

for i in iter(range(10**6))
pass

How would this break stuff?

how would that speed anything up?

</F>

 

[Georg Brandl]

No, I hadn't. I presume these could all be defined.

How would xrange(100).remove(1) work?

Georg

 

[Paddy]

I wondered at the tone of some of the replies, re-read the repliess and
your original message. On first readings ithought that your original
message was OK and that the replies were a bit 'strong' . On second
reading I thought that the original could be interpreted a little less
nicely, but I had to go looking.

Please don't be put off using this news group. I guesss we all have to
be extra careful about tone when no one can see your facial expressions


- Pad.

P.S. I was having problems with the Google server - sorry if this is
replicated.

 

[Steve R. Hastings]

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time.

Ah! Of course.

Thank you very much for explaining this.

 

[Fredrik Lundh]

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time. There might even be a good reason to do so.... Point is,
optimizing the call to range can break compatibility even in the for
loop.

that could of course be addressed by turning for-in into a special method
on the range object... map

for variable in expression:
block

to

_value = expression
try:
_for = _value.__for__
except AttributeError:
_for = iter(_value).__for__
_for(variable, block)

and tweak as necessary.

</F>

 

[Alan Morgan]

How would xrange(100).remove(1) work?

One way is by first converting the xrange to a list. If we think of
the xrange as an efficient and space lean way to store certain types
of lists then it isn't unreasonable to return a regular list when
the conditions no longer hold.

Or extend the xrange object to allow multiple ranges and return the
union of xrange(0,1) and xrange(2,100). No reason why you can't
define the whole range of list operations over xrange and still leave
it as a nice, lazy list (other languages do). It's possible that no
one needs it enough to make it worthwhile (although having full fledged
lazy structures can really helpful when you need them) but it could
be done.

Alan

 

[Fredrik Lundh]

One way is by first converting the xrange to a list. If we think of
the xrange as an efficient and space lean way to store certain types
of lists then it isn't unreasonable to return a regular list when
the conditions no longer hold.

do you understand Python's object model to be able to suggest how such
a conversion could be done, or is that someone else's problem ?

</F>