The rap against "while True:" loops

Jorgen Grahn · Oct 14, 2009

.

I was a bit surprised that nobody in this discussion so far bantered
around the phrase "loop invariant", but then I looked in
http://en.wikipedia.org/wiki/Loop_invariant and found it was draped in
so much formalism that it's sure to put off all but the most dedicated
of Computer Science fans.

Haven't read it. But much of the CS parts of the Wikipedia sucks, and
whoever writes there doesn't own the trademark on loop invariants
anyway.

IME, a loop invariant is a simple and useful tool for thinking about
the correctness of code. Class invariants (or whatever they are called)
are even better.

I haven't been in college in 35 years, so
I'll admit to being rusty on this, but as I remember it, any time we
wrote a loop, we were expected to be able to say what the loop
invariant is.

Yes, it's as simple as that.

my_prissy_little_indicator_variable = true
while (my_prissy_little_indicator_variable){
<body>
}
isn't satisfying because it doesn't guard the <body> with any
assurance that the loop invariant will be true before you enter into
that block of code.

Why not? To me, it obviously does.

It would also help if you didn't use intentionally meaningless and
annoying variable names in your examples. In reality you would have a
meaningful expression like "not inputqueue.empty()" or
"time() < deadline" or something.

/Jorgen

Jorgen Grahn · Oct 14, 2009

Except the test at the start is meaningless when it comes to
reading the code and troubleshooting. What counts are
assignments to my_prissy_little_indicator_variable inside the
loop. And those aren't really any easier to spot that "break"
statements.

It's a red herring. A good loop tends to *not* have a boolean
variable as the while ... expression. That smells like flag
programming, and if I cannot come up with anything better that that, I
often prefer a "while 1" with breaks in it.

For a real-life loop, see for example

http://en.wikipedia.org/wiki/Binary_search#Iterative

(except it confuses me because it's a repeat ... until and it's in
Pascal with that quaint 1-based indexing)

/Jorgen

Jorgen Grahn · Oct 14, 2009

Dennis said:
Dennis said:

One thing to note is that "break" ONLY exits the innermost loop --
Ada adds the confusion that one could define a label on the loops, and
have the innermost use
exit outer_label [when condition]

THAT I find scary... Since you have to match the label name to
something that occurs somewhere prior to the "exit", and THEN have to
find the end of that loop.

Click to expand...

But we have exceptions. And I know somebody, in other languages, thinks
it's a Best Practice to avoid using exceptions for flow control.

A lot of C++ programmers think so, and Stroustrup himself says
"exceptions are for exceptional things" or something to that effect.
Is that what you're thinking of?

Thankfully, Stroustrup doesn't use the dreaded phrase "Best Practice",
which as far as I can tell is designed to shut down rational thought
in the audience.

Thankfully, python programmers are less dogmatic, and use whatever makes
sense to use. I hope.

Calling it "dogmatic" is unfair. C++ is very different from Python,
and has a different implementation of exceptions. You also tend to use
the language to solve a different set of problems.

That said, I still don't fully understand the rationale behind that
advice or rule ... so I'm willing to break it, and sometimes I do.

/Jorgen

Raymond Hettinger · Oct 14, 2009

And I know somebody, in other languages, thinks

it's a Best Practice to avoid using exceptions for flow control.

Ah, now we have two code prions in just one thread.
I hope no newbie or supervisor reads this thread and latches
on to those two counter-productive ideas.

ISTM, both ideas are dangerous contagious because they are
true in some limited contexts but useless (and harmful)
when applied to programming in general.

IIRC, the C++ admonition against using exceptions for flow control
was rooted in performance concerns specific to that language and
its compilers. It was not stylistic advice and did not deny that
flow control exceptions could provide elegant solutions to some
programming challenges.

Python's IndexError and KeyError are all about flow control.
The notion is deeply embedded in the language and it would
be a disservice to advise people to not use the language as
designed.

Likewise, the use of "while True" tends to be more important
in Python than in other languages because we can't combine
assignment with a conditional as we can in C. So instead,
we have this idiom:

while True:
s = f.read(blocksize)
if not s:
break
...

Suggested further reading for those who are interested:

"The Little MLer" -- a chunk of this book is devoted to showing
how exceptions can simplify code that would otherwise be
somewhat awkward to express (the remainder of the book is devoted
to thinking about types and how to compose program components).

"Structured Programming with go to Statements" by Donald Knuth
has an in-depth comparative analysis of many different looping
constructs.

Raymond

Paul Rubin · Oct 14, 2009

Raymond Hettinger said:
IIRC, the C++ admonition against using exceptions for flow control
was rooted in performance concerns specific to that language and
its compilers. It was not stylistic advice and did not deny that
flow control exceptions could provide elegant solutions to some
programming challenges.

I've been wondering about that. I know that Java exceptions are
ridiculously expensive but I didn't realize it was so bad with C++.

Python's IndexError and KeyError are all about flow control.
The notion is deeply embedded in the language and it would
be a disservice to advise people to not use the language as
designed.

Well, usually they're wrapped in iterators etc.

So instead, we have this idiom:

while True:
s = f.read(blocksize)
if not s:
break
...

Well,

while iter(lambda: f.read(blocksize), ''):

evolved because of the awkwardness of that idiom...

"The Little MLer" -- a chunk of this book is devoted to showing
how exceptions can simplify code that would otherwise be
somewhat awkward to express (the remainder of the book is devoted
to thinking about types and how to compose program components).

Interesting--I've been wanting to look at that book. I wonder whether
its uses of exceptions could mostly be better handled with coroutines.

"Structured Programming with go to Statements" by Donald Knuth
has an in-depth comparative analysis of many different looping
constructs.

Found some pdf's:

http://scholar.google.com/scholar?cluster=17368311454828547380

Keep in mind that the article is 35 years old though, and is purely
imperative. Lots of stuff done with cockamamie looping constructs is
more cleanly done with Python generators, itertools, higher-order
functions, etc.

Tim Rowe · Oct 14, 2009

2009/10/12 RDrewD said:
I was a bit surprised that nobody in this discussion so far bantered
around the phrase "loop invariant", but then I looked in
http://en.wikipedia.org/wiki/Loop_invariant and found it was draped in
so much formalism that it's sure to put off all but the most dedicated
of Computer Science fans.

I think in this case the loop variant is more use than the loop variant.

Basically, the loop variant is what is true on every pass of the loop.
If you're being formal, you have to show that it's true on entry to
the loop and remains true on every pass of the loop. That means that
on exit from the loop you can guarantee the loop invariant and the
exit condition.

The loop variant is a finite natural number (positive or zero integer)
that is guaranteed to decrease on every pass of the loop. Because a
finite natural number cannot decrease indefinitely, if you can define
a loop variant then you gurantee that the loop will terminate.

Even if you are not being formal, just considering what the loop
variants and invariants can save no end of trouble with tricky loops.

Steven D'Aprano · Oct 15, 2009

Duh. You DO know that 'r' is next to 'e' on the keyboard?

Only on QWERTY keyboards. Not on Dvorak keyboards.

Do you know how to proof-read your writing before hitting send? If not,
please learn. A spell checker may help. If you do know how, if you care
so little for what you write that you can't be bothered, why should
anyone care enough to read what you write? Either way, there's no call
for you to be snarky when people call you on stupid typos. Your mistake
could happen to anyone, but it was still *your* mistake.

[...]

And what will that accomplish? The problem isn't using while True, it's
the fact that you are escaping the loop.

That's not a problem.

Best Practice is to EXIT the loop properly, not escape from it.

A break does exit the loop properly. That's what it is for, it would be a
pretty stupid language that had break exit the loop improperly.

Nobody is forcing you to use break. You can write Pascal in any language
you like.

Steven D'Aprano · Oct 15, 2009

And with enough static analysis to guarantee that the break will be
reached? I think it would be a bit much to expect Python to solve the
halting problem!

That's a stupid objection. Python doesn't guarantee that any of the
following will halt:

for x in iterator:
pass

while flag:
pass

for x in [1, 10, 20, 10**100]:
time.sleep(x)

(Technically, that last one will eventually halt, if you're prepared to
wait long enough... about a billion trillion trillion trillion trillion
trillion trillion trillion years.)

Why should Python make that guarantee about this hypothetical "loop
forever" construct?

Steven D'Aprano · Oct 15, 2009

A lot of C++ programmers think so, and Stroustrup himself says
"exceptions are for exceptional things" or something to that effect. Is
that what you're thinking of?

Thankfully, Stroustrup doesn't use the dreaded phrase "Best Practice",
which as far as I can tell is designed to shut down rational thought in
the audience.

Calling it "dogmatic" is unfair. C++ is very different from Python, and
has a different implementation of exceptions. You also tend to use the
language to solve a different set of problems.

That said, I still don't fully understand the rationale behind that
advice or rule ... so I'm willing to break it, and sometimes I do.

Setting up a try...except block is cheap in Python. According to my
tests, the overhead is little more than that of a single pass statement.

But actually raising and catching the exception is not cheap. If you use
a lot of exceptions for flow control, performance will probably suffer.

In C++, exceptions are expensive, whether you catch one or not.

Also, using exceptions this way is a structured form of GOTO -- it's easy
to abuse and turn it into spaghetti code. Actually, not that easy to
abuse, because you can't jump back into the try block. It's more like a
multi-level break outside of a loop than a general GOTO.

Paul Rubin · Oct 15, 2009

Steven D'Aprano said:
Why should Python make that guarantee about this hypothetical "loop
forever" construct?

It doesn't make much sense for Python as normally practiced.
Termination proofs (aka proofs of progress) are used in formal
verification systems to make sure that the verification logic is
consistent and that type-level inferences are valid. They generally
have very little to do with making sure that the program's actual
running time is bounded by anything reasonable. In fact infinite
loops are permitted in some such systems, but only on infinite data
streams (e.g. the request stream of a web server).

http://en.wikipedia.org/wiki/Total_functional_programming

cites a pretty accessible paper by D. Turner that explains the
idea in more detail.

Mensanator · Oct 15, 2009

Only on QWERTY keyboards. Not on Dvorak keyboards.

Does anyone actually use those things?

Do you know how to proof-read your writing before hitting send?

Yeah. And I missed it. Maybe because my laptop has a few
broken columns of pixels.

If not,
please learn. A spell checker may help.

Not with Google.

If you do know how, if you care
so little for what you write that you can't be bothered, why should
anyone care enough to read what you write?

Conclusion based on false premise.

Either way, there's no call
for you to be snarky when people call you on stupid typos.

I suppose this wasn't snarky:

Your mistake
could happen to anyone, but it was still *your* mistake.

No, it wasn't. You should learn the difference between
an error and a mistake. There's nothing to be learned from
pointing out a typo. Whereas a mistake, such as using
"their" in place of "there" should be pointed out so
as to prevent future occurences.

[...]

And what will that accomplish? The problem isn't using while True, it's
the fact that you are escaping the loop.

Click to expand...

That's not a problem.

It can be.

A break does exit the loop properly. That's what it is for, it would be a
pretty stupid language that had break exit the loop improperly.

Nobody is forcing you to use break.

And no one is forcing the OP to stop using "while True"
even if it is not considered Best Practice.

You can write Pascal in any language you like.

My experience with Pascal is it tended to produce
bullet-proof code. The lessons learned from Pascal
can only make my Python programs better.

Mensanator · Oct 15, 2009

Not on mine -- it's next to 'o' and 'u'. ï¿½ ï¿½Go Dvorak!

Does that mean you think "whilr" is a word?

I don't think anyone's arguing the opposite. ï¿½

I get the impression many people are, in fact,
arguing the opposite.

What I *am* seeing argued
is if it's the only correct way to do it, and that anyone who does it
any other way is a scoundrel and a knave. ï¿½;-)

First of all, _I_ didn't bring up the concept of Best Practice,
nor have I insisted Best Practice means there is only one
correct way to do something. I interpret it as meaning there
may be many correct ways to do something, but of those,
this is the best way, barring special circumstances.

For what it's worth, most of my loops run to completion, with no sign of
a break anywhere. ï¿½Some have a break, and use it. ï¿½Some, even, (dare I
say it?) use break *and* else! ï¿½

Breaks can be used properly, but it's easy to use them
improperly. How many of your loops start "while True"?

Tim Rowe · Oct 16, 2009

Not to what I thought was being proposed -- it seemed to make the
break mandatory, not merely the only clean way to exit.

Tim Rowe · Oct 16, 2009

2009/10/15 Steven D'Aprano said:
Setting up a try...except block is cheap in Python. According to my
tests, the overhead is little more than that of a single pass statement.

But actually raising and catching the exception is not cheap. If you use
a lot of exceptions for flow control, performance will probably suffer.

In C++, exceptions are expensive, whether you catch one or not.

Also, using exceptions this way is a structured form of GOTO -- it's easy
to abuse and turn it into spaghetti code. Actually, not that easy to
abuse, because you can't jump back into the try block. It's more like a
multi-level break outside of a loop than a general GOTO.

I don't think it's *only* the performance thing, it's also clarity.
The understood meaning of throwing an exception is to say "something
happened that shouldn't have". If one uses it when something has
happened that *should* have, because it happens to have the right
behaviour (even if the overhead doesn't matter), then one is
misrepresenting the program logic.

Aahz · Oct 16, 2009

The understood meaning of throwing an exception is to say "something
happened that shouldn't have". If one uses it when something has
happened that *should* have, because it happens to have the right
behaviour (even if the overhead doesn't matter), then one is
misrepresenting the program logic.

Except, of course, that your "understood meaning" is wrong for Python.
Perhaps you should go look up StopIteration.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"To me vi is Zen. To use vi is to practice zen. Every command is a
koan. Profound to the user, unintelligible to the uninitiated. You
discover truth everytime you use it." (e-mail address removed)

Bearophile · Oct 16, 2009

Paul Rubin:

http://scholar.google.com/scholar?cluster=17368311454828547380

Keep in mind that the article is 35 years old though, and is purely
imperative. Lots of stuff done with cockamamie looping constructs is
more cleanly done with Python generators, itertools, higher-order
functions, etc.

That's a famous and very good paper, a good read for people that like
to program.
The Example1 and Example2 can be rewritten in several different ways,
but several compilers today are not able to perform such optimizations
yet, so what Knuth has written there are still among the faster ways
to implement that algorithm.

Bye,
bearophile

Russ P. · Oct 16, 2009

I'm coaching a group of biologists on basic Python scripting. One
of my charges mentioned that he had come across the advice never
to use loops beginning with "while True". Of course, that's one
way to start an infinite loop, but this seems hardly a sufficient
reason to avoid the construct altogether, as long as one includes
an exit that is always reached. (Actually, come to think of it,
there are many situations in which a bona fide infinite loops
(typically within a try: block) is the required construct, e.g.
when implementing an event loop.)

I use "while True"-loops often, and intend to continue doing this
"while True", but I'm curious to know: how widespread is the
injunction against such loops? Has it reached the status of "best
practice"?

Never, ever use "while True". It's an abomination. The correct form is
"while 1".

But seriously, folks ... "while condition" is nice when the desired
break is at the beginning or end of the block, but otherwise it forces
unnecessary contortions that can hamper readability.

Steven D'Aprano · Oct 17, 2009

I don't think it's *only* the performance thing, it's also clarity. The
understood meaning of throwing an exception is to say "something
happened that shouldn't have". If one uses it when something has
happened that *should* have, because it happens to have the right
behaviour (even if the overhead doesn't matter), then one is
misrepresenting the program logic.

No, you have a fundamental misunderstanding. They're called exceptions,
not errors, because they represent exceptional cases. Often errors are
exceptional cases, but they're not the only sort of exceptional case.

Python uses exceptions for flow control: e.g. for-loops swallow
StopIteration or IndexError to indicate the end of the loop. In the
context of a for-loop, StopIteration or IndexError doesn't represent an
error. It doesn't represent an unexpected case. It represents an
expected, but exceptional (special) case: we expect that most sequences
are finite, and it is normal to eventually reach the end of the sequence,
after which the loop must change behaviour.

Similarly, it's hardly an *error* for [1, 2, 3].index(5) to fail -- who
is to say that the list is supposed to have 5 in it? ValueError (a
slightly misleading name in this situation) is used to indicate an
exceptional, but not unexpected, occurrence.

Likewise, KeyboardInterrupt is used to allow the user to halt processing;
SystemExit is used to shut down the Python virtual machine; and warnings
are implemented using exceptions. There may be others among the built-ins
and standard library, but even if there aren't, there is plenty of
precedence for us to do the same.

Tim Rowe · Oct 17, 2009

2009/10/17 Steven D'Aprano said:
No, you have a fundamental misunderstanding. They're called exceptions,
not errors, because they represent exceptional cases. Often errors are
exceptional cases, but they're not the only sort of exceptional case.

The whole reason for the mechanism, across all languages that have it,
is to deal with situations that you don't know how to deal with
locally. That's why they have the overhead that they do.

Python uses exceptions for flow control:

Yes, and in some cases I think that's a serious language wart. Not
enough to put me off the language, but a serious wart nevertheless.

Similarly, it's hardly an *error* for [1, 2, 3].index(5) to fail -- who
is to say that the list is supposed to have 5 in it? ValueError (a
slightly misleading name in this situation) is used to indicate an
exceptional, but not unexpected, occurrence.

That one is, I think, a legitimate use of an exception. The result
returned by index is defined if the index is in bounds. If not, index
doesn't know whether it was supposed to be in bounds or not, and so
can't handle the case locally. It could suggest an error or merely
(IMHO) poor programming. Because index cannot know what the proper
action is, an exception is the appropriate response.

Likewise, KeyboardInterrupt is used to allow the user to halt processing;
SystemExit is used to shut down the Python virtual machine; and warnings
are implemented using exceptions.

Again, I think it's fair to treat a program being killed from outside
as an exception as far as the program is concerned. They're not things
the program has brought about, and the original handling code has no
way of knowinng what response is appropriate. Same with warnings; they
*probably* shouldn't happen but only the application programmer can
know whether they should or not.

The point is that an exception causes a change in program flow, so of
course they're used for flow control. It's what they do. The question
is in what cases it's appropriate to use them.

Aahz · Oct 17, 2009

The point is that an exception causes a change in program flow, so of
course they're used for flow control. It's what they do. The question
is in what cases it's appropriate to use them.

Standard Python idiom:

try:
d[key] += value
except KeyError:
d[key] = value

Maybe you need to re-think "appropriate".
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"To me vi is Zen. To use vi is to practice zen. Every command is a
koan. Profound to the user, unintelligible to the uninitiated. You
discover truth everytime you use it." (e-mail address removed)

The True Religion	0	Dec 1, 2009
noob question on loops	16	Jan 21, 2013
Noob: Loops and the 'else' construct	7	Oct 19, 2007
The True Religion	0	Dec 1, 2009
While(1) or for(;;) for infinite loops?	106	Dec 4, 2009
a potential pep to extend the syntax of for loops	9	Mar 9, 2009
How to test a URL request in a "while True" loop	11	Dec 30, 2009
while-loops enter the last time after condition is filled?	4	Apr 6, 2008

The rap against "while True:" loops

Jorgen Grahn

Jorgen Grahn

Jorgen Grahn

Raymond Hettinger

Paul Rubin

Tim Rowe

Steven D'Aprano

Steven D'Aprano

Steven D'Aprano

Paul Rubin

Mensanator

Mensanator

Tim Rowe

Tim Rowe

Aahz

Bearophile

Russ P.

Steven D'Aprano

Tim Rowe

Aahz

Ask a Question

Similar Threads

Members online

Forum statistics

Latest Threads