Should I use "if" or "try" (as a matter of speed)?

[Thorsten Kampe]

* John Roth (2005-07-09 21:48 +0100)

It depends on what you're doing, and I don't find a "one size fits all"
approach to be all that useful.

I think, it's a common opinion in the Python community (see for
instance "Python in a Nutshell") that EAFP is the Pythonic way to go
and - except in very rare cases - much preferred to LBYL.

Speed considerations and benchmarking should come in after you wrote
the program. "Premature optimisation is the root of all evil" and
"first make it work, then make it right, then make it fast" (but only
if it's not already fast enough) - common quotes not only with Python
developers.

 

[Jorgen Grahn]

Joel Spolsky might be a great C++ programmer, and his advice on user
interface design is invaluable, but Python is not C++ or Java, and his
arguments about exceptions do not hold in Python.

Of course, his arguments do not even "hold" in C++ or Java, in the sense
that everyone should be expected to accept them. Most C++ programmers would
find his view on exceptions slightly ... exotic.

He has a point though: exceptions suck. But so do error codes. Error
handling is difficult and deadly boring.

(Then there's the debate about using exceptions for handling things that
aren't really errors, and what the term 'error' really means ...)

/Jorgen

 

[Aahz]

Roy, I know you actually know this stuff, but for the benefit of
beginners....

3) In some cases, they can lead to faster code. A classic example is
counting occurances of items using a dictionary:

count = {}
for key in whatever:
try:
count[key] += 1
except KeyError:
count[key] = 1

compared to

count = {}
for key in whatever:
if count.hasKey(key):
count[key] += 1
else:
count[key] = 1

Except that few would write the second loop that way these days::

for key in whatever:
if key in count:
...

Using ``in`` saves a bytecode of method lookup on ``has_key()`` (which is
the correct spelling). Or you could choose the slightly more convoluted
approach to save a line::

for key in whatever:
count[key] = count.get(key, 0) + 1

If whatever had ``(key, value)`` pairs, you'd do::

key_dict = {}
for key, value in whatever:
key_dict.setdefault(key, []).append(value)

 

[Roy Smith]

Using ``in`` saves a bytecode of method lookup on ``has_key()`` (which is
the correct spelling).

You are right. My example is somewhat out of date w/r/t newer language
features, and writing hasKey() instead of has_key() was just plain a
mistake. Thanks for the corrections.

 

[Ron Adam]

Optimize for readability and maintainability first. Worry about speed
later.

Yes, and then...

If it's an application that is to be used on a lot of computers, some of
them may be fairly old. It might be worth slowing your computer down
and then optimizing the parts that need it.

When it's run on faster computers, those optimizations would be a bonus.

Cheers,
Ron

 

[Sybren Stuvel]

Steve Juranich enlightened us with:

Without fail, when I start talking with some of the "old-timers"
(people who have written code in ADA or Fortran), I hear the same
arguments that using "if" is "better" than using "try".

Then here is the counter argument:

- Starting a 'try' is, as said somewhere else in this thread, a single
bytecode, hence simple.

- Those old-timers will probably check the data before they pass it to
a function. Because their function has to be stable and idiot-proof
as well, the function itself performs another check. Maybe the data
is passed even further down a function chain, making checks before
and after the function calls. After all, we want to be sure to only
call a function when we're sure it won't fail, and every function
has to gracefully bail when it's being passed bad data.

This means that there will be a _lot_ of checks in the code. Now
compare this by setting up a single try-block, and catching the
exception at the proper place in case it's being thrown.

The important part is this: those old-timers' "if" statements are
always executed - it doesn't matter whether the data is correct or
incorrect. The "heavy" part of exceptions only comes into play when
the data is incorrect, which by proper design of the program shouldn't
happen often anyway.

As far as I see it, try/except blocks are "cheaper" than "if"
statements.

Sybren

 

[Dark Cowherd]

def joels_function(args):
error_result = 0
good_result = None
process(args)
if error_condition():
error_result = -1 # flag for an error
elif different_error_conditon():
error_result = -2
else:
more_processing()
if another_error_conditon():
error_result = -3
do_more_work()
good_result = "Success!"
if error_result != 0:
return (False, error_result)
else:
return (True, good_result)


and then call it with:

status, msg = joels_function(args)
if status == False:
print msg
# and fail...
else:
print msg
# and now continue...


This is how I would write it in Python:

def my_function(args):
process(args)
if error_condition():
raise SomeError("An error occurred")
elif different_error_conditon():
raise SomeError("A different error occurred")
more_processing()
if another_error_conditon():
raise SomeError("Another error occurred")
do_more_work()
return "Success!"

and call it with:

try:
result = my_function(args)
print "Success!!!"
except SomeError, msg:
print msg
# and fail...
# and now continue safely here...


In the case of Python, calling a function that may raise an exception is

I tend to use exceptions, but I think Joel has a point.

Taking the example code that you have given above.

Let us assume that somebody else is using my_function and DOES NOT
write a try except block.

This code will run fine except, when the exception is thrown and it
will suddenly pop up in some other error handler which may not be
handling the situation correctly. You have to plan and create a series
of errorhandling classes to handle such situations.

However Joels_function forces the caller to write some kind of error
handler. If he doesnt write the program will not run.

After reading that I have been giving this option some thought. The
nice thing about Python is I can easily return tuples. In C++ you have
to jump through hoops because you cant return two values easily.

DarkCowherd

 

[corey.coughlin]

It really does depend. For instance, some other programmers where I
work came up with a way to represent a hierarchical, somewhat random
data set by creating each object and then adding attributes to those
for each subobject, and so on down the tree. However, you could never
really be sure that the object you wanted was really there, so for
every access call they just wrapped it in a try ...except loop. Now
that may seem like a good way to go, but when I rewrote some code to
use hasattr() instead, it ran a lot faster. So yeah, exceptions can be
handy, but if you code requires exception handling for everything, you
may want to rethink things.

 

[Steven D'Aprano]

I tend to use exceptions, but I think Joel has a point.

Joel being "Joel On Software" Joel.

Taking the example code that you have given above.

Let us assume that somebody else is using my_function and DOES NOT
write a try except block.

Then, like any other piece of Python code, if it raises an
exception the exception will be propagated and the interpreter will halt.
That is the correct behaviour. On crash, halt.

This code will run fine except, when the exception is thrown and it will
suddenly pop up in some other error handler which may not be handling
the situation correctly. You have to plan and create a series of
errorhandling classes to handle such situations.

Why? You should be aiming to write correct code that doesn't produce
random exceptions at random times, rather than wrapping everything in
exception handlers, which are wrapped in exception handlers, which are
wrapped in exception handlers...

Exceptions are useful, but you shouldn't just cover them up when they
occur. The idea of exceptions is to recover from them safely, if you can,
and if not, bring the program to a halt safely.

Dealing with exceptions is no different from dealing with data. If there
is data you don't deal with correctly ("what if the user passes -1 as an
argument, when we expect an integer larger than zero?") you have a bug. If
there is a exception you don't deal with correctly ("what if this function
raises ZeroDivisionError when we expect a ValueError?"), you have an
uncaught exception which will halt your program, exactly as exceptions are
designed to do.

However Joels_function forces the caller to write some kind of error
handler. If he doesnt write the program will not run.

And Python gives you that error handler already built in. It is called
try...except, and if you don't write it, and your program raises an
exception, your program will stop.

After reading that I have been giving this option some thought. The nice
thing about Python is I can easily return tuples.

Yes. But returning a tuple (success, data) where the meaning of data
varies depending on whether success is True or False is an abuse of the
language. I can think of a few places where I might do that, but not very
many.

The problem with this tuple (success, data) idiom is that it leads to
duplicated code. All your functions end up looking like this:

def func(data):
if data[0]:
result = do_real_work(data[1])
if error_condition:
return (False, msg)
return (True, result)
else:
return data

But then you want to make do_real_work bulletproof, don't you? So
do_real_work ends up filled with error checking, just in case the calling
function passes the wrong data. So you end up losing all the advantages of
wrapping your data in a tuple with a flag, but keeping the disadvantages.

In any case, writing masses of boilerplate code is a drain on programmer
productivity, and a frequent cause of bugs. Boilerplate code should be
avoided whenever you can.

 

[Edvard Majakari]

Speed considerations and benchmarking should come in after you wrote
the program. "Premature optimisation is the root of all evil" and
"first make it work, then make it right, then make it fast" (but only
if it's not already fast enough) - common quotes not only with Python
developers.

Just a minor note: regarding quote

"first make it work, then make it right, then make it fast"

Shouldn't one avoid doing it the wrong way from the very beginning? If you
make it "just work" the first time, you'll probably use the old code later on
because "functionality is already there" and temptatation to build on probably
relatively bad architecture can be too strong.

How about

First make it work (but avoid ad-hoc designs), then make it right, then make
it fast

Of course, such emphasis doesn't go well with classic idioms..

(yeah, programmer's block at the moment: I should clean up a 120+ -line
if-elif-elif-elif... else -block which tests a single variable and calls
different methods with variable number of parameters depending on the value of
the variable - guess I should apply command pattern or similar...)

--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!

$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";

 

[Steven D'Aprano]

Just a minor note: regarding quote

"first make it work, then make it right, then make it fast"

Shouldn't one avoid doing it the wrong way from the very beginning? If you
make it "just work" the first time, you'll probably use the old code later on
because "functionality is already there" and temptatation to build on probably
relatively bad architecture can be too strong.

How about

First make it work (but avoid ad-hoc designs), then make it right, then make
it fast

Optimizing sometimes means refactoring your code. Sometimes it even means
throwing it away and starting again.

However, your point to bring us back to a discussion held on this
newsgroup not long ago, about whether or not it was good for computer
science students to learn how to program in low-level languages like C so
as to avoid silly mistakes like this:

result = ""
for s in really_big_list_of_strings:
result = result + s
return result

instead of just "".join(really_big_list_of_strings).

The first method is slow, but you might not know that unless you have some
understanding of the implementation details of string concatenation.

My opinion is, no, you don't need to be a C programmer, or an assembly
programmer, or a hardware level physicist who understands NAND gates, but
it is very useful to have some understanding of what is going on at the
low-level implementation.

The way I see it, programmers need to be somewhat aware of the eventual
optimization stage in their program, so as to avoid poor design choices
from the start. But you can't always recognise poor design up front, so
even more important is careful encapsulation and design, so you
can make significant implementation changes without needing to throw away
your work. (Well, that's the theory.)

 

[Peter Hansen]

"first make it work, then make it right, then make it fast"

Shouldn't one avoid doing it the wrong way from the very beginning? If you
make it "just work" the first time, you'll probably use the old code later on
because "functionality is already there" and temptatation to build on probably
relatively bad architecture can be too strong.

The expression describes (most recently, if not originally) the practice
in Test-Driven Development (TDD) of making your code pass the test as
quickly as possible, without worrying about how nice it is.

The "right" part doesn't refer to correctness, but to structure, style,
readability, and all those other nice things that an automated test
can't check. You aren't doing it "wrong" at first, just expediently.

And it really does make sense, because at that early stage, you aren't
even absolutely certain that your test is entirely correct, so making
your code a paragon of elegance is a potential waste of time, and
distracting. Once you've been able to pass that test (and all the
others, since you have to make sure all previous tests still pass as
well), then and only then is it sensible -- and required! -- to refactor
the code to make it elegant, concise, clean, etc.

Of course, your point about temptation is sound. Extreme Programming
tries to avoid that problem partly by pairing programmers together, and
it is the responsibility of both partners to encourage^H^H^H^H^H insist
that the refactor "make it right" stage must occur _now_, before we
check the code in. If you skip this step, you're failing to be an agile
programmer, and your code base will become a tar pit even more quickly
than it would in a traditional (non-agile) project...

-Peter

 

[Dark Cowherd]

I use Delphi in my day job and evaluating and learning Python over the
weekends and spare time. This thread has been very enlightening to me.

The comments that Joel of Joel on Software makes here
http://www.joelonsoftware.com/items/2003/10/13.html was pretty
convincing. But I can see from the comments made by various people
here that since Python uses Duck typing and encourages typless styles
of functions exceptions may actually be the better way to go.

But one advise that he gives which I think is of great value and is
good practice is
"Always catch any possible exception that might be thrown by a library
I'm using on the same line as it is thrown and deal with it
immediately."

DarkCowherd

 

[Christopher Subich]

But one advise that he gives which I think is of great value and is
good practice is
"Always catch any possible exception that might be thrown by a library
I'm using on the same line as it is thrown and deal with it
immediately."

That's fine advice, except for when it's not. Consider the following code:

try:
f = file('file_here')
do_setup_code
do_stuff_with(f)
except IOError: # File doesn't exist
error_handle

To me, this code seems very logical and straightfoward, yet it doesn't
catch the exception on the very next line following its generation. It
relies on the behavior of the rest of the try-block being skipped -- the
"implicit goto" that Joel seems to loathe. If we had to catch it on the
same line, the only alternative that comes to mind is:
try: f=file('file_here')
except IOError: #File doesn't exist
error_handle
error_flag = 1
if not error_flag:
do_setup_code
do_stuff_with(f)

which nests on weird, arbitrary error flags, and doesn't seem like good
programming to me.

 

[Thomas Lotze]

try:
f=file('file_here')
except IOError: #File doesn't exist
error_handle
error_flag = 1
if not error_flag:
do_setup_code
do_stuff_with(f)

which nests on weird, arbitrary error flags, and doesn't seem like good
programming to me.

Neither does it to me. What about

try:
f=file('file_here')
except IOError: #File doesn't exist
error_handle
else:
do_setup_code
do_stuff_with(f)

(Not that I'd want to defend Joel's article, mind you...)

 

[Roy Smith]

try:
f = file('file_here')
do_setup_code
do_stuff_with(f)
except IOError: # File doesn't exist
error_handle

It's also a good idea to keep try blocks as small as possible, so you
know exactly where the error happened. Imagine if do_setup_code or
do_stuff_with(f) unexpectedly threw an IOError for some reason totally
unrelated to the file not existing.

 

[Mike Meyer]

But one advise that he gives which I think is of great value and is
good practice is
"Always catch any possible exception that might be thrown by a library
I'm using on the same line as it is thrown and deal with it
immediately."

Yuch. That sort of defeats the *purpose* of exceptions in Python:
letting you get on with the coding, and dealing with the errors when
it's convenient. Consider:

try:
out = file(datafile, "wb")
out.write(genData1())
out.write(genData2())
out.write(genData3())
except IOError, msg:
print >>sys.stderr, "Save failed:", msg
if os.path.exists(datafile):
os.unlink(datafile)

I don't even want to *think* writing the try/except clause for each
line. It reminds me to much of:

if (!(out = open(datafile, "w"))) {
/* handle errors */
return ;
}
if (write(out, genData1()) <0) {
/* handle errors */
return ;
}
etc.

Generally, I treat exceptions as exceptional. I catch the ones that
require something to be changed to restore the program or environment
to a sane state - and I catch them when it's convenient. The rest I
catch globally and log, so I can figure out what happened and do
something to prevent it from happening again.

<mike

 

[Dark Cowherd]

OK, I can see that the Python way of doing things is very different.
However I think Roy made a very pertinent point
"Imagine if do_setup_code or
do_stuff_with(f) unexpectedly threw an IOError for some reason totally
unrelated to the file not existing."
This is the kind of situation that the rule 'catch it on the next
line' is trying to avoid

What I didnt realise till I read Thomas comment is that the try except
had an else clause. This is nice.

But seriously, if you expected to write reasonably large business
applications with multiple people in the team and teams changing over
time what would you give as a guideline for Error handling

DarkCowherd

 

[Christopher Subich]

Neither does it to me. What about

try:
f=file('file_here')
except IOError: #File doesn't exist
error_handle
else:
do_setup_code
do_stuff_with(f)

(Not that I'd want to defend Joel's article, mind you...)

That works. I'm still not used to having 'else' available like that. I
wonder how Joel advocates managing in C++-likes that don't have a
try/catch/else semantic.

 

[Edvard Majakari]

My opinion is, no, you don't need to be a C programmer, or an assembly
programmer, or a hardware level physicist who understands NAND gates, but
it is very useful to have some understanding of what is going on at the
low-level implementation.

Yes, I fully agree: in the example presented, it is sufficient to understand
that string concatenation is (relatively) expensive. Yet I'd emphasize that
most often speed is improved by better algorithms, not by low-level
optimisations and language-specific features (if speed is even an issue, that
is).

The way I see it, programmers need to be somewhat aware of the eventual
optimization stage in their program, so as to avoid poor design choices
from the start. But you can't always recognise poor design up front, so
even more important is careful encapsulation and design, so you
can make significant implementation changes without needing to throw away
your work. (Well, that's the theory.)

So true, extra emphasis on encapsulation and independence. Even seasoned
professionals fail to create dazzling products at version 1.0. Good component
design is crucial because you eventually want to do major rewrites later.

--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!

$_ = '456476617264204d616a616b6172692c20612043687269737469616e20'; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60281449,'es'),2,4),"\n";