Exception Handling Practices / Patterns

[snarf]

Greetings,

As I tread through my journey of OO I am trying to determine if there is a good approach for exception handling within classes.

From my readings and gatherings - it seems I have found a common theme, but I am trying to solicit from the experts.

Here is what I have found (I may be restating the obvious so please forgive me in advance):

* Seems like exception handing within Classes is largely avoided and is typically only used when calling external libraries.
* Try/Except type statements seem to be used more within modules, main functions, wrapper scripts.
* Classes should be coded in a way that exceptions
* Better to never write your own exceptions (unless you absolutely have to).
* Using Exception is typically a bad. More specific the better.
* Exceptions should never fail silently. (Should exceptions always be logged?)

Best site I have found for exceptions (hopefully this helps someone):
* http://c2.com/cgi/wiki?ExceptionPatterns


I'd be interested in hearing others thoughts on this topic with regards to best practices for when to use exceptions, and when to avoid using exceptions.

Thank you in advance!

 

[Dave Angel]

Greetings,

As I tread through my journey of OO I am trying to determine if there is a good approach for exception handling within classes.

From my readings and gatherings - it seems I have found a common theme, but I am trying to solicit from the experts.

Here is what I have found (I may be restating the obvious so please forgive me in advance):

* Seems like exception handing within Classes is largely avoided and is typically only used when calling external libraries.
* Try/Except type statements seem to be used more within modules, main functions, wrapper scripts.

Exceptions are used when useful. I don't see any bias towards any one
location.

* Classes should be coded in a way that exceptions

You seem to be missing the last part of this sentence.

* Better to never write your own exceptions (unless you absolutely have to).

If you mean to avoid writing exception classes, then I say nonsense.
Just derive them from the closest meaningful exception class, so that a
user can combine handlers when reasonable.

* Using Exception is typically a bad. More specific the better.

If you mean in an except statement, then I'd agree.

* Exceptions should never fail silently. (Should exceptions always be logged?)

Exceptions should be caught if you can handle them, or if you need to
convert them to a different exception that someone further up the stack
can handle. Sometimes handling means do nothing.

Best site I have found for exceptions (hopefully this helps someone):
* http://c2.com/cgi/wiki?ExceptionPatterns

But that's for Java. java is not C++, and neither is it Python. For
one thing, Python exception overhead is deliberately much less, and they
are used more freely.

Notice that exceptions are used to terminate for loops, and that's a
*normal* exit to the loop. They also appear in other places under the
covers. Don't be afraid of them.

 

[Steven D'Aprano]

On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:

[...]

* Seems like exception handing within Classes is largely avoided and is
typically only used when calling external libraries.

There is certainly no rule "avoid exceptions inside classes". Methods
often raise exceptions to signal an error, e.g.:

"My string".index("spam")


Less common, methods can raise exceptions as part of their flow control.
The most obvious example is StopIteration, used by iterators and often by
__iter__ or next methods.

* Try/Except type
statements seem to be used more within modules, main functions, wrapper
scripts.

It depends on whose code you are reading. I don't write a lot of classes,
but when I do, I often use try...except inside them.

If try...except gets used more frequently in module's top level, it is
because the sort of things that you do at the top level often needs
exception handling. For example, you might have a fallback module:

try:
import this_module
except ImportError:
import that_module as this_module


You will very rarely see that inside a class, since you very rarely
import modules inside a class.

* Classes should be coded in a way that exceptions

I think you forgot to finish the sentence.

* Better to
never write your own exceptions (unless you absolutely have to).

That depends.

On the one hand, nobody wants a million different exception types. On the
other hand, nobody wants just *one* exception type, and no way to
distinguish between different kinds of errors. Somewhere between one and
one million is an appropriate number of exception types.

The right answer is to be conservative about creating new exceptions, but
don't be scared to create one when you need one.

But when you do, it is often better to subclass from an appropriate built-
in exception like ValueError or TypeError or similar, than to subclass
from Exception itself.

* Using
Exception is typically a bad. More specific the better.

Yes, you should always try to catch the specific exceptions you care
about:


# Best
except ValueError, OverflowError, ZeroDivisionError:


# Not so good
except Exception:


# Worst
except:


Don't use the last one, except maybe in the interactive interpreter,
since it will catch *everything*, even exceptions that probably shouldn't
be caught like KeyboardInterrupt.

* Exceptions
should never fail silently. (Should exceptions always be logged?)

Certainly not. Exceptions should fail silently if you don't care about
them. For example, when connecting to a website, there are many temporary
errors that can occur. The best way to handle them is to catch the
exception, sleep for a little while, then try again. You need only care
if repeated attempts to connect, with larger and larger sleeps, continue
to fail.

Of course, you might have a debug mode that logs all of these, but if
your web browser logged every single time a webpage was slow to respond,
you would soon run out of disk space

*Errors* should never fail silently, unless explicitly silenced. But an
error isn't an error if you don't care about it, and an exception is not
necessarily an error.

This is an error, because converting a number to uppercase cannot
possibly mean anything:

mystring = 42
mystring.upper()


This is not necessarily an error, since "the list is empty" could be a
legitimate situation:

mylist = []
first = mylist[0]

In this case, it may be appropriate to catch the exception, and either
silently swallow it, or do something else.

Best site I have found for exceptions (hopefully this helps someone): *
http://c2.com/cgi/wiki?ExceptionPatterns

I haven't read that page for a long time, but as I recall the c2.com
website, a lot of the ideas there are better suited to Java and C/C++
(and occasionally Lisp) rather than Python. But still, a valuable (if
often confusing) resource.

I'd be interested in hearing others thoughts on this topic with regards
to best practices for when to use exceptions, and when to avoid using
exceptions.

The try part of a try...except is *very* fast to set up. It's about as
fast as a "pass" (do nothing), so it has little overhead.

On the other hand, actually *catching* an exception is quite heavy. So
code that catches lots and lots of exceptions may be slow. In that case,
it may be faster to "look before you leap" and test ahead of time:

# this is faster if most lists are not empty
try:
process(mylist[0])
except IndexError:
handle_empty_list()

# this is faster if many lists are empty
if mylist:
process(mylist[0])
else:
handle_empty_list()


Only profiling your data can tell you which you should use.


On the other hand, here you should *always* use try...except:

try:
myfile = open("name")
except IOError:
handle_error()


Because this code is wrong:

if os.path.exists("name"):
myfile = open("name")
else:
handle_error()


It's wrong for a couple of reasons:

- just because the file exists, doesn't mean you can open it;

- even if the file exists when you call the os.path.exists function,
doesn't mean it will still exist a millisecond later when you try to open
it.



Hope this is helpful,

 

[MRAB]

Yes, you should always try to catch the specific exceptions you care
about:


# Best
except ValueError, OverflowError, ZeroDivisionError:

That should be:

except (ValueError, OverflowError, ZeroDivisionError):

# Not so good
except Exception:


# Worst
except:

[snip]

 

[Terry Reedy]

On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:

[...]

* Seems like exception handing within Classes is largely avoided and is
typically only used when calling external libraries.

There is certainly no rule "avoid exceptions inside classes". Methods
often raise exceptions to signal an error, e.g.:

"My string".index("spam")

The rule only makes sense if it referring to try: except: in top-level
class code, outside of def statements. Even then, it is wrong.

....

# Worst
except:

Don't use the last one, except maybe in the interactive interpreter,

Stick with never. "except:" means the same thing as "except
BaseException:", except that the latter indicates a deliberate choice
rather than an indication of carelessness or laziness.

A bare except: is a disservice to the next maintainer of the code.

since it will catch *everything*, even exceptions that probably shouldn't
be caught like KeyboardInterrupt.

In Idle, when you type 'expression(' and hesitate, Idle tries to
evaluate 'expression' to a function, behind the scenes, in order to
provide a calltip with the function signature. Any error in
'eval(expression)' should be caught and ignored so the user can continue
typing.

This is one place where the original authors were too specific. They
only caught NameError and AttributeError, when those were not the only
possible eval errors in practice. The result was that people would
occasionally type '(' and see idle quit.

idlelib .py files have about 20 bare 'except:'s, which I will try to
fill out when I have reviewed the try part and understand what should be
caught. It would be easier for me to read the code if the original
authors had added their best guess as to what should be expected and caught.

 

[Steven D'Aprano]

Stick with never. "except:" means the same thing as "except
BaseException:", except that the latter indicates a deliberate choice
rather than an indication of carelessness or laziness.

A bare except: is a disservice to the next maintainer of the code.

Do you know anyone who maintains code typed in the interactive
interpreter?

 

[frank.ruiz]

Appreciate the feedback. I was hoping to get as much perspective as possible.

 

[frank.ruiz]

Hi Steven,

Yea this is great. Thanks for the feedback.

On Fri, 23 Aug 2013 22:25:55 -0700, snarf wrote:



[...]

* Seems like exception handing within Classes is largely avoided and is

typically only used when calling external libraries.

There is certainly no rule "avoid exceptions inside classes". Methods

often raise exceptions to signal an error, e.g.:



"My string".index("spam")





Less common, methods can raise exceptions as part of their flow control.

The most obvious example is StopIteration, used by iterators and often by

__iter__ or next methods.

* Try/Except type

statements seem to be used more within modules, main functions, wrapper

scripts.

It depends on whose code you are reading. I don't write a lot of classes,

but when I do, I often use try...except inside them.



If try...except gets used more frequently in module's top level, it is

because the sort of things that you do at the top level often needs

exception handling. For example, you might have a fallback module:



try:

import this_module

except ImportError:

import that_module as this_module





You will very rarely see that inside a class, since you very rarely

import modules inside a class.

* Classes should be coded in a way that exceptions

I think you forgot to finish the sentence.

* Better to

never write your own exceptions (unless you absolutely have to).

That depends.



On the one hand, nobody wants a million different exception types. On the

other hand, nobody wants just *one* exception type, and no way to

distinguish between different kinds of errors. Somewhere between one and

one million is an appropriate number of exception types.



The right answer is to be conservative about creating new exceptions, but

don't be scared to create one when you need one.



But when you do, it is often better to subclass from an appropriate built-

in exception like ValueError or TypeError or similar, than to subclass

from Exception itself.

* Using

Exception is typically a bad. More specific the better.

Yes, you should always try to catch the specific exceptions you care

about:





# Best

except ValueError, OverflowError, ZeroDivisionError:





# Not so good

except Exception:





# Worst

except:





Don't use the last one, except maybe in the interactive interpreter,

since it will catch *everything*, even exceptions that probably shouldn't

be caught like KeyboardInterrupt.

* Exceptions

should never fail silently. (Should exceptions always be logged?)

Certainly not. Exceptions should fail silently if you don't care about

them. For example, when connecting to a website, there are many temporary

errors that can occur. The best way to handle them is to catch the

exception, sleep for a little while, then try again. You need only care

if repeated attempts to connect, with larger and larger sleeps, continue

to fail.



Of course, you might have a debug mode that logs all of these, but if

your web browser logged every single time a webpage was slow to respond,

you would soon run out of disk space



*Errors* should never fail silently, unless explicitly silenced. But an

error isn't an error if you don't care about it, and an exception is not

necessarily an error.



This is an error, because converting a number to uppercase cannot

possibly mean anything:



mystring = 42

mystring.upper()





This is not necessarily an error, since "the list is empty" could be a

legitimate situation:



mylist = []

first = mylist[0]



In this case, it may be appropriate to catch the exception, and either

silently swallow it, or do something else.

Best site I have found for exceptions (hopefully this helps someone): *

http://c2.com/cgi/wiki?ExceptionPatterns

I haven't read that page for a long time, but as I recall the c2.com

website, a lot of the ideas there are better suited to Java and C/C++

(and occasionally Lisp) rather than Python. But still, a valuable (if

often confusing) resource.

I'd be interested in hearing others thoughts on this topic with regards

to best practices for when to use exceptions, and when to avoid using

exceptions.

The try part of a try...except is *very* fast to set up. It's about as

fast as a "pass" (do nothing), so it has little overhead.



On the other hand, actually *catching* an exception is quite heavy. So

code that catches lots and lots of exceptions may be slow. In that case,

it may be faster to "look before you leap" and test ahead of time:



# this is faster if most lists are not empty

try:

process(mylist[0])

except IndexError:

handle_empty_list()



# this is faster if many lists are empty

if mylist:

process(mylist[0])

else:

handle_empty_list()





Only profiling your data can tell you which you should use.





On the other hand, here you should *always* use try...except:



try:

myfile = open("name")

except IOError:

handle_error()





Because this code is wrong:



if os.path.exists("name"):

myfile = open("name")

else:

handle_error()





It's wrong for a couple of reasons:



- just because the file exists, doesn't mean you can open it;



- even if the file exists when you call the os.path.exists function,

doesn't mean it will still exist a millisecond later when you try to open

it.







Hope this is helpful,