Best Way to Handle All Exceptions

[seldan24]

Hello,

I'm fairly new at Python so hopefully this question won't be too
awful. I am writing some code that will FTP to a host, and want to
catch any exception that may occur, take that and print it out
(eventually put it into a log file and perform some alerting action).
I've figured out two different ways to do this, and am wondering which
is the best (i.e. cleanest, 'right' way to proceed). I'm also trying
to understand exactly what occurs for each one.

The first example:

from ftplib import FTP
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except Exception, err:
print err

This works fine. I read through the documentation, and my
understanding is that there is a built-in exceptions module in python,
that is automatically available in a built-in namespace. Within that
module is an 'Exception' class which would contain whatever exception
is thrown. So, I'm passing that to the except, along with err to hold
the value and then print it out.

The second example:

from ftplib import FTP
import sys
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except:
print sys.exc_info()

Here I, for the most part, get the same thing. I'm not passing
anything to except and just printing out the exception using a method
defined in the sys module.

So, I'm new to Python... I've made it this far and am happy, but want
to make sure I'm coding correctly from the start. Which method is the
better/cleaner/more standard way to continue? Thanks for any help.

 

[Diez B. Roggisch]

Hello,

I'm fairly new at Python so hopefully this question won't be too
awful. I am writing some code that will FTP to a host, and want to
catch any exception that may occur, take that and print it out
(eventually put it into a log file and perform some alerting action).
I've figured out two different ways to do this, and am wondering which
is the best (i.e. cleanest, 'right' way to proceed). I'm also trying
to understand exactly what occurs for each one.

The first example:

from ftplib import FTP
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except Exception, err:
print err

This works fine. I read through the documentation, and my
understanding is that there is a built-in exceptions module in python,
that is automatically available in a built-in namespace. Within that
module is an 'Exception' class which would contain whatever exception
is thrown. So, I'm passing that to the except, along with err to hold
the value and then print it out.

The second example:

from ftplib import FTP
import sys
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except:
print sys.exc_info()

Here I, for the most part, get the same thing. I'm not passing
anything to except and just printing out the exception using a method
defined in the sys module.

So, I'm new to Python... I've made it this far and am happy, but want
to make sure I'm coding correctly from the start. Which method is the
better/cleaner/more standard way to continue? Thanks for any help.

The latter is - unfortunately - the better. This comes from python allowing
all kinds of objects being thrown as exceptions, not only those extending
from a common ancestor like Exception.

You seem to have a sensible take on this, but anyway I'd like to mention
that using these kinds of catch-all code is rarely justified, as it imposes
the danger of not handling errors at all. So make sure the program spits
out a sensible error-message, and then dies. Unless it's a server-process
of course.

Diez

 

[MRAB]

Hello,

I'm fairly new at Python so hopefully this question won't be too
awful. I am writing some code that will FTP to a host, and want to
catch any exception that may occur, take that and print it out
(eventually put it into a log file and perform some alerting action).
I've figured out two different ways to do this, and am wondering which
is the best (i.e. cleanest, 'right' way to proceed). I'm also trying
to understand exactly what occurs for each one.

The first example:

from ftplib import FTP
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except Exception, err:
print err

This works fine. I read through the documentation, and my
understanding is that there is a built-in exceptions module in python,
that is automatically available in a built-in namespace. Within that
module is an 'Exception' class which would contain whatever exception
is thrown. So, I'm passing that to the except, along with err to hold
the value and then print it out.

There isn't an "exceptions module"; exceptions are part of the language.

The second example:

from ftplib import FTP
import sys
try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except:
print sys.exc_info()

This is called a "bare exception handler". It's virtually never the
right way to do it.

Here I, for the most part, get the same thing. I'm not passing
anything to except and just printing out the exception using a method
defined in the sys module.

So, I'm new to Python... I've made it this far and am happy, but want
to make sure I'm coding correctly from the start. Which method is the
better/cleaner/more standard way to continue? Thanks for any help.

You should use the most specific exception possible because at lot of
things could raise an exception:

Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
foo
NameError: name 'foo' is not defined foo
except Exception, e:
print "*** Caught an exception ***"
print e

*** Caught an exception ***
name 'foo' is not defined

 

[seldan24]

There isn't an "exceptions module"; exceptions are part of the language.



This is called a "bare exception handler". It's virtually never the
right way to do it.



You should use the most specific exception possible because at lot of
things could raise an exception:

 >>> foo

Traceback (most recent call last):
   File "<pyshell#0>", line 1, in <module>
     foo
NameError: name 'foo' is not defined
 >>> try:
     foo
except Exception, e:
     print "*** Caught an exception ***"
     print e

*** Caught an exception ***
name 'foo' is not defined- Hide quoted text -

- Show quoted text -

Thank you both for your input. I want to make sure I get started on
the right track. For this particular script, I should have included
that I would take the exception contents, and pass those to the
logging module. For this particular script, all exceptions are fatal
and I would want them to be. I just wanted a way to catch them and
log them prior to program termination. I can understand why folks
should always specify exact exceptions where possible if they plan on
doing something with them, but I just want to log and terminate (and
alarm) when any exception, irregardless of what it is, occurs; that's
why I'm using the catch-all approach. I most likely should have put
in an exit() in my snippet; sorry about that.

I did notice that Python allows different objects to be thrown. This
threw me off a bit because, at first, I was expecting everything to
come through as nice, easy tuples. It turns out that ftplib throws
some class from the sockets module (my terminology may be off here).
As for terminology, sorry about the 'exceptions' misuse. The Python
documentation refers to 'exceptions' as a module, albeit built-in, so
I used that language accordingly (link to doc:
http://docs.python.org/library/exceptions.html?highlight=exceptions#module-exceptions).

Anyway, thanks again for the help and advice, it is greatly
appreciated.

 

[Floris Bruynooghe]

The first example:

from ftplib import FTP
try:
    ftp = FTP(ftp_host)
    ftp.login(ftp_user, ftp_pass)
except Exception, err:
    print err

*If* you really do want to catch *all* exceptions (as mentioned
already it is usually better to catch specific exceptions) this is the
way to do it.

To know why you should look at the class hierarchy on
http://docs.python.org/library/exceptions.html. The reason is that
you almost never want to be catching SystemExit, KeyboardInterrupt
etc. catching them will give you trouble at some point (unless you
really know what you're doing but then I would suggest you list them
explicitly instead of using the bare except statement).

While it is true that you could raise an object that is not a subclass
from Exception it is very bad practice, you should never do that. And
I've haven't seen an external module in the wild that does that in
years and the stdlib will always play nice.


Regards
Floris

 

[Vilya Harvey]

Thank you both for your input.  I want to make sure I get started on
the right track.  For this particular script, I should have included
that I would take the exception contents, and pass those to the
logging module.  For this particular script, all exceptions are fatal
and I would want them to be.  I just wanted a way to catch them and
log them prior to program termination.

The logging module has a function specifically to handle this case:

try:
# do something
except:
logging.exception("Uh oh...")

The exception() method will automatically add details of the exception
to the log message it creates; as per the docs, you should only call
it from within an exception handler.

Hope that helps,

Vil.

 

[Piet van Oostrum]

s> Hello,
s> I'm fairly new at Python so hopefully this question won't be too
s> awful. I am writing some code that will FTP to a host, and want to
s> catch any exception that may occur, take that and print it out
s> (eventually put it into a log file and perform some alerting action).
s> I've figured out two different ways to do this, and am wondering which
s> is the best (i.e. cleanest, 'right' way to proceed). I'm also trying
s> to understand exactly what occurs for each one.

s> The first example:

s> from ftplib import FTP
s> try:
s> ftp = FTP(ftp_host)
s> ftp.login(ftp_user, ftp_pass)
s> except Exception, err:
s> print err

I think you should restrict yourself to those exceptions that are
related to ftp. Do you want to catch an exception like a misspelling in
one of the variables?

from ftplib import FTP, all_errors

try:
ftp = FTP(ftp_host)
ftp.login(ftp_user, ftp_pass)
except all_errors, err:
print err

 

[Terry Reedy]

The latter is - unfortunately - the better. This comes from python allowing
all kinds of objects being thrown as exceptions, not only those extending
from a common ancestor like Exception.

Fixed in Python 3. exceptions, without exceptions, are instances of
BaseException, either directly or as instances of subcleasses of
BaseException.

One of many reasons to switch when possible.
tjr

 

[Carl Banks]

I think you should restrict yourself to those exceptions that are
related to ftp. Do you want to catch an exception like a misspelling in
one of the variables?

He quite reasonably could want that, such as if the program is
designed to be run from a cron job, or in some other non-interactive
way.

Just because something is usually a bad idea doesn't mean it always
is.


Carl Banks

 

[Steven D'Aprano]

He quite reasonably could want that, such as if the program is designed
to be run from a cron job, or in some other non-interactive way.

Why is it okay for non-interactive programs to silently have incorrect
behaviour?

What's the point of a cron job that doesn't do what it is supposed to,
because it has a bug which is silently swallowed?



"I find it amusing when novice programmers believe their main job is
preventing programs from crashing. ... More experienced programmers
realize that correct code is great, code that crashes could use
improvement, but incorrect code that doesn't crash is a horrible
nightmare."

http://www.pphsg.org/cdsmith/types.html

 

[Steven D'Aprano]

Instead of catching *all* exceptions at a specific point in your code,
and then having nothing to do but end the program (possibly after some
pre-exit action like logging), I think you would be better served by
installing a custom top-level exception handler. ....
Any other exceptions that you don't specifically catch will then go
through to your default handle-it-just-before-we-exit ‘sys.excepthook’
exception handler.

Isn't that risky though? Won't that potentially change the exception-
handling behaviour of functions and classes he imports from other modules?

 

[Carl Banks]

Why is it okay for non-interactive programs to silently have incorrect
behaviour?

What's the point of a cron job that doesn't do what it is supposed to,
because it has a bug which is silently swallowed?

Seriously, do you *ever* take more than 2 seconds to consider whether
you might be missing something obvious before following up with these
indignant knee-jerk responses?

I never said a thing about swallowing errors. I was talking about
catching exceptions, whence you might do all kinds of things besides
swallowing (like logging). I was pointing out there are reasons why
you might want to catch variable misspelling errors when you're
running non-interactively (like for instance, to log the error
somewhere).

Or would you rather let all unexpected exceptions print to standard
error, which is often a black hole in non-interactive sitations?


Carl Banks

 

[Steven D'Aprano]

Seriously, do you *ever* take more than 2 seconds to consider whether
you might be missing something obvious before following up with these
indignant knee-jerk responses?

Obviously not.


[...]

Or would you rather let all unexpected exceptions print to standard
error, which is often a black hole in non-interactive sitations?

Fair point. Of course you're right.

 

[Lawrence D'Oliveiro]

In message <93f6a517-63d8-4c80-

Or would you rather let all unexpected exceptions print to standard
error, which is often a black hole in non-interactive sitations?

Since when?

Cron, for example, collects standard error and mails it to you.

 

[Lawrence D'Oliveiro]

In message <d82cea0b-3327-4a27-

For this particular script, all exceptions are fatal
and I would want them to be. I just wanted a way to catch them and
log them prior to program termination.

You don't need to. They will be written to standard error anyway.

 

[Carl Banks]

Seriously, do you *ever* take more than 2 seconds to consider whether
you might be missing something obvious before following up with these
indignant knee-jerk responses?

Obviously not.

[...]

Or would you rather let all unexpected exceptions print to standard
error, which is often a black hole in non-interactive sitations?

Fair point. Of course you're right.

I don't want to be mean or anything. I agree with you on 95% of
issues probably, of course I only follow up to disagree....

Carl Banks

 

[Carl Banks]

In message <93f6a517-63d8-4c80-



Since when?

Cron, for example, collects standard error and mails it to you.

1. Cron is only one way to run programs non-interactively. (Would that
work with, say, Windows services?)
2. Many systems don't run MTAs these days
3. In my experience the pipeline from cron to mail delivery is error
prone, mainly due to the complexity of configuring MTAs
4. Even if all this works, you might just want to do your logging
directly in Python anyway


Carl Banks

 

[Piet van Oostrum]

CB> On Jul 14, 4:48 am, Lawrence D'Oliveiro <l...@geek-

CB> 1. Cron is only one way to run programs non-interactively. (Would that
CB> work with, say, Windows services?)
CB> 2. Many systems don't run MTAs these days
CB> 3. In my experience the pipeline from cron to mail delivery is error
CB> prone, mainly due to the complexity of configuring MTAs
CB> 4. Even if all this works, you might just want to do your logging
CB> directly in Python anyway

Even then, I think the program would get structured better if the FTP
code would only catch FTP-specific errors, including socket errors
(which is what all_errors does) and to catch programming errors etc on
another level, e.g by the mentioned sys.excepthook.

 

[Steven D'Aprano]

Seriously, do you *ever* take more than 2 seconds to consider whether
you might be missing something obvious before following up with these
indignant knee-jerk responses?

Obviously not.

[...]

Or would you rather let all unexpected exceptions print to standard
error, which is often a black hole in non-interactive sitations?

Fair point. Of course you're right.

I don't want to be mean or anything. I agree with you on 95% of issues
probably, of course I only follow up to disagree....

No problem Carl. You were right, in this instance I was being a smartarse
and ended up being too clever for my own good. There are cases where you
want to catch all exceptions, not to hide them, but to process them in
some way (possibly logging the error and then exiting).

 

[Jean]

Hello,

I'm fairly new at Python so hopefully this question won't be too
awful.  I am writing some code that will FTP to a host, and want to
catch any exception that may occur, take that and print it out
(eventually put it into a log file and perform some alerting action).
I've figured out two different ways to do this, and am wondering which
is the best (i.e. cleanest, 'right' way to proceed).  I'm also trying
to understand exactly what occurs for each one.

The first example:

from ftplib import FTP
try:
    ftp = FTP(ftp_host)
    ftp.login(ftp_user, ftp_pass)
except Exception, err:
    print err

This works fine.  I read through the documentation, and my
understanding is that there is a built-in exceptions module in python,
that is automatically available in a built-in namespace.  Within that
module is an 'Exception' class which would contain whatever exception
is thrown.  So, I'm passing that to the except, along with err to hold
the value and then print it out.

The second example:

from ftplib import FTP
import sys
try:
    ftp = FTP(ftp_host)
    ftp.login(ftp_user, ftp_pass)
except:
    print sys.exc_info()

Here I, for the most part, get the same thing.  I'm not passing
anything to except and just printing out the exception using a method
defined in the sys module.

So, I'm new to Python... I've made it this far and am happy, but want
to make sure I'm coding correctly from the start.  Which method is the
better/cleaner/more standard way to continue?  Thanks for any help.

The second example is "better" if you need your code to work in Python
3.0 *and* in 2.X.

For Python 3.0, the first example has to be written as:

....
except Exception as err:
....


/Jean Brouwers