Problem overriding sys.excepthook

[Lunchtimemama]

Yo all, I'm getting into Python for the first time and I'm really
having a blast. I've hit a bit of a snag and was wondering if someone
could lend some insight. Here be the code:

import sys
def myexcepthook(type, value, tb):
import traceback
rawreport = traceback.format_exception(type, value, tb)
report = '\n'.join(rawreport)
errorlog = open('error.log','a')
errorlog.write(('%s\n' + '-'*30 + '\n\n') % report)
errorlog.close()
sys.excepthook = myexcepthook

Now here's the trouble: if I enter that line-by-line into the
interpreter in interactive mode, the custom exception hook will handle
all exceptions, but if I put that in a script that I run from the
shell, it only catches some exceptions. For example, it would catch an
undefined name, like if I just put:

spam

into the program above, the override would work. But if I made a
syntactical error, like:

1 = spam

then it would fall to the standard sys.excepthook. Is there some lazy
evaluation that I don't know about? I'm on Windows, if that makes a
difference. Thank for the help.

-LTM

 

[Patrick Maupin]

Yo all, I'm getting into Python for the first time and I'm really
having a blast. I've hit a bit of a snag and was wondering if someone
could lend some insight. Here be the code:

import sys
def myexcepthook(type, value, tb):
import traceback
rawreport = traceback.format_exception(type, value, tb)
report = '\n'.join(rawreport)
errorlog = open('error.log','a')
errorlog.write(('%s\n' + '-'*30 + '\n\n') % report)
errorlog.close()
sys.excepthook = myexcepthook

Now here's the trouble: if I enter that line-by-line into the
interpreter in interactive mode, the custom exception hook will handle
all exceptions, but if I put that in a script that I run from the
shell, it only catches some exceptions. For example, it would catch an
undefined name, like if I just put:

spam

into the program above, the override would work. But if I made a
syntactical error, like:

1 = spam

then it would fall to the standard sys.excepthook. Is there some lazy
evaluation that I don't know about? I'm on Windows, if that makes a
difference. Thank for the help.

Python first compiles, then executes. However, since an "import" is
considered to be an execution, you can retrieve this sort of
compile-time error, but only on modules which are imported _after_ you
hook the exception handler.

HTH,
Pat

 

[Lunchtimemama]

Forgive my ignorance, but I'm not quite sure what you mean. I tried
importing the traceback module at the beginning of the script, but that
didn't make a difference. Could you provide example code to illustrate
your comment? Thanks.

-LTM

 

[Patrick Maupin]

Forgive my ignorance, but I'm not quite sure what you mean. I tried
importing the traceback module at the beginning of the script, but that
didn't make a difference. Could you provide example code to illustrate
your comment? Thanks.

Assume your main module has your exception hook code in it, up to and
including the line "sys.excepthook = myexcepthook".

As you have noticed, "1 = spam" in your main module would cause an
uncaught exception. This is because this exception is raised during
the compilation phase of the module, before you hooked the exception
vector. This is true NO MATTER WHERE in the main module this statement
is -- Python will compile the entire module before executing it.

Now assume that the statement "1 = spam" is in module "foo.py" instead
of in your main module. If you "import foo" at the top of your main
module, you will have the same result but for slightly different
reasons-- foo.py will be compiled during execution of the main module's
"import" statement (although AFTER compilation of the main module), so
the exception will still be raised before you have assigned your
exception hook.

HOWEVER, if you "import foo" AFTER the line "sys.excepthook =
myexcepthook", then the compilation of foo.py will occur after your
exception hook has been set, and your exception handler will execute on
the syntax error "1 = spam" inside foo.py.

So the bottom line is that you can't log syntax errors for your main
module, but you _can_ log syntax errors for other modules which your
main module imports.

I think one of the reasons you are confused is that the interactive
mode, by necessity, has to compile incrementally. But compilation of
imported modules, and compilation of a main module in non-interactive
mode, occurs before any code in that particular module is executed.

Please let me know if it's still unclear.

Regards,
Pat

 

[Lunchtimemama]

Thank you for a fine explanation Pat, that clears things up very
nicely. I have one remaining question which I imagine amounts to taste.
What is the superior method of exception handling:

A) To, as you suggest above, import the code as a module from within a
program with special exception handling code.

or

B) To pipe sys.stderr to a file and keep the OS on the lookout for a
non-zero errorlevel (via a batch file or some such) and then launch
another Python script to further handle the error output.

My goal is to upload error reports to a server. My criteria for
preference are first performance and second conformance with Python
style. Any overwhelming opinion or alternative solution? Thanks again.

 

[Patrick Maupin]

What is the superior method of exception handling:

....

For a start, note that the exception hook does not _really_ have to be
in the main module, just imported before any "protected" code is to be
executed.

Having said that, what I personally typically do for exception trapping
doesn't even require an exception hook. I will execute all my code
from inside a try/except block in the main module. If the exception is
invoked, you can query and get all the data about the exception like
you are currently doing, and then re-raise the exception (or not).

I haven't given too much thought to when I would set an exception hook.
It seems like more of a debugger kind of thing. Especially if you
have control over the entire app, a try/except at the top level feels
cleaner to me.

Regards,
Pat

 

[Lunchtimemama]

Try/except sounds like the way to go. Thanks.