Pop return from stack?

[bvdp]

Assuming I have a module 'foo.py' with something like this:

def error(s):
print "Error", s
sys.exit(1)

def func(s):
... do some processing
... call error() if bad .. go to system exit.
... more processing

and then I write a new program, test.py, which:

import foo

def myerror(s):
print "new error message"

foo.error = myerror

a = foo.func(..)

Now, if an error is encountered myerror() is called. Fine. But
execution resumes in func(). Not exactly what I wanted.

I can "fix" this simply by wrapping the call to foo.func() in a try/
expect and have myerror() raise an exception. This appears to work,
but I'm hesitant to use this out of fear that I'm building up some
kind of stack overflow or something which will bite me later.

Is there a better way? Simplest for an old assembler guy like me would
be pop a return address off the stack ... but python isn't
assembler

I don't want to change stuff in the foo.py module since it's part of
an existing program. But, if I must, I suppose I could. I'd prefer to
just short-circuit this if possible.

Thanks.

 

[Thomas Jollans]

Assuming I have a module 'foo.py' with something like this:

def error(s):
print "Error", s
sys.exit(1)

def func(s):
... do some processing
... call error() if bad .. go to system exit.
... more processing

and then I write a new program, test.py, which:

import foo

def myerror(s):
print "new error message"

foo.error = myerror

a = foo.func(..)

Now, if an error is encountered myerror() is called. Fine. But
execution resumes in func(). Not exactly what I wanted.

I can "fix" this simply by wrapping the call to foo.func() in a try/
expect and have myerror() raise an exception. This appears to work,
but I'm hesitant to use this out of fear that I'm building up some
kind of stack overflow or something which will bite me later.

An exception will walk up the stack, calling any cleaning-up code that needs
to be done (removing object references, executing finally: blocks, exiting
context managers properly. It won't break anything. Don't be afraid of
Python's high-level features!

Is there a better way? Simplest for an old assembler guy like me would
be pop a return address off the stack ... but python isn't
assembler

Now that has a decent chance of messing things up and you (if you wrote decent
assembly ;-)) know it -- without properly cleaning up before resuming
execution in the right place, you could end up in a right state with memory
leaks, leaked file descriptors, half-arsed database transactions, etc etc.

I don't want to change stuff in the foo.py module since it's part of
an existing program. But, if I must, I suppose I could. I'd prefer to
just short-circuit this if possible.

Exceptions. Simple. In the end, all system.exit does is raise a SystemExit
exception...


- Thomas

 

[Steven D'Aprano]

Assuming I have a module 'foo.py' with something like this:

def error(s):
print "Error", s
sys.exit(1)

def func(s):
... do some processing
... call error() if bad .. go to system exit. ... more processing

and then I write a new program, test.py, which:

import foo

def myerror(s):
print "new error message"

foo.error = myerror

a = foo.func(..)

This general technique is called "monkey patching".

Now, if an error is encountered myerror() is called. Fine. But execution
resumes in func(). Not exactly what I wanted.

Of course it does. Your new error handler fails to exit, so execution
resumes like it does after any other function.

You can either manually exit from your own error handler:

def myerror(s):
print "new error message"
sys.exit(2)


or call the original error handler:


def myerror(s):
print "new error message"
foo._error(s)


That second technique requires some preparation before hand. In module
foo, after defining the error() function, you then need to create a
second, private, name to it:

_error = error

I can "fix" this simply by wrapping the call to foo.func() in a try/
expect and have myerror() raise an exception. This appears to work, but
I'm hesitant to use this out of fear that I'm building up some kind of
stack overflow or something which will bite me later.

Exceptions are the standard way of doing things. That's what sys.exit()
does -- it raises SystemExit exception.

With very few exceptions, if you're writing your own error handlers like
this, you're doing it wrong. Your error handler throws away useful
debugging information, and it gives you no extra information that a
standard Python traceback couldn't give.

Is there a better way? Simplest for an old assembler guy like me would
be pop a return address off the stack ... but python isn't assembler

Oh my ... I've seen people writing Java in Python, C++ in Python, Perl in
Python, even VB in Python, but this is the first time I've meet some one
who wants to write assembler in Python

 

[Carl Banks]

Assuming I have a module 'foo.py' with something like this:

def error(s):
    print "Error", s
    sys.exit(1)

def func(s):
    ... do some processing
    ... call error() if bad .. go to system exit.
    ...  more processing

and then I write a new program, test.py, which:

import foo

def myerror(s):
    print "new error message"

foo.error = myerror

a = foo.func(..)

Now, if an error is encountered myerror() is called. Fine. But
execution resumes in func(). Not exactly what I wanted.

I can "fix" this simply by wrapping the call to foo.func() in a try/
expect and have myerror() raise an exception. This appears to work,
but I'm hesitant to use this out of fear that I'm building up some
kind of stack overflow or something which will bite me later.

What do you think a few words of data the stack are going to do?

Just do it this way.


Carl Banks

 

[Chris Rebert]

On Sat, Aug 14, 2010 at 5:23 PM, Steven D'Aprano

Oh my ... I've seen people writing Java in Python, C++ in Python, Perl in
Python, even VB in Python, but this is the first time I've meet some one
who wants to write assembler in Python

+1 QOTW

Cheers,
Chris

 

[bvdp]

An exception will walk up the stack, calling any cleaning-up code that needs
to be done (removing object references, executing finally: blocks, exiting
context managers properly. It won't break anything. Don't be afraid of
Python's high-level features!

Okay, I believe you (and the rest of the gang. In my trivial program
the exception in working ... so I'll leave it alone.

Thanks.

 

[bvdp]

This general technique is called "monkey patching".

New term for me

Of course it does. Your new error handler fails to exit, so execution
resumes like it does after any other function.

I guess I wasn't being clear. I don't want to exit in my new bit of
code. Just continue a loop (which I didn't show in the example).

Exceptions are the standard way of doing things. That's what sys.exit()
does -- it raises SystemExit exception.

Okay, didn't know that exit() was really an exception. Good to know.
But, like I said, I'm not looking to exit.

With very few exceptions, if you're writing your own error handlers like
this, you're doing it wrong. Your error handler throws away useful
debugging information, and it gives you no extra information that a
standard Python traceback couldn't give.

Yeah, but I really don't want a traceback printed out for a user just
because a file can't be found, or he's got a bad bit of syntax in his
file. So, that's why I have the specific error routine. Works fine in
the main program.

Oh my ... I've seen people writing Java in Python, C++ in Python, Perl in
Python, even VB in Python, but this is the first time I've meet some one
who wants to write assembler in Python

Naw, I had my fun with assembler in the good old days. Never want to
write another line of it

 

[Dave Angel]

This general technique is called "monkey patching".


<snip>
You can either manually exit from your own error handler:

def myerror(s):
print "new error message"
sys.exit(2)


or call the original error handler:


def myerror(s):
print "new error message"
foo._error(s)


That second technique requires some preparation before hand. In module
foo, after defining the error() function, you then need to create a
second, private, name to it:

_error = error

Small point. The OP's request was that he not modify the called module,
which is why he was considering monkey-patching. And you can readily
avoid adding that line to the file. Just do something like this:

import foo
_olderror_func = foo.error

def myerror(s)
print "new error message"
_olderror_func(s)

DaveA

 

[John Nagle]

Assuming I have a module 'foo.py' with something like this:

def error(s):
print "Error", s
sys.exit(1)

def func(s):
... do some processing
... call error() if bad .. go to system exit.
... more processing

Fix "func". That's terrible Python. No standard Python library
module calls system exit to handle an error. So that must be in
your code. Standard procedure for errors is to raise an
exception.

John Nagle

 

[bvdp]

    Fix "func".  That's terrible Python.   No standard Python library
module calls system exit to handle an error.  So that must be in
your code.   Standard procedure for errors is to raise an
exception.

Not to belabor the point .. but "func" is not a standard lib module.
It's part of a much larger application ... and in that application it
makes perfect sense to terminate the application if it encounters an
error. I fail to see the problem with this. Why would an APPLICATION
raise a error or not exit to the system?

Does it help to note that error() as defined in the application prints
out a helpful message, etc?

The whole problem I was having is that I was trying to tie a small
application (an helper to the main application) to use a bit of the
existing code as a pseudo-library. Certainly, if the code I was
interfacing with was a standar Python module ... well, then this
thread would not exist in the first place.

However, I have gotten hit with more than one comment like yours. So,
could you please clarify? Is it bad form to exit an application with
sys.exit(1) when an error in a file the application is processing is
found?

Honestly, I'm not trying to be argumentative ... just trying to
understand.

Thanks.

 

[Carey Tilden]

Not to belabor the point .. but "func" is not a standard lib module.
It's part of a much larger application ... and in that application it
makes perfect sense to terminate the application if it encounters an
error. I fail to see the problem with this. Why would an APPLICATION
raise a error or not exit to the system?

For me, the main reason is to ensure the application has only one exit
point. There's only one spot to maintain the code for closing files,
cleaning up network connections, displaying errors to the user, etc.
It makes it really easy to change your mind later about what to do
with errors. It also lets you unify handling of your own errors with
errors generated by 3rd party code.

Carey

 

[Carl Banks]

Not to belabor the point .. but "func" is not a standard lib module.
It's part of a much larger application ... and in that application it
makes perfect sense to terminate the application if it encounters an
error. I fail to see the problem with this. Why would an APPLICATION
raise a error or not exit to the system?

Does it help to note that error() as defined in the application prints
out a helpful message, etc?

The whole problem I was having is that I was trying to tie a small
application (an helper to the main application) to use a bit of the
existing code as a pseudo-library. Certainly, if the code I was
interfacing with was a standar Python module ... well, then this
thread would not exist in the first place.

However, I have gotten hit with more than one comment like yours. So,
could you please clarify? Is it bad form to exit an application with
sys.exit(1) when an error in a file the application is processing is
found?

Honestly, I'm not trying to be argumentative ... just trying to
understand.

The One Obvious Way to handle errors in Python is to raise an
exception, and catching it wherever you can proceed. If you can't
proceed, either don't catch it, or catch it at the top level. Example
(Python 2.6):


class MyException(Exception):
pass


def some_function deep in call tree():
# do some stuff
if is_error():
# will be caught way up the stack, in main
raise MyException(error_message)
# do some stuff if there wasn't an error


def main():
try:
run_program()
except MyException as exc:
print >> sys.stderr, str(exc)
sys.exit(1)


If you call sys.exit() deep within your call tree, the world won't
come to an end, but raising an exception is the preferred way to do
it.

FWIW, I think it perfectly reasonable to let an application print a
traceback on an error. I've gotten a few bug reports on a little tool
I maintain where the user copies the traceback to me, it it's helped
me diagnose their issues a lot.


Carl Banks

 

[Steven D'Aprano]

For me, the main reason is to ensure the application has only one exit
point. There's only one spot to maintain the code for closing files,
cleaning up network connections, displaying errors to the user, etc.

You mean all the things that Python's garbage collector already does for
you?


*wink*

 

[Steven D'Aprano]

On Sun, 15 Aug 2010 18:43:49 -0700, bvdp wrote:

[...]

However, I have gotten hit with more than one comment like yours. So,
could you please clarify? Is it bad form to exit an application with
sys.exit(1) when an error in a file the application is processing is
found?

My two cents worth...

The distinction I like to make is between *planned* and *unplanned*
exits. Planned exits are things like the user choosing Quit from the
menu, or a command line tool exiting after printing a usage message.
"Exit the application" is part of the application's user interface, and
therefore it is appropriate to exit the application. These aren't errors,
the exception isn't called "SystemError" but SystemExit, and it's not
necessary to print a traceback.

Such exits should usually only exist in the front-end (user-interface)
code, rarely or never in the back-end.

But *unplanned* exits are errors, and so you should not disguise them as
planned exits by raising SystemExit, any more than you would disguise
them as a keyboard interruption by raising KeyboardInterrupt. Raise an
appropriate exception, and let Python's normal exception-handling code
clean up and exit for you.

I make one more design choice: if there's an unrecoverable error in user
input (as opposed unexpectedly bad data or a program bug), I suppress the
traceback, but any other unrecoverable error, I allow it to print.
Something vaguely like this:


if __name__ == '__main__':
try:
main()
except getopt.GetoptError, e:
# Unrecoverable error in command line options.
print e.args[0]
sys.exit(42) # or whatever value is appropriate
except Exception, e:
log_exception(e) # whatever...
raise


SystemExit won't be caught, and will just exit normally. Any other
exception won't be caught either, but will lead to a traceback.

 

[Carey Tilden]

You mean all the things that Python's garbage collector already does for
you?

Are you actually disagreeing with the point or just poking fun at my
examples? If the former, well, phooey. If the latter, c'est la vie.

*wink*

Carey

 

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

Oh my ... I've seen people writing Java in Python, C++ in Python, Perl in
Python, even VB in Python, but this is the first time I've meet some one
who wants to write assembler in Python

+1 QOTW

 

[Gregory Ewing]

The whole problem I was having is that I was trying to tie a small
application (an helper to the main application) to use a bit of the
existing code as a pseudo-library.

This is precisely the reason that it's a bad idea to
directly terminate the program from somewhere deep inside
the code. It makes it hard to re-use the code in another
context.

It's much better to raise an exception containing an
appropriate error message, catch it at the top level
of the application and print the message and exit there.
Then you can easily re-use any of the code in a context
where it's not appropriate to have it exit out from
under you.

 

[bvdp]

This is precisely the reason that it's a bad idea to
directly terminate the program from somewhere deep inside
the code. It makes it hard to re-use the code in another
context.

It's much better to raise an exception containing an
appropriate error message, catch it at the top level
of the application and print the message and exit there.
Then you can easily re-use any of the code in a context
where it's not appropriate to have it exit out from
under you.

Thanks Greg. That makes a lot of sense ... for the next program I
write

 

[Aahz]

FWIW, I think it perfectly reasonable to let an application print a
traceback on an error. I've gotten a few bug reports on a little tool
I maintain where the user copies the traceback to me, it it's helped
me diagnose their issues a lot.

That only really works for non-GUI applications. For GUI apps, log files
are the way to go.