How to pop the interpreter's stack?

[John Nagle]

On Dec 24, 1:24 am, Steven D'Aprano<steve
(e-mail address removed)> wrote: All I'm

How about

raise ValueError("Bad input %s to validate_arg" % (repr(arg),))

You can pass arguments to most exceptions, and the content of the
exception is determined entirely by the code raising it.

If end users are seeing uncaught tracebacks, the program is broken.
It's usually worth it to catch EnvironmentError near the outermost
level of the program, since most non program bug events, like I/O
and network errors. will raise some subclass of EnvironmentError.

John Nagle

 

[Ethan Furman]

As a user I can criticize the decision of the implementor to
needlessly filter information, and declare that it's borne out of the
author's arrogance in thinking he knows what I want when I get a
traceback.

I can also opine that Python language shouldn't make it easy for
library implementors to be arrogant like this.


Arrogance. Who gave you the right to decide what is completely
irrelevant to user? I, as the user, decide what's relevant. If I
want implementation-dependent information, it's my business.

I don't want the language to make it easy for arrogant people, who
think they know what information I want better than I do, to hide that
information from me.

One of the many things I love about Python is that it stays out of the
way of me getting my work done. I think a truly pythonic
program/library/module must do the same. So in this regard I agree with
Carl.

There are also times when I change the exception being raised to match
what python expects from that type of object -- for example, from
WhatEverException to KeyError for a dict-like object. So in this regard
I agree with Steven.

For kj's concern, which seems to be along the lines of functional as
opposed to custom object, I don't think the traceback should be monkied
with -- either use a decorator to keep the traceback short, or give the
_pre_func name a good name and don't worry about it. I know when I see
a traceback, I start at the bottom and only work my way up if I need to.

~Ethan~

 

[Robert Kern]

There are also times when I change the exception being raised to match
what python expects from that type of object -- for example, from
WhatEverException to KeyError for a dict-like object. So in this regard I agree
with Steven.

Steven isn't arguing that particular point here, nor is anyone arguing against it.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

But that of course is nonsense, because as the user you don't decide
anything of the sort. The developer responsible for writing the function
decides what information he provides you, starting with whether you get
an exception at all, where it comes from, the type of exception, and the
error message (if any).

Carl isn't arguing that the user is or should be responsible for this sort of
thing. He is arguing that developers should be responsible for doing this in
such a way that is beneficial for the developer/user down the road.

Once this information has been passed on to you,
you're free to do anything you like with it, but you never get to choose
what information you get -- I'm not suggesting any change there. All I'm
suggesting is that there should be a way of reducing the boilerplate
needed for this idiom:

def _validate_arg(x):
if x == 'bad input': return False
return True

def f(arg):
if not _validate_arg(arg):
raise ValueError
process(arg)

to something more natural that doesn't needlessly expose implementation
details that are completely irrelevant to the caller.

Except that the *caller* never gets the traceback (unless if it deliberately
inspects the stack for some metaprogramming reason). It gets the exception, and
that is the same no matter what you do. The developer/user gets the traceback,
and those implementation details *are* often important to them.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Steven D'Aprano]

Steven isn't arguing that particular point here, nor is anyone arguing
against it.

Emphasis on *here*.

You will note that in Python 3, if you raise an exception inside an
except block, both the *original* and the new exception are printed. This
is great for fixing bugs inside except blocks, but terribly disruptive
for catching one error and raising another error in it's place, e.g.:

try:
x+0
except ValueError, TypeError as e:
# x is not a numeric value, e.g. a string or a NAN.
raise MyError('x is not a number')


The explicit raise is assumed to indicate a bug in the except block, and
the original exception is printed as well.

But that's a separate issue from what is being discussed here. What we're
discussing here is the idea that a function should be able to delegate
work to private subroutines without the caller being aware of that fact.
When you return a value, the caller doesn't see the internal details of
how you calculated the value, but if you deliberately raise an exception,
the caller does. Often this is the right thing to do, but sometimes it
isn't. E.g. you can't delegate input validation to a subroutine and raise
inside the subroutine without obfuscating the traceback.

 

[Steven D'Aprano]

How about

raise ValueError("Bad input %s to validate_arg" % (repr(arg),))

You can pass arguments to most exceptions, and the content of the
exception is determined entirely by the code raising it.

I know that exceptions can take arguments (usually string error
messages). I was writing in short-hand. My apologies, I thought that
would have been obvious

Perhaps you have missed the context of the discussion. The context is
that the called function delegates the job of validating input to a
private function, which should be hidden from the caller (it's private,
not part of the public API, subject to change, hidden, etc.) but
tracebacks expose that information, obscuring the cause of the fault.
(The fault being bad input to the public function, not an accidental bug
in the private function.)

If end users are seeing uncaught tracebacks, the program is broken.

Well, perhaps, but that's a separate issue. We're talking about the
caller of the function seeing internal details, not the end use.

 

[Carl Banks]

But that's a separate issue from what is being discussed here. What we're
discussing here is the idea that a function should be able to delegate
work to private subroutines without the caller being aware of that fact.

I can't fathom any possible reason why this could be considered a good
thing, especially in Python where the culture is, "We're all adults
here". By the same logic, it would be a good idea to prevent the user
from accessing private members of a class, internal variables of a
module, or importing an internal module directly.

There is a convention in Python: internal objects are preceded by
underscore. The fact that your internal function is marked this way
in the traceback is more than enough information to the user that this
is an internal function.

Python is not, and never has been, about hiding internal details.
It's about openness, and there's no reason a traceback should hide
internal details any more than a class should--in fact hiding
information in the traceback is far worse, because you're suppressing
information that could be crucial for debugging.


Carl Banks

 

[Carl Banks]

Carl isn't arguing that the user is or should be responsible for this sort of
thing. He is arguing that developers should be responsible for doing this in
such a way that is beneficial for the developer/user down the road.

I'm not even arguing that; I think I would be content if the developer
merely doesn't actively work to harm the user.


Carl Banks

 

[Ethan Furman]

Python is not, and never has been, about hiding internal details.
It's about openness, and there's no reason a traceback should hide
internal details any more than a class should--in fact hiding
information in the traceback is far worse, because you're suppressing
information that could be crucial for debugging.

+100

~Ethan~

 

[Ethan Furman]

Emphasis on *here*.

You will note that in Python 3, if you raise an exception inside an
except block, both the *original* and the new exception are printed. This
is great for fixing bugs inside except blocks, but terribly disruptive
for catching one error and raising another error in it's place...

Yes, this is where I was agreeing with Steven. While I love python3,
the current nested exception behavior is horrible.

~Ethan~

 

[Ethan Furman]

Right. But I have thought of a clever trick to get the result KJ was
asking for, with the minimum of boilerplate code. Instead of this:


def _pre_spam(args):
if condition(args):
raise SomeException("message")
if another_condition(args):
raise AnotherException("message")
if third_condition(args):
raise ThirdException("message")

def spam(args):
_pre_spam(args)
do_useful_work()


you can return the exceptions instead of raising them (exceptions are
just objects, like everything else!), and then add one small piece of
boilerplate to the spam() function:


def _pre_spam(args):
if condition(args):
return SomeException("message")
if another_condition(args):
return AnotherException("message")
if third_condition(args):
return ThirdException("message")

def spam(args):
exc = _pre_spam(args)
if exc: raise exc
do_useful_work()

-1

You failed to mention that cleverness is not a prime requisite of the
python programmer -- in fact, it's usually frowned upon. The big
problem with the above code is you are back to passing errors in-band,
pretty much completely defeating the point of have an out-of-band channel.

~Ethan~

 

[kj]

Except that the *caller* never gets the traceback (unless if it deliberately
inspects the stack for some metaprogramming reason). It gets the exception, and
that is the same no matter what you do. The developer/user gets the traceback,
and those implementation details *are* often important to them.

Just look at what Python shows you if you pass the wrong number of
arguments to a function:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: spam() takes exactly 3 arguments (2 given)


That's it. The traceback stops at the point of the error. Python
doesn't show you all the underlying C-coded machinery that went
into detecting the error and emitting the error message. *No one*
needs this information at this point. All I'm saying is that I
want to do the same thing with my argument validation code as Python
does with its argument validation code: keep it out of sight. When
my argument validation code fires an exception ***there's no bug
in **my** code***. It's doing exactly what it's supposed to do.
Therefore, there's no need for me to debug anything, and certainly
no need for me to inspect the traceback all the way to the exception.
The bug is in the code that called my function with the wrong
arguments. The developer of that code has no more use for seeing
the traceback all the way to where my code raises the exception
than I have for seeing the traceback of Python's underlying C code
when I get an error like the one shown above.

~kj

 

[kj]

You failed to mention that cleverness is not a prime requisite of the
python programmer -- in fact, it's usually frowned upon.

That's the party line, anyway. I no longer believe it. I've been
crashing against one bit of cleverness after another in Python's
unification of types and classes...

 

[John Nagle]

I know that exceptions can take arguments (usually string error
messages). I was writing in short-hand. My apologies, I thought that
would have been obvious

Perhaps you have missed the context of the discussion. The context is
that the called function delegates the job of validating input to a
private function, which should be hidden from the caller (it's private,
not part of the public API, subject to change, hidden, etc.) but
tracebacks expose that information, obscuring the cause of the fault.
(The fault being bad input to the public function, not an accidental bug
in the private function.)



Well, perhaps, but that's a separate issue. We're talking about the
caller of the function seeing internal details, not the end use.

No, that is the issue, unless the program itself is examining the
stack traceback data. Python exception-catching has no notion of
what code raised the exception. Only the contents of the exception
object are normally available. So the "private function" is always
"hidden", unless you're debugging, in which case it shouldn't be
hidden.

Traceback is purely a debugging feature. In some Python
implementations, such as Shed Skin, you don't get tracebacks unless
you're running under a debugger.

John Nagle

 

[Steve Holden]

That's the party line, anyway. I no longer believe it. I've been
crashing against one bit of cleverness after another in Python's
unification of types and classes...

Well if you can find a way to implement a class system that doesn't use
clever tricks *in its implementation* please let me know.

regards
Steve

 

[Steve Holden]

That's the party line, anyway. I no longer believe it. I've been
crashing against one bit of cleverness after another in Python's
unification of types and classes...

Well if you can find a way to implement a class system that doesn't use
clever tricks *in its implementation* please let me know.

regards
Steve

 

[misnomer]

You seem to have completely missed that there will be no bug report,
because this isn't a bug. (Or if it is a bug, the bug is elsewhere,
external to the function that raises the exception.) It is part of the
promised API. The fact that the exception is generated deep down some
chain of function calls is irrelevant.

While the idea of being able to do this (create a general validation
exception) sounds mildly appealing at first, what happens when the
module implementing this documented API and documented error, has a
bug? It seems that the user, or even module developer in the midst of
writing, would now have no tools to easily tackle the problem, and no
useful information to submit in the required bug report.

 

[Carl Banks]

Just look at what Python shows you if you pass the wrong number of
arguments to a function:


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: spam() takes exactly 3 arguments (2 given)

That's it.  The traceback stops at the point of the error.  Python
doesn't show you all the underlying C-coded machinery that went
into detecting the error and emitting the error message.  *No one*
needs this information at this point.  All I'm saying is that I
want to do the same thing with my argument validation code as Python
does with its argument validation code: keep it out of sight.  When
my argument validation code fires an exception ***there's no bug
in **my** code***.  It's doing exactly what it's supposed to do.
Therefore, there's no need for me to debug anything, and certainly
no need for me to inspect the traceback all the way to the exception.
The bug is in the code that called my function with the wrong
arguments.  The developer of that code has no more use for seeing
the traceback all the way to where my code raises the exception
than I have for seeing the traceback of Python's underlying C code
when I get an error like the one shown above.

Python makes no attempt to hide its machinery in tracebacks (that I'm
aware of); in fact stack points from internal Python functions,
classes, and modules appear in tracebacks all the time. The reason
you don't see traceback lines for Python's argument validation is it's
written in C. If it bothers you that much, you're welcome to write
you own argument validation in C, too.


Carl Banks

 

[Carl Banks]

Emphasis on *here*.

You will note that in Python 3, if you raise an exception inside an
except block, both the *original* and the new exception are printed. This
is great for fixing bugs inside except blocks, but terribly disruptive
for catching one error and raising another error in it's place, e.g.:

try:
    x+0
except ValueError, TypeError as e:
    # x is not a numeric value, e.g. a string or a NAN.
    raise MyError('x is not a number')

The explicit raise is assumed to indicate a bug in the except block, and
the original exception is printed as well.

But that's a separate issue from what is being discussed here. What we're
discussing here is the idea that a function should be able to delegate
work to private subroutines without the caller being aware of that fact.
When you return a value, the caller doesn't see the internal details of
how you calculated the value, but if you deliberately raise an exception,
the caller does. Often this is the right thing to do, but sometimes it
isn't. E.g. you can't delegate input validation to a subroutine and raise
inside the subroutine without obfuscating the traceback.

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/re.py", line 190, in compile
return _compile(pattern, flags)
File "/usr/lib/python2.6/re.py", line 245, in _compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis


OHMYGOD HOW DARE the standard library allow the traceback list an
internal function that does input valididation!


Carl Banks

 

[Steven D'Aprano]

-1

You failed to mention that cleverness is not a prime requisite of the
python programmer -- in fact, it's usually frowned upon. The big
problem with the above code is you are back to passing errors in-band,
pretty much completely defeating the point of have an out-of-band
channel.

How is that any worse than making _pre_spam() a validation function that
returns a bool?

def spam(args):
flag = _pre_spam(args)
if flag: raise SomeException()
do_useful_work()


Is that also frowned upon for being too clever?