PyWart: NameError trackbacks are superfluous

[Rick Johnson]

Sometimes many levels of trace messages can be helpful when detecting bugs,however, in the case of NameErrors, these "nuggets" ejected from deep within the bowls of the Python interpreter are nothing more than steaming piles of incomprehensible crap!

We don't need multiple layers of traces for NameErrors. Python does not have *real* global variables; and thank Guido for that! All we need to know iswhich module the error occurred in AND which line of that module contains the offensive lookup of a name that does not exist.

============================================================
Here is a fine example
============================================================

------------------------------
Contents of mod1.py
------------------------------
print symbolNonExistant

------------------------------
Contents of mod2.py
------------------------------
import mod1

------------------------------
Contents of mod3.py
------------------------------
import mod2

============================================================
Results of executing mod3.py
============================================================
Traceback (most recent call last):
File "C:/a/b/c/mod3.py", line 2, in <module>
import mod2
File "C:/a/b/c/mod2.py", line 1, in <module>
import mod1
File "C:/a/b/c/mod1.py", line 2, in <module>
print symbolNonExistant
NameError: name 'symbolNonExistant' is not defined

Why did i need to see all that junk when all i really need to see was this:

Traceback (most recent call last):
File "C:/a/b/c/mod1.py", line 2, in <module>
print symbolNonExistant
NameError: name 'symbolNonExistant' is not defined

Or event better:

NameError: name 'symbolNonExistant' is not defined
File "C:/a/b/c/mod1.py", line 2, in <module>
print symbolNonExistant

 

[Oscar Benjamin]

Sometimes many levels of trace messages can be helpful when detecting bugs, however, in the case of NameErrors, these "nuggets" ejected from deep within the bowls of the Python interpreter are nothing more than steaming piles of incomprehensible crap!

We don't need multiple layers of traces for NameErrors. Python does not have *real* global variables; and thank Guido for that! All we need to know is which module the error occurred in AND which line of that module contains the offensive lookup of a name that does not exist.

[SNIP]

NameErrors can occur conditionally depending on e.g. the arguments to
a function. Consider the following script:

# tmp.py
def broken(x):
if x > 2:
print(x)
else:
print(undefined_name)

broken(1)

When run it gives a NameError with a traceback:

$ python tmp.py
Traceback (most recent call last):
File "tmp3.py", line 8, in <module>
broken(1)
File "tmp3.py", line 6, in broken
print(undefined_name)
NameError: global name 'undefined_name' is not defined

The traceback shows the arguments passed to the broken function that
caused the NameError to be generated. Different arguments would not
have generated the NameError. This information can be useful if the
logic of the function in question is complicated. It also hints at why
you were calling the function and what your code is trying to do.


Oscar

 

[Rick Johnson]

NameErrors can occur conditionally depending on e.g. the
arguments to a function. Consider the following script:

# tmp.py
def broken(x):
if x > 2:
print(x)
else:
print(undefined_name)

broken(1)

Why would anyone write code like that? That's like arming your toilet paperholder with a bomb set to explode if the RPMs of the spinning roll exceed a small threshold. Sure, you could do it, but why the hell would you? The only way your code could be any worse is by picking a random RPM threshold every morning!

import random
from home.bathroom import ToiletPaperHolder
RPMS = range(100)

def maybeGoBoom(event):
maxRpm = random.choice(RPMS)
if event.RPM > maxRpm:
roll.explode()

tph = ToiletPaperHolder()
if not tph.has_roll():
tph.load_roll()
roll = tph.get_active_roll()

The traceback shows the arguments passed to the broken
function that caused the NameError to be generated.
Different arguments would not have generated the
NameError. This information can be useful if the logic of
the function in question is complicated. It also hints at
why you were calling the function and what your code is
trying to do.

If you want to observe your code "in action" there are much better ways than eyeball-parsing lines and lines of trackbacks. The code you posted is nonsense, maybe you can provide a better example that will convince me, but that one failed miserably.

 

[Oscar Benjamin]

If you want to observe your code "in action" there are much better ways than eyeball-parsing lines and lines of trackbacks. The code you posted is nonsense, maybe you can provide a better example that will convince me, but that one failed miserably.

I wasn't looking to convince *you*, just to set the record straight
that this behaviour is sometimes useful. In any case, even when the
traceback information is not helpful, printing it is really not a
problem and hardly a "wart".


Oscar

 

[Tim Chase]

Why would anyone write code like that?

Because, in the real world, that example looks something like

def broken(intelligence_level):
if intelligence_level < 100:
return dumb_down(intellegence_level)
else:
return make_harder(intelligence_level)
broken(op.iq)

Pylint, pyflakes or some other such linter should catch it, but this
happens ALL THE TIME in actual development, occasionally leaking into
production.

-tkc

 

[Steven D'Aprano]

Sometimes many levels of trace messages can be helpful when detecting
bugs, however, in the case of NameErrors, these "nuggets" ejected from
deep within the bowls of the Python interpreter are nothing more than
steaming piles of incomprehensible crap!

We don't need multiple layers of traces for NameErrors. Python does not
have *real* global variables; and thank Guido for that! All we need to
know is which module the error occurred in AND which line of that
module contains the offensive lookup of a name that does not exist.

[SNIP]

/head-desk

Is Rick still pushing these stupid "PyWart" ideas?

NameErrors can occur conditionally depending on e.g. the arguments to a
function. Consider the following script:

[...]

Correct, although in your example, simply pointing at the relevant line
of code is enough to establish the error.

But that's an easy case. Tracebacks aren't printed because you need them
to fix the easy bugs. Tracebacks are printed so you have a hope of fixing
the hard bugs. NameError is no different in this than any other
exception, and the Zen applies:

Special cases aren't special enough to break the rules.

NameErrors are exceptions like any other. They aren't special enough to
suppress the full traceback when a NameError occurs. In the easy cases,
you can just ignore the full traceback, and no harm is done. In the hard
cases, you will need it.

Since name bindings ("variables") in Python are dynamic, not static,
whether or not a name exists at any time can depend *when* and *how* you
call a line of code, not just which line of code. That is, whether or not
a line of code will raise NameError can depend on which lines of code are
called before it, and that depends on the function call chain shown by
the traceback.

Here's a truly trivial case where code will succeed or fail depending on
the order of function calls.

def display():
print("spam = %d" % spam)

def start():
global spam
spam = 23

def stop():
global spam
del spam

def run():
print("*** Succeeds ***")
start()
display()
stop()

def fail():
print("*** Fails ***")
start()
stop()
display()


run()
fail()


It's not enough to know that the print line in display() fails, because
that's merely the side-effect. The actual problem occurs in the caller,
fail(). If NameError suppressed the traceback, that would be more
difficult to solve.

 

[Chris Angelico]

Because, in the real world, that example looks something like

def broken(intelligence_level):
if intelligence_level < 100:
return dumb_down(intellegence_level)
else:
return make_harder(intelligence_level)
broken(op.iq)

Pylint, pyflakes or some other such linter should catch it, but this
happens ALL THE TIME in actual development, occasionally leaking into
production.

That's actually an argument in favour of declared variables. NameError
becomes a parse-time failure

ChrisA

 

[Rick Johnson]

I wasn't looking to convince *you*, just to set the record
straight that this behaviour is sometimes useful.

And you claim to "set the record strait" by posting code that *purposely* raises a NameError when some function parameter is not within a predefined range? That's ludicrous!

Look, i don't want you to think that i am arguing with you, i just want you to show us an example that proves your argument to be true; but you cannot prove the argument by doing foolish things. Imagine the following scenario:

* CarMakerA claims their new automobile is safest on the
road.

* CarMakerB purposely drives the car into a ditch and then
claims the car is unsafe and CarMakerA is a liar.

That's what your example just did! Please provide a "real world" example that proves your argument. I am open to changing my mind *IF* someone can provide proof.

In any case, even when the traceback information is not
helpful, printing it is really not a problem and hardly a
"wart".

* Warts are ugly

* Superfluous trackbacks are not only ugly, they damage
productivity.

Therefore this *IS* a wart.

 

[Chris Angelico]

Here's a truly trivial case where code will succeed or fail depending on
the order of function calls.
(chop code)
It's not enough to know that the print line in display() fails, because
that's merely the side-effect. The actual problem occurs in the caller,
fail(). If NameError suppressed the traceback, that would be more
difficult to solve.

A good example. It would be logically equivalent to set spam=None in
stop(), and nobody would expect the TypeError to omit the traceback,
so why should delling the name be any different?

ChrisA

 

[Chris Angelico]

* Superfluous trackbacks are not only ugly, they damage
productivity.

Extraordinary claims require extraordinary evidence. Start evidencing.

ChrisA

 

[Rick Johnson]

[...]
NameErrors can occur conditionally depending on e.g. the
arguments to a function. Consider the following script:

[...]

Correct, although in your example, simply pointing at the
relevant line of code is enough to establish the error.
EXACTLY!

[...]
Here's a truly trivial case where code will succeed or
fail depending on the order of function calls.

def display():
print("spam = %d" % spam)

def start():
global spam
spam = 23

def stop():
global spam
del spam

def run():
print("*** Succeeds ***")
start()
display()
stop()

def fail():
print("*** Fails ***")
start()
stop()
display()

run()

fail()

It's not enough to know that the print line in display()
fails, because that's merely the side-effect. The actual
problem occurs in the caller, fail().

No, the "ACTUAL PROBLEM" is in the author.

Who would be stupid enough to write code that depends on globals that *may* or *may not* exist, and then go an add insult to injury by not testing for the name before executing the code? Your example is a fine example of why using globals is foolish.

Congratulations Steven, you drove the car into the ditch -- even a noob can do that!

 

[Michael Torrie]

No, the "ACTUAL PROBLEM" is in the author.

Surely any NameException can also be blamed on the author then, by your
logic?

 

[Steven D'Aprano]

Surely any NameException can also be blamed on the author then, by your
logic?

Any exception at all is obviously the author's fault. I propose that
Python stops wasting our time with debugging information and tracebacks,
and on any error, simply prints the following message then dump core:


PEBKACError: Programmer is an idiot. You did something wrong, you moron,
turn your computer off, you're obviously too stupid to program.


That will certainly improve productivity.

 

[Chris Angelico]

Any exception at all is obviously the author's fault. I propose that
Python stops wasting our time with debugging information and tracebacks,
and on any error, simply prints the following message then dump core:


PEBKACError: Programmer is an idiot. You did something wrong, you moron,
turn your computer off, you're obviously too stupid to program.


That will certainly improve productivity.

Why dump core? That seems far too useful to fit with the rest of your
proposal. Just exit 0 - after all, it wasn't Python's fault, so it
obviously succeeded.

ChrisA

 

[Benjamin Kaplan]

Any exception at all is obviously the author's fault. I propose that
Python stops wasting our time with debugging information and tracebacks,
and on any error, simply prints the following message then dump core:


PEBKACError: Programmer is an idiot. You did something wrong, you moron,
turn your computer off, you're obviously too stupid to program.


That will certainly improve productivity.

Don't have it dump core. Have it print a pink slip to the default printer.

 

[Jason Swails]

[snip junk]
We don't need multiple layers of traces for NameErrors. Python does not
have *real* global variables; and thank Guido for that! All we need to know
is which module the error occurred in AND which line of that module
contains the offensive lookup of a name that does not exist.
[snip more junk]

2 comments here.

1) Where's the consistency?? NameError is an exception. All other
exceptions get full tracebacks. A NameError is not special enough to
deserve special treatment (zen of Python? PEP 8? I don't remember). If
you like the full traceback (like I do), it's there. If you just want the
error, look at the last frame only.

==== Example where you want a full traceback ====

Pyflakes doesn't catch all NameErrors. If you don't set all of an object's
possible attributes inside its constructor, but some are applied later-on
(i.e., when they are needed, like in some sort of specialized setup routine
that you only call in certain circumstances), then you can benefit from a
full traceback even for a NameError. Consider the DataSet class below:

import numpy as np
from scipy.stats.kde import gaussian_kde as kde
class DataSet(object):

def __init__(self, data):
self._dataset = np.asarray(data)
def setup_kde(self):
"Set up KDE. Do not do by default since it is expensive for large
data sets"
self.kde = kde(self._dataset)
def resampled(self):
return DataSet(self.kde.resample())

In this case generating the KDE can become time- and RAM-consuming for
large data sets, so it's worthwhile to only generate the KDE if you
actually plan on doing something with that requires it. If you call
resampled before setup_kde, you get a NameError, and it would be helpful to
have the full traceback when debugging. Yes, you can get around having the
'non-trivial' NameError, but having the full traceback if someone _did_
write code like this makes your job a hell of a lot easier to debug if you
don't have to go stack tracing yourself. And for a large project with
multiple coders that is built in stages and has functionality added as it's
needed, this type of situation is not at all unusual. While one approach
is to take to the (email) streets and proclaim how incompetent all coders
that came before you truly were and that such a project is not worth
modifying in view of their ineptitude, the approach that does _not_ lead to
your firing would benefit from a full NameError traceback. Go ahead and
fire away about how stupid I am for the suggestion anyway, though.

2) Fine. You don't like long tracebacks for NameErrors, write a 8-line
module that you import at the top of every program (like division from
__future__ if you work with Python 2 like I have to):

--- file ricksstupididea.py ---
import sys, traceback
def excepthook(exctype, value, tb):
if exctype is NameError:
print 'Traceback (only printing last frame):'
print ' File "%s", line %d, in %s\n %s' %
traceback.extract_tb(tb).pop()
print 'NameError: %s' % value
else:
sys.__excepthook__(exctype, value, tb)
sys.excepthook = excepthook
--- end ricksstupididea.py ---

Once you do this, you get your special behavior for NameErrors like you
want. Observe, padawan:

-- begin nameerror.py --
import ricksstupididea
def func1(x):
return func2(x)

def func2(x):
return func3(x)

def func3(x):
return func4(x)

def func4(x):
return noname

func1(1)

$ python noname.py
Traceback (only printing last frame):
File "noname.py", line 12, in func4
return noname
NameError: global name 'noname' is not defined

-- begin divzero.py --
import ricksstupididea
def func1(x):
return func2(x)

def func2(x):
return func3(x)

def func3(x):
return func4(x)

def func4(x):
return x / 0

func1(1)

$ python divzero.py
Traceback (most recent call last):
File "divzero.py", line 14, in <module>
func1(1)
File "divzero.py", line 3, in func1
return func2(x)
File "divzero.py", line 6, in func2
return func3(x)
File "divzero.py", line 9, in func3
return func4(x)
File "divzero.py", line 12, in func4
return x / 0
ZeroDivisionError: integer division or modulo by zero

GvR's time machine strikes again. The only bad thing about this approach
is that you have to actually write that extra 8 lines of code, and I get to
keep the behavior I like rather than being forced into the (inconsistent)
behavior you would prefer I use instead. But since the 8 lines has been
provided to you here free of charge, the only real cost for you is how I
like my NameErrors given to me.

I hereby leave you enlightened.

</rant>
Jason