args attribute of Exception objects

[Sébastien de Menten]

Hi,

When I need to make sense of a python exception, I often need to parse the
string exception in order to retrieve the data.
Example:

try:
print foo
except NameError, e:
print e.args
symbol = e.args[0][17:-16]
==> ("NameError: name 'foo' is not defined", )

or

try:
(4).foo
except NameError, e:
print e.args
==> ("'int' object has no attribute 'foo'",)

Moreover, in the documentation about Exception, I read
"""Warning: Messages to exceptions are not part of the Python API. Their
contents may change from one version of Python to the next without warning
and should not be relied on by code which will run under multiple versions
of the interpreter. """

So even args could not be relied upon !

Two questions:
1) did I miss something in dealing with exceptions ?
2) Could this be changed to .args more in line with:
a) first example: e.args = ('foo', "NameError: name 'foo' is not
defined")
b) second example: e.args = (4, 'foo', "'int' object has no attribute
'foo'",)
the message of the string can even be retrieved with str(e) so it is also
redundant.
BTW, the Warning in the doc enables to change this To be backward
compatible, the error message could also be the first element of the tuple.

Seb

ps: There may be problems (that I am not aware) with an exception keeping
references to other objects

 

[Jeremy Bowers]

Hi,

When I need to make sense of a python exception, I often need to parse the
string exception in order to retrieve the data.

What exactly are you doing with this info? (Every time I started to do
this, I found a better way. Perhaps one of them will apply for you.)

(As a general comment, I'd point out that you don't have to check the
entire error message; checking for a descriptive substring, while still
not "safe", is at least safe*r*.)

 

[Sebastien de Menten]

What exactly are you doing with this info? (Every time I started to do
this, I found a better way. Perhaps one of them will apply for you.)

(As a general comment, I'd point out that you don't have to check the
entire error message; checking for a descriptive substring, while still
not "safe", is at least safe*r*.)

I have symbolic expressions in a dictionnary like:

dct = dict( a = "b**2 + c", b = "cos(2.3) + sin(v)", v = "4", c =
"some_very_expensive_function(v)")

I want to build a function that finds the links between all those
expressions (think about computing dependencies between cells in a
spreadsheet).

All I do is:
def link(name):
dependencies = {}
while True:
try:
eval(dct[name], globals(), dependencies)
except NameError,e:
dependencies[e.args[0][6:-16]] = 1
else:
return dependencies

globals() can be replaced by a custom dictionnary for security
purposes.
variation on the theme can:
- check SyntaxError and give interlligent feedback to user (BTW,
SyntaxError args are much smarter)
- find or/and eval recursively the whole tree and keep in cache
values,...

Seb

 

[Steve Holden]

What exactly are you doing with this info? (Every time I started to do
this, I found a better way. Perhaps one of them will apply for you.)

(As a general comment, I'd point out that you don't have to check the
entire error message; checking for a descriptive substring, while still
not "safe", is at least safe*r*.)

I have symbolic expressions in a dictionnary like:

dct = dict( a = "b**2 + c", b = "cos(2.3) + sin(v)", v = "4", c =
"some_very_expensive_function(v)")

I want to build a function that finds the links between all those
expressions (think about computing dependencies between cells in a
spreadsheet).

All I do is:
def link(name):
dependencies = {}
while True:
try:
eval(dct[name], globals(), dependencies)
except NameError,e:
dependencies[e.args[0][6:-16]] = 1
else:
return dependencies

globals() can be replaced by a custom dictionnary for security
purposes.
variation on the theme can:
- check SyntaxError and give interlligent feedback to user (BTW,
SyntaxError args are much smarter)
- find or/and eval recursively the whole tree and keep in cache
values,...

Seb

Wouldn't it actually be better to use a namespace that reported attempts
to use its members? Then when you evaluated an expression you would be
in control of how the attempted accesses were recorded, and could
provide exactly the needed information.

regards
Steve

 

[Bengt Richter]

I have symbolic expressions in a dictionnary like:

dct = dict( a = "b**2 + c", b = "cos(2.3) + sin(v)", v = "4", c =
"some_very_expensive_function(v)")

I want to build a function that finds the links between all those
expressions (think about computing dependencies between cells in a
spreadsheet).

Here's a way of finding the symbols being used in the expressions, and dependencies:
(no guarantees, I haven't explored any special corner cases or caught syntax errors etc ;-)

And note that eval is not safe even for expressions, if you don't know what's in the string.
A safer approach would be to use compiler.parse and a visitor to extract names from the ast.
... a = "b**2 + c",
... b = "cos(2.3) + sin(v)",
... v = "4",
... c = "some_very_expensive_function(v)") ... seen = set()
... def _tree(sym, level=0):
... print '%s%s'%(' '*level, sym)
... seen.add(sym)
... for dep in depdict.get(sym, []): _tree(dep, level+1)
... for sym in sorted(depdict.keys()):
... if sym in seen: continue
... _tree(sym)
...

>>> tree(sorted(symdep.keys())[0], symdep)

a
b
cos
sin
v
c
some_very_expensive_function
v

Skipping fancy visitor stuff ;-), getting names brute force from the ast, should be safer
than eval('lambda:'+expr).func_code.co_names):
(not tested beyond what you see ;-)
... ast = compiler.parse(expr, 'eval')
... names = []
... def descend(node):
... if isinstance(node, compiler.ast.Name): names.append(node.name)
... if not hasattr(node, 'getChildren'): return
... for child in node.getChildren(): descend(child)
... descend(ast)
... return sorted(names)
...

>>> getnames('a*b+c**foo(x)') ['a', 'b', 'c', 'foo', 'x']
>>> for v,x in sorted(dct.items()):

... print '%5s: %s'%(v, getnames(x))
...
a: ['b', 'c']
b: ['cos', 'sin', 'v']
c: ['some_very_expensive_function', 'v']
v: []

All I do is:
def link(name):
dependencies = {}
while True:
try:
eval(dct[name], globals(), dependencies)
except NameError,e:
dependencies[e.args[0][6:-16]] = 1
else:
return dependencies

globals() can be replaced by a custom dictionnary for security
purposes.

I would use something getnames above, and put getnames(expr)
in the expression for symdep = dict(...) in place of
sorted(eval('lambda:'+expr).func_code.co_names))

variation on the theme can:
- check SyntaxError and give interlligent feedback to user (BTW,
SyntaxError args are much smarter)

compiler.parse will raise SyntaxError in getnames, so you could make your own message, e.g.,
... except SyntaxError, e:
... print 'SyntaxError: %s<<--BROKE HERE-->>%s' % (e.text[:e.offset], e.text[e.offset:])
...

- find or/and eval recursively the whole tree and keep in cache
values,...

HTH

Regards,
Bengt Richter

 

[Sebastien de Menten]

Thank you guys for those good advices (and the very interesting
example of AST hacking).

However, this was an example of use for Exception.args. It does not
alleviate my concerns about the fact that the args attribute is poorly
designed for standard Exceptions.
It is as if the Exception design was only made for end users (display
of a string in an interpreter) without thinking about original ways to
use them

Seb

 

[Bengt Richter]

Thank you guys for those good advices (and the very interesting
example of AST hacking).

However, this was an example of use for Exception.args. It does not
alleviate my concerns about the fact that the args attribute is poorly
designed for standard Exceptions.
It is as if the Exception design was only made for end users (display
of a string in an interpreter) without thinking about original ways to
use them

Look into it some more ;-)

Have you thought of subclassing Exception for your own purpose?
You can override the base exception class's Exception.__init__
which UIAM effectively does

def __init__(self, *args):
self.args = args

and no more. Using an existing special purpose exception like NameError
(IIRC seeing that correctly) to carry info for you is pretty weird ;-)

Here is a subclass that takes what would be e.args[0] and delivers it
instead as e.answer (well, "e" is "retans" below ;-)
... def __init__(self, answer):
... self.answer = answer
...

>>> for ans in ['string', 123, 4.5, 67L, 8+9j,

... type('MyClass',(),{}), type('MyInst',(),{})()]:
... try: raise BailingOutWithAnswer(ans)
... except BailingOutWithAnswer, retans:
... print '%r -> retans.answer: %r' %(ans, retans.answer)
...
'string' -> retans.answer: 'string'
123 -> retans.answer: 123
4.5 -> retans.answer: 4.5
67L -> retans.answer: 67L
(8+9j) -> retans.answer: (8+9j)
<class '__main__.MyClass'> -> retans.answer: <class '__main__.MyClass'>
<__main__.MyInst object at 0x02EF14EC> -> retans.answer: <__main__.MyInst object at 0x02EF14EC>

It's the easiest thing in the world to define a minimal Exception subclass to give you
back in e.args whatever you pass it as args when you raise it. Obviously I could have used
a table of arg tuples and just raise XEasy(*argtuples) instead of separating them out,
but I wanted to show how plain raise expression syntax with
different numbers of arguments map to e.args.
... try:
... if i==0: raise XEasy(123)
... elif i==1: raise XEasy(4.5, 67L)
... elif i==2: raise XEasy(*'this may seem tricky ;-)'.split())
... elif i==3: raise XEasy, 'usual ordinary message form'
... elif i==4: raise XEasy(8+9j, type('AClass',(),{}))
... elif i==5: raise XEasy(type('AClass',(),{})(),'<=[class instance]')
... elif i==6: raise XEasy('--------------------------------------------')
... except XEasy, e:
... print e.args
...
(123,)
(4.5, 67L)
('this', 'may', 'seem', 'tricky', ';-)')
('usual ordinary message form',)
((8+9j), <class '__main__.AClass'>)
(<__main__.AClass object at 0x02EF166C>, '<=[class instance]')
('--------------------------------------------',)

Oops, left out the no-arg raises
... except XEasy, e: print e.args
...
() ... except XEasy, e: print e.args
...
()

Did I misunderstand the problem again?

Regards,
Bengt Richter

 

[Sebastien de Menten]

Did I misunderstand the problem again?

Hmmm, yes ! But I think I am not expressing clearly my concern

So my problem is that "default exception" are badly designed in their
use of their args attribute.

I know it is possible to subclass Exception as every object and add
the attributes I want with the name I want, etc...

BUT, some (if not most) default exceptions in __builtins__ use the
args attribute and put a string in it without delivering interesting
information about the exception.

So 1) I think it is nice to keep args as name for the attribute as it
avoids looking at the documentation for looking at each specific
exception/attribute name.
2) but args should be more cleverly designed (again, I am speaking
about exceptions in __builtins__ not user defined exceptions) and not
be a simple string.

Now, forcing user to avoid using default exceptions (i.e. force them
to use user defined exceptions) for anything but interactive use or
debugging is short minded (nobody says that, but the current situation
implies that).

Anyway, I log this as request on SF.

Best regards,

Sed