Different kinds of Import Errors

[Thomas Guettler]

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise


I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

 

[kyosohma]

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise

I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

I would assume that in the future versions of Python, it would still
mention the word "management". In that case, you could do something
like this in the except clause:

# untested
args = exc.args[0]
if args.find('management') != -1:
raise


Mike

 

[Aahz]

# untested
args = exc.args[0]
if args.find('management') != -1:
raise

YM

if 'management' in args:
raise

HTH, HAND ;-)

 

[kyosohma]

# untested
args = exc.args[0]
if args.find('management') != -1:
raise

YM

if 'management' in args:
raise

HTH, HAND ;-)

Oops...I've used this method myself. Your method is definitely nicer
and likelier more "Pythonic" than mine. Live and learn.

Mike

 

[Graham Dumpleton]

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise

I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

Perhaps check for the presence of the module in sys.modules.

if (app_name + '.management') in sys.modules:
raise

If not there, module couldn't be found. If module there but exception
occurred then some other import related error occurred.

Graham

 

[Peter Otten]

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise


I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

An alternative way to determine the expected message:

try:
import sys.pink_elephant
except ImportError, e:
expected_message = e.message.replace("pink_elephant", "management")
else:
raise Exception("your python has swallowed a pink elephant")

I'm probably not serious.

Peter

 

[Thomas Guettler]

Sorry, but this does not work. If there is an ImportError
during importing the existing module, it won't get inserted
into sys.modules. I just tried it with a small example.

An other solution would be to inspect the traceback. If the
app_name+'.management' is in it, it exists.

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise

I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

Perhaps check for the presence of the module in sys.modules.

if (app_name + '.management') in sys.modules:
raise

If not there, module couldn't be found. If module there but exception
occurred then some other import related error occurred.

Graham

 

[Graham Dumpleton]

Sorry, but this does not work. If there is an ImportError
during importing the existing module, it won't get inserted
into sys.modules. I just tried it with a small example.

An other solution would be to inspect the traceback. If the
app_name+'.management' is in it, it exists.

If you look at this code, you see there are two kind of ImportErrors:
1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise
# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise
I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.
Any better solution?

Perhaps check for the presence of the module in sys.modules.

if (app_name + '.management') in sys.modules:
raise

If not there, module couldn't be found. If module there but exception
occurred then some other import related error occurred.

What example did you use to test it? What version of Python are you
using?

Curious, as at least for Python 2.3 it behaves as I described with the
simple examples I tested. I do vaguely remember behaviour related to
this changing though, so possible that newer version works
differently.

First try case of importing module that doesn't exist:
Traceback (most recent call last):

sys.modules['z']

Traceback (most recent call last):
File "<stdin>", line 1, in ?
KeyError: 'z'

Thus ImportError occurred, and not in sys.modules. Therefore module
must not have been able to be found.

Now lets try existing module, but non ImportError case first:

$ cat a.py
print 'a1'
xxx
print 'a2'

import sys
sys.modules['a']

Traceback (most recent call last):
a1
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "a.py", line 2, in ?
xxx
NameError: name 'xxx' is not defined

sys.modules['a']

<module 'a' from 'a.py'>

Thus, although an exception occurred in importing file, the module is
still in sys.modules.

Now lets try for an ImportError for where target module exists:

$ cat a.py
print 'a1'
import z
print 'a2'
a1
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "a.py", line 2, in ?
import z
ImportError: No module named z

sys.modules['a']

<module 'a' from 'a.py'>

In this case ImportError occurred but is in sys.modules, thus
ImportError can only have been related to a sub module import as top
level module did actually exist.

I'll have to try this at work next week on newer version of Python
than my Mac has.

Graham

 

[Thomas Guettler]

What example did you use to test it? What version of Python are you
using?

Yes, this changed. Python 2.3 includes the half imported module. Python 2.4 does not.
But the traceback of the exception contains the needed information:

Here are the two example files:

# foo.py
import sys
print sys.version
try:
import foomod
except ImportError, exc:
mods=sys.modules.keys()
mods.sort()
print mods
import traceback
print traceback.extract_tb(sys.exc_info()[2])

# foomod.py
import does_not_exist

 

[Gabriel Genellina]

Curious, as at least for Python 2.3 it behaves as I described with the
simple examples I tested. I do vaguely remember behaviour related to
this changing though, so possible that newer version works
differently.

Yes. It was changed on 2.4. From http://www.python.org/doc/2.4/whatsnew/node12.html:

"Encountering a failure while importing a module no longer leaves a
partially-initialized module object in sys.modules. The incomplete
module object left behind would fool further imports of the same
module into succeeding, leading to confusing errors. (Fixed by Tim
Peters.)"

 

[Jeremy C B Nicoll]

If you look at this code, you see there are two kind of ImportErrors:

1. app_name has no attribute or file managment.py: That's OK.
2. managment.py exists, but raises an ImportError: That's not OK: reraise

# Import the 'management' module within each installed app, to register
# dispatcher events.
for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise


I am searching a better solution, since in a future version of python
the string 'No module namend management' might be changed.

Any better solution?

Don't know; I want to ask a possibly related question. My impression is
that while writing and testing something like the above code one has somehow
to know that "ImportError" can be raised, so one can explicitly define how
it is to be treated. How does one find that out?

While in this case, one might hope that an import action could only fail
with ImportError, isn't it possible that - say - one might get an IOError
because of a problem on the module search path ... and maybe there are lots
of other less obvious exceptions that could be raised?

How does one write a try/except piece of code that works (ie traps whatever
exception occurs, though obviously it can't necessarily fix an arbitrary
exception) for any exception, even those not known of in advance by the
author?

 

[Steven D'Aprano]

for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise

....
I want to ask a possibly related question. My impression is
that while writing and testing something like the above code one has
somehow to know that "ImportError" can be raised, so one can explicitly
define how it is to be treated. How does one find that out?

Reading the docs. Trial and error. Background knowledge about what the
function is doing. There's no perfect solution; unfortunately many
functions fail to document what exceptions they can be expected to raise.

In principle, any exception that isn't documented is a bug. You can't be
expected to recover from bugs, all you can do is report it to the
function author.

While in this case, one might hope that an import action could only fail
with ImportError, isn't it possible that - say - one might get an
IOError because of a problem on the module search path ... and maybe
there are lots of other less obvious exceptions that could be raised?

The author of a function has a choice to make:

(1) expose any exceptions that are raised by his code to the caller; or

(2) swallow those exceptions and raise one or two known exceptions.

Neither is the "right" answer in all circumstances. Sometimes you might
want to just propagate exceptions, especially if the caller is in a
position to do something about them. Sometimes the right solution is to
hide those exceptions, and just expose a single MyModuleError exception
to callers.

Sometimes neither is ideal.

How does one write a try/except piece of code that works (ie traps
whatever exception occurs, though obviously it can't necessarily fix an
arbitrary exception) for any exception, even those not known of in
advance by the author?

Generally, you don't. If you can't recover from an exception, there's
little point in trapping it. Exceptions are:

(1) You're writing some sort of library function and you want to hide the
internal errors that occur, so you might write:

def function():
try:
processing()
except (IndexError, NameError):
raise MyModuleError("bad things happened")
except (IOError, MemoryError):
raise MyModuleError("your computer is broken")


(But notice how much potentially useful information you've thrown away by
doing this. Are you sure you want to hide that?)


(2) At the very top level of your application, where you might do
something like this:

# Untested
if __name__ == '__main__':
try:
main()
except (SystemExit, KeyboardInterrupt):
raise
except Exception, e:
log_exception(e)
print >>sys.stderr, "Die die die!!!"
sys.exit(45)
except: # Catch things that don't inherit from exceptions
print >>sys.stderr, "This should never happen"
sys.exit(46)



Hope that helps.

 

[Jeremy C B Nicoll]

for app_name in settings.INSTALLED_APPS:
try:
__import__(app_name + '.management', {}, {}, [''])
except ImportError, exc:
if exc.args[0]!='No module named management':
raise

...
I want to ask a possibly related question. My impression is
that while writing and testing something like the above code one has
somehow to know that "ImportError" can be raised, so one can explicitly
define how it is to be treated. How does one find that out?

Reading the docs. Trial and error. Background knowledge about what the
function is doing. There's no perfect solution; unfortunately many
functions fail to document what exceptions they can be expected to raise.

In principle, any exception that isn't documented is a bug. You can't be
expected to recover from bugs, all you can do is report it to the
function author.

While I've always been an advocate of (in other languages) checking return
codes from things, that's been when generally there's only been a small
number of those documented, or at least they could be split into "it wored
ok", "it almost worked", "minor problem", "huge problem". So quite often
one could write some fairly generic error handling based only on rc being 0,
or <8, or <20 or something. That's not to say that within that one couldn't
record the exact rc, error messages that went with them etc and pass them on
(to the caller, say).


Python's system seems messier.

The author of a function has a choice to make:

(1) expose any exceptions that are raised by his code to the caller; or

(2) swallow those exceptions and raise one or two known exceptions.

Neither is the "right" answer in all circumstances. Sometimes you might
want to just propagate exceptions, especially if the caller is in a
position to do something about them. Sometimes the right solution is to
hide those exceptions, and just expose a single MyModuleError exception
to callers.

Sometimes neither is ideal.



Generally, you don't. If you can't recover from an exception, there's
little point in trapping it.

The reason I want to trap them is so as not to have a more naive user
suddenly be faced with an unfriendly traceback. I'd want my program to save
details of what it'd been doing, the exception type etc, do its own cleanup
of whatever it was doing, report it had a problem, and then stop in a
'clean' fashion. I can look at the details later on...

Exceptions are:

(1) You're writing some sort of library function and you want to hide the
internal errors that occur, so you might write:

def function():
try:
processing()
except (IndexError, NameError):
raise MyModuleError("bad things happened")
except (IOError, MemoryError):
raise MyModuleError("your computer is broken")


(But notice how much potentially useful information you've thrown away by
doing this. Are you sure you want to hide that?)

It seems to me that I'd generally want something like:

def function():
try:
processing()
except (IndexError, NameError):
raise MyModuleError("bad things happened")
except (IOError, MemoryError):
raise MyModuleError("your computer is broken")
except (*):
raise MyModuleError("wallop! :" + details() )

where in the last section I want to trap any exception that wasn't one of
the named/expected ones. Your

raise MyModuleError("blah")

examples suggest that an exception can have just one string of details,
presuably passed back up the line... Is that the case or can one pass a
list/tuple of symptoms?

I did see a example somewhere something like:

except (SomeError), s:

in which 's' was then set to a list of - I think - two subsidiary values
which the exception handler could look at. But I don't know if the
equivalent of 's' for any exception always has just 2 values. And I
couldn't get anything like:

except s:

or

except *, s:

or

except (), s:

etc to work meaning "match an arbitrary exception but give me access to
whatever symptoms or location-of-error or whatever" info is available.



I also find that if the function concerned actually is a function (ie
returns something if it works properly) it's easier to think (as I have in
other languages) of it returning error information the same way, so eg:

def function2():
try:
what = processing()
except (IndexError, NameError):
what = "Error: details type 1"
except (IOError, MemoryError):
what = "Error: details type 2"
except (*):
what = "Error: unexpected: " + details()
return what

so that then the caller of the function bases their next action on whether
or not the returned value starts "Error:". That way I know that control
always comes back from that function via that return, rather than either by
a return or an exception. I don't like functions/procedures with multiple
exit points (though I realise there's a difference between normal return and
exceptional circumstance returns).

Hope that helps.

Oh, it does... Thank-you.