is parameter an iterable?

[Roy Smith]

try:
for item in obj:
do_stuff(item)
except TypeError, msg:
if msg == "iteration over non-sequence":
handle_non_iterator()
else:
# re-raise the exception
raise

That's the obvious solution, but it's a poor one because it depends on an
undocumented feature of the language. What's documented is that TypeError
is raised; it's not documented what the text of the message will be.

 

[Steven D'Aprano]

But what it do_stuff tries to iterate over a non-sequence?

I can think of two solutions:

(1) Write do_stuff carefully, and test it thoroughly on its own. If there
are no bugs in do_stuff, you can be sure it isn't going to try to iterate
over a non-iterator. This is a good argument for writing small,
encapsulated, easily analysed and debugged pieces of code.

(2) Muck about writing fragile, possibly buggy code:

# untested
try:
for item in obj:
flag = False
try:
do_stuff(item)
except TypeError, msg:
if msg == "iteration over non-sequence":
flag = True
# re-raise the exception
raise
except TypeError, msg:
if flag:
raise
if msg == "iteration over non-sequence":
handle_non_iterator()
else:
raise



I think you can guess where my preference lies.

 

[Steven D'Aprano]

That's the obvious solution, but it's a poor one because it depends on an
undocumented feature of the language. What's documented is that TypeError
is raised; it's not documented what the text of the message will be.

It would be nice if The Powers That Be would document the specific error
messages, so that we could rely on them in total safety.

But even without that, the consequences aren't especially dire. What
happens if the error message changes in some future version of Python?
Then the error won't be caught, the exception will be re-raised, and your
testing will catch it immediately.


It isn't the ideal solution, but it is a solution.

 

[Rick Wotnaz]

def foo(inputVal):
try:
for val in inputVal:
# do stuff
except TypeError, msg:
if msg == "iteration over non-sequence":
# handle non-iterable case
else:
# some other TypeError is a bug, so re-raise the
exception raise

Does this in fact work on your system? On mine (2.4.1 (#65, Mar 30
2005, 09:13:57) [MSC v.1310 32 bit (Intel)]), it doesn't seem to. I
tried
if msg.find("iteration over non-sequence") >= 0:
.... but I got a traceback, and
AttributeError: TypeError instance has no attribute 'find'
.... which leads me to belive that 'msg' is not type(str). It can be
coerced (str(msg).find works as expected). But what exactly is msg?
It appears to be of <type 'instance'>, and does not test equal to a
string. This is not the least surprise to me.

 

[Roy Smith]

def foo(inputVal):
try:
for val in inputVal:
# do stuff
except TypeError, msg:
if msg == "iteration over non-sequence":
# handle non-iterable case
else:
# some other TypeError is a bug, so re-raise the
exception raise

Does this in fact work on your system? On mine (2.4.1 (#65, Mar 30
2005, 09:13:57) [MSC v.1310 32 bit (Intel)]), it doesn't seem to. I
tried
if msg.find("iteration over non-sequence") >= 0:
... but I got a traceback, and
AttributeError: TypeError instance has no attribute 'find'
... which leads me to belive that 'msg' is not type(str). It can be
coerced (str(msg).find works as expected). But what exactly is msg?
It appears to be of <type 'instance'>, and does not test equal to a
string. This is not the least surprise to me.

It's an easy experiment to do:

-------------------
Roy-Smiths-Computerlay$ cat ex.py
#!/usr/bin/env python

try:
1 + "foo"
except TypeError, msg:
print type(msg)
print msg
print repr(msg)
print dir(msg)
Roy-Smiths-Computerlay$ py ex.py
<type 'instance'>
unsupported operand type(s) for +: 'int' and 'str'
<exceptions.TypeError instance at 0x36d968>
['__doc__', '__getitem__', '__init__', '__module__', '__str__', 'args']
---------------------

 

[Fredrik Lundh]

Rick Wotnaz wrote.

... which leads me to belive that 'msg' is not type(str). It can be
coerced (str(msg).find works as expected). But what exactly is msg?
It appears to be of <type 'instance'>, and does not test equal to a
string.

it's an instance of the exception type, of course.

:::

if you do

raise SomeError, value

Python will actually do

raise SomeError(value)

(that is, create a SomeError exception and pass the value as its
first argument).

you can use either form in your code (I prefer the latter myself).

:::

as for catching the exceptions, if you do

try:
...
except SomeError, v:
...

Python will treat this as

try:
...
except:
# some exception occurred
typ = sys.exc_type
exc = sys.exc_value
if issubclass(typ, SomeError):
v = exc
...
else:
raise # propagate!

(where typ and exc are internal variables)

</F>

 

[Rick Wotnaz]

Rick Wotnaz wrote.


it's an instance of the exception type, of course.

:::

if you do

raise SomeError, value

Python will actually do

raise SomeError(value)

(that is, create a SomeError exception and pass the value as its
first argument).

you can use either form in your code (I prefer the latter
myself).

:::

as for catching the exceptions, if you do

try:
...
except SomeError, v:
...

Python will treat this as

try:
...
except:
# some exception occurred
typ = sys.exc_type
exc = sys.exc_value
if issubclass(typ, SomeError):
v = exc
...
else:
raise # propagate!

(where typ and exc are internal variables)

Thank you (and Roy Smith) for helping to clarify this. I see that
my mental image of an Exception (which, I admit, was not based on
extensive R'ing of TFM) was way off. Judging by Steven D'Aprano's
code sample, I'm not the only one who was mistaken about the nature
of v in your example. I'd always assumed it was the human-
readable string associated with the TypeError. Wrong, I see.

 

[Steven D'Aprano]

On Wed, 16 Nov 2005 09:06:01 -0500, Rick Wotnaz wrote:

[cutting to the important bit]

Does this in fact work on your system? On mine (2.4.1 (#65, Mar 30
2005, 09:13:57) [MSC v.1310 32 bit (Intel)]), it doesn't seem to.

Dammit, that will teach me not to test my code before posting.

No it doesn't: msg is an object of type exceptions.TypeError.

The easy fix is to just coerce it to a string:

if str(msg) == "iteration over non-sequence":

which *does* work on my system. But perhaps a better way is to do this:


# Create an instance of the exception you expect:
try:
for i in 0:
pass
except TypeError, ITER_OVER_NON_SEQ:
pass
# Now run your code...
try:
...blah blah blah...
except TypeError, msg
if str(msg) == str(ITER_OVER_NON_SEQ):
...blah blah blah...

This means we're no longer assuming what the error message will be,
which makes our code a lot more future-proof and implementation-proof: if
some version of Python changes the error string from "iteration over
non-sequence" to something else, the code should continue to work
correctly.

 

[Fredrik Lundh]

This means we're no longer assuming what the error message will be,
which makes our code a lot more future-proof and implementation-proof: if
some version of Python changes the error string from "iteration over
non-sequence" to something else, the code should continue to work
correctly.

Alex has already posted the right way to do this. can you please stop

</F>

 

[Fredrik Lundh]

Alex has already posted the right way to do this. can you please stop

posting crappy non-solutions to a problem that has a very simple solution (split
things up), that should be obvious to anyone who didn't sleep through exceptions
101.

</F>

 

[Bengt Richter]

Rick Wotnaz wrote.


it's an instance of the exception type, of course.

:::

if you do

raise SomeError, value

Python will actually do

raise SomeError(value)

Depending on what you mean by "actually" I guess ...
(I'm sure you know this and more ;-)
1 0 LOAD_NAME 0 (SomeError)
3 LOAD_NAME 1 (value)
6 RAISE_VARARGS 2
9 LOAD_CONST 0 (None)
12 RETURN_VALUE 1 0 LOAD_NAME 0 (SomeError)
3 LOAD_NAME 1 (value)
6 CALL_FUNCTION 1
9 RAISE_VARARGS 1
12 LOAD_CONST 0 (None)
15 RETURN_VALUE

I guess that comes from the grammar of the raise statement, i.e.,

raise_stmt: 'raise' [test [',' test [',' test]]]

which allows up to three arguments for raise, apparently all general
expressions, but with some specific run-time requirements for what
combinations of argument expression values are allowable.

I.e., it seems (I haven't looked in ceval.c(?)) that RAISE_VARARGS must
look at the first item in its count of args on the stack, and in the
SomeError, value case find an exception class, and decide to instantiate it
and throw the instance, but if it finds the instance ready made, as in
SomeError(value), it must skip the instantiation (and disallow further args BTW).
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in ?
Exception: arg

Those looks the same, but sensitivity to the type of the first arg is revealed by
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in ?
Exception: ('arg', 'what now?')

(that is, create a SomeError exception and pass the value as its
first argument).

you can use either form in your code (I prefer the latter myself).

Just to drive home the general expression allowability in raise,
and instance vs class as the first arg:

>>> extab = [StopIteration('stop stop ;-)'), ValueError('wrong value')]
>>> raise extab[1]

Traceback (most recent call last):

>>> raise extab[0]

Traceback (most recent call last):

>>> extab = [StopIteration, 'stop stop ;-)', ValueError, 'wrong value']
>>> raise extab[2], extab[3]

Traceback (most recent call last):

>>> raise extab[0], extab[1]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
StopIteration: stop stop ;-)

Ok, I'll stop ;-)

Regards,
Bengt Richter

 

[Roy Smith]

# Create an instance of the exception you expect:
try:
for i in 0:
pass
except TypeError, ITER_OVER_NON_SEQ:
pass
# Now run your code...
try:
...blah blah blah...
except TypeError, msg
if str(msg) == str(ITER_OVER_NON_SEQ):
...blah blah blah...

This means we're no longer assuming what the error message will be,
which makes our code a lot more future-proof and implementation-proof: if
some version of Python changes the error string from "iteration over
non-sequence" to something else, the code should continue to work
correctly.

Now you're assuming that the message is a constant for all TypeErrors
caused by attempting to iterate over a non-iterable object. I don't
see anything in the docs which prevents

try:
for i in 0:
pass
except TypeError, ex:
print ex

from printing "Cannot iterate over integer with value 0", or even
"Cannot iteratate over object at address <0x456778>". Let's not even
talk about some future implementor who decides that the strings should
contain timestamps. None of these things are forbidden by the spec,
therefore you should assume they will all happen.

The best solution I've seen suggested so far is to coerce the
passed-in object to an iter and see what happens:

try:
thingieIterator = iter (thingie)
except TypeError:
raise NonIterableThingieError

Once you've done that, I think you can either do:

for item in thingie:

or

for item in thingieIterator:

with equal effect. And I'm reasonably sure this will work fine with
non-idempotent thingies.

 

[Steven D'Aprano]

Steven D'Aprano wrote:




Alex has already posted the right way to do this.

Has he? Where and when? Is my crystal ball broken
again? I hate it when I miss the one Usenet post all
the cool kids are talking about.

can you please stop

Can I please stop what? Using punctuation and
capitalization correctly? Learning? Thinking for
myself? Posting to Usenet? Programming in Python? Your
post is unclear.

 

[Steven D'Aprano]

Ah, found the right solution! Thanks to Fredrik Lundh
for so gently pointing me at this post. Just one
question, if I may:-

It's not hard...:

try:
_it = iter(whatever)
except TypeError:
print 'non-iterable'
else:
for i in _it: # etc, etc

Alas and alack, I have to write code which is backwards
compatible with older versions of Python:

Python 2.1.1 (#1, Aug 25 2001, 04:19:08)
[GCC 3.0.1] on sunos5
Type "copyright", "credits" or "license" for more
information.Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'iter' is not defined

What should I do when I can't rely on functions that
don't exist in older versions of Python?

 

[Steven D'Aprano]

Now you're assuming that the message is a constant for all TypeErrors
caused by attempting to iterate over a non-iterable object.

Yes I am, and of course you are correct that that could
change and so shouldn't be relied on.

Seems that my crappy solution is not quite as
future-proof as I thought it was. Since a better
solution does exist, I agree it should be used in
preference to my crappy one, assuming you are running a
recent enough version of Python.

 

[Duncan Booth]

What should I do when I can't rely on functions that
don't exist in older versions of Python?

Ideally:

if 'iter' not in dir(__builtins__):
import sys
sys.exit('Archaic Python not supported, please upgrade')


Alternatively fill in the blanks with appropriate fallback code:

if 'iter' not in dir(__builtins__):
def iter(obj):
... remainder of definition left as an exercise ...
__builtins__.iter = iter

where iter does roughly what the iter builtin does.

 

[Roy Smith]

Alas and alack, I have to write code which is backwards
compatible with older versions of Python:
[...]
NameError: name 'iter' is not defined

What should I do when I can't rely on functions that
don't exist in older versions of Python?

It really sucks trying to support obsolete environments. Been there, done
that. But sometimes you just have to pull out the duct tape and deal with
the problem any way you can. Parsing undocumented exception strings is
sort of like screen scraping; it's good to know that it's possible as an
absolute last resort, but you really want to put some effort into finding a
better way.

You could use two nested try blocks that both catch TypeErrors, one
enclosing the entire loop, the other enclosing just the body. Anything
that gets caught by the outer try had to have been generated within the
loop control code itself:

-------------------------
Roy-Smiths-Computerlay$ cat iter.py
#!/usr/bin/env python

def doSomething (obj):
sum = 0
try:
for i in obj:
try:
sum += i
except TypeError, ex:
print "exception (%s) in the inner block" % ex
return
except TypeError, ex:
print "non iterable object (%s)" % ex
return
print "sum = %d" % sum

doSomething ([1, 2, 3])
doSomething (["one", "two", "three"])
doSomething (0)
Roy-Smiths-Computerlay$ ./iter.py
sum = 6
exception (unsupported operand type(s) for +=: 'int' and 'str') in the
inner block
non iterable object (iteration over non-sequence)
-------------------------

This is pretty ugly, but I think it does what you want. It could be made a
bit less ugly by refactoring the body of the for loop into another function.

BTW, if you do resort to trying to parse the exception messages, and it
breaks in some future code, you might have me to blame. I submitted a bug
report a while back which, if it's ever acted on, would break your code!

http://sourceforge.net/tracker/index.php?func=detail&aid=1187437&group_id=54
70&atid=105470

 

[Micah Elliott]

sys.exit('Archaic Python not supported, please upgrade')

+1 QOTW.

I recently gave up on trying to support (backport to) pre-2.2 in my
projects. It's been ~3 years since 2.2 released and that seem like a
pretty reasonable support window. I wound up with HACK and
PORTABILITY comments all over the place and too much boilerplate. Now
I go with a startup check on sys.hexversion, and exit with similar
wording.

 

[Roy Smith]

I recently gave up on trying to support (backport to) pre-2.2 in my
projects. It's been ~3 years since 2.2 released and that seem like a
pretty reasonable support window.

It depends on what you're doing. If you're developing for in-house
use, you have control over what you run and can upgrade whenever you
feel you can justify devoting the resources to the upgrade effort. If
you're a one-man band, upgrading to a new version may be an hour's
work to download and install the latest and greatest. If you're a cog
in a large organization, it may be quite a bit more hassle.

If you're developing for outside customers, it's a whole different
story. Lots of people are running operating systems which are several
years old (heck, I'm typing this on a Windows-2000 box), and those
operating systems may have shipped with versions of langauges which
were several years old at the time they shipped.

Consider the following conversation with a prospective customer:

Prospect: "I really love your product, the price is OK, and I've got
budget approval for the purchase. Now, what do we need to run it?"

You: "All you need is a box running Python 2.2".

Prospect: "Oh, bummer, we run 2.1 [or 2.0, or 1.5.2] in-house on all
our machines".

You: "No problem, it's easy to upgrade to 2.2"

Prospect: "Unfortunately, not around here it isn't. I.T. owns the
servers, and they won't do the upgrade. Looks like we won't be able
to use your product after all. Sorry."

I'm not saying you shouldn't use new stuff, but don't fool yourself
about how long a lifetime old versions have in the field. And, the
old adage that "The customer is always right" is a good one.

 

[Steve Holden]

+1 QOTW.

I recently gave up on trying to support (backport to) pre-2.2 in my
projects. It's been ~3 years since 2.2 released and that seem like a
pretty reasonable support window. I wound up with HACK and
PORTABILITY comments all over the place and too much boilerplate. Now
I go with a startup check on sys.hexversion, and exit with similar
wording.

We could all take an example from Fredrik Lundh, who still regularly
releases products that run on everything back to Python 1.5.2. Kudos to
the effbot!

regards
Steve