cascading python executions only if return code is 0

[Steven D'Aprano]

I'd think the biggest problem with this is that if one runs Python with
optimization turned on, the assert statement itself vanishes, leaving one
with an empty else clause...

Try debugging that problem!

Actually, that's not correct. If the assert statement vanishes, so does the
else clause, since there's nothing in it.


[steve@ando ~]$ python2.7 -c "from dis import dis

code = compile('''\
for x in [1, 2, 3]:
pass
else:
assert x
''', '', 'exec')
dis(code)"""

1 0 SETUP_LOOP 35 (to 38)
3 LOAD_CONST 0 (1)
6 LOAD_CONST 1 (2)
9 LOAD_CONST 2 (3)
12 BUILD_LIST 3
15 GET_ITER19 STORE_NAME 0 (x)

2 22 JUMP_ABSOLUTE 16
4 26 LOAD_NAME 0 (x)
29 POP_JUMP_IF_TRUE 38
32 LOAD_GLOBAL 1 (AssertionError)
35 RAISE_VARARGS 141 RETURN_VALUE


Compare to the case with optimizations on:


[steve@ando ~]$ python2.7 -O -c "from dis import dis
code = compile('''\
for x in [1, 2, 3]:
pass
else:
assert x
''', '', 'exec')
dis(code)"""
1 0 SETUP_LOOP 23 (to 26)
3 LOAD_CONST 0 (1)
6 LOAD_CONST 1 (2)
9 LOAD_CONST 2 (3)
12 BUILD_LIST 3
15 GET_ITER19 STORE_NAME 0 (x)

2 22 JUMP_ABSOLUTE 16
4 >> 26 LOAD_CONST 3 (None)
29 RETURN_VALUE

 

[Peter Otten]

I am not a zealot (I'm not! Really!! . I just find it alarming to
have major pieces of software rely on a feature that can be so easily
tuned out, and it wasn't clear from your comment that it was /any/
exception.

Mostly I don't want newbies thinking "Hey! I can use assertions for all
my confidence testing!"

Just as one data point OpenERP, which has a lot of good features,
unfortunately uses assert to test user input.

Put the following at the beginning of every affected module:

try:
assert 0
except AssertionError:
pass
else:
raise AssertionError("won't run with assertions turned off")

 

[Roy Smith]

Roy Smith wrote:


Until such time as somebody decides they can speed up your code by 5% by
running with optimizations turned on.

Well, there's lots of changes people could make that would break things. Many of them are in the name of efficiency. [1] But, let's say they did that. We would fall off the end of the function, return None, and the caller would end up doing:

with None:
whatever

leaving somebody to puzzle over why the logs contained a stack dump ending in:

AttributeError: __exit__

So, here's the deeper question. Is your issue strictly that -O elides assert statements? [2] That's a purely mechanical issue that could be solved by using the rather more verbose:

if not condition:
raise AssertionError("....")

Would you feel differently then?

So there's always tension between "why am I testing something that can't
fail?" and "but what if it does?"

Trust, but verify. Especially when the thing you're verifying is your understanding of how your own code works

[1] and most of those are premature optimizations. To a first order approximation, for us, application speed is all about database performance, and not at all about Python code execution speed. That's a pretty good second order approximation as well.

[2] In which case, we would just add some middleware which did:

assert "-O" not in sys.argv

 

[Chris Angelico]

[2] In which case, we would just add some middleware which did:

assert "-O" not in sys.argv

Aside from the fact that this wouldn't work, it won't work By the
time you see argv, the -O option has been eaten. But why stop at that?

def assertions_working():
try:
assert False
except AssertionError:
return True
return False

assert assertions_working() # refuse to run in -O mode

Can't imagine why that wouldn't work......

ChrisA

 

[Nick Cash]

assert assertions_working() # refuse to run in -O mode

Can't imagine why that wouldn't work......

Why overthink this?

assert not sys.flags.optimize

is clearly the one, and only one, obvious way to do it.

Of course, it works about as well as the rest of these solutions. Which is to say, not at all.

-Nick Cash

 

[Steven D'Aprano]

So, here's the deeper question.  Is your issue strictly that -O elides
assert statements? That's a purely mechanical issue that could be
solved by using the rather more verbose:

if not condition:
raise AssertionError("....")

Would you feel differently then?

Not quite. As I said in my initial rant, there's also the issue of using an
appropriate exception. Naming is important for comprehending what the code
does. Just as it's bad to do:

my_list = 42
my_str = [my_list * 2]


so it's bad to have badly chosen exceptions:

def handle_string(s):
if not isinstance(s, str):
raise ImportError("not a string")
...


Substitute AssertionError for ImportError:

def handle_string(s):
if not isinstance(s, str):
raise AssertionError("not a string")
...


and it's still wrong because the exception is misleading and/or confusing to
whoever has to deal with the code.

I've been known to use asserts inside private functions and explicit tests
inside public ones:

def func(x):
if x <= 0:
raise ValueError("x must be positive")
return _func(x + 1)

def _func(x):
assert x > 1
return sqrt(1/(x-1))


If I think that the test is checking an internal invariant, assert is okay,
if not, it isn't. In this case, the fact that the actual calculation of
func(x) is factored out into a private function _func is purely an internal
implementation issue. I might refactor the code and move _func inline:

def func(x):
if x <= 0:
raise ValueError("x must be positive")
x += 1
assert x > 1
return sqrt(1/(x-1))


in which case the assertion is obviously checking an internal invariant and
therefore acceptable. So it's still checking an internal invariant even
when the code is moved into a purely internal function.

The other rule I use is that if an assert might ever be triggered in the
normal run of the program, it shouldn't be an assert. In principle, we
should run Python code with -O in production.

(It would have been better if Python optimized code by default, and had a -D
switch to turn debugging on, including asserts. I believe that's what
languages like Eiffel do.)

 

[Roy Smith]

So, here's the deeper question.  Is your issue strictly that -O elides
assert statements? That's a purely mechanical issue that could be
solved by using the rather more verbose:

if not condition:
raise AssertionError("....")

Would you feel differently then?

Not quite. As I said in my initial rant, there's also the issue of using an
appropriate exception.
[...]
If I think that the test is checking an internal invariant, assert is okay,
if not, it isn't.

Well, in this case, that's what it's doing, so I guess I'm good

 

[Roy Smith]

Mostly I don't want newbies thinking "Hey! I can use assertions for all my
confidence testing!"

How about this one, that I wrote yesterday;

assert second >= self.current_second, "time went backwards"

I think that's pretty high up on the "can never happen" list.

 

[Chris Angelico]

How about this one, that I wrote yesterday;

assert second >= self.current_second, "time went backwards"

I think that's pretty high up on the "can never happen" list.

assert second >= self.current_second, "user changed the clock"

ChrisA

 

[Dennis Lee Bieber]

assert second >= self.current_second, "user changed the clock"

Why blame the user if an NTP synch made the adjustment? <G>

 

[Chris Angelico]

Why blame the user if an NTP synch made the adjustment? <G>

True, it could have been NTP, or maybe the original assert was right
and we're on the other side of an event horizon. But it's much more
fun to blame the user.

ChrisA

 

[Tim Chase]

assert second >= self.current_second, "user changed the clock"

There appears to be a Whovian at the wheel...
'doctor4'


Or perhaps a Star Trek fan:
4


It's-clearly-past-my-bedtime-or-is-it-before'ly yers,

-tkc

 

[Roy Smith]

True, it could have been NTP, or maybe the original assert was right
and we're on the other side of an event horizon. But it's much more
fun to blame the user.

ChrisA

NTP is never supposed to move the clock backwards. If your system clock
is fast, it's supposed to reduce the rate your clock runs until it's
back in sync. Well, maybe it only does that for small corrections?

 

[Steven D'Aprano]

How about this one, that I wrote yesterday;

assert second >= self.current_second, "time went backwards"

I think that's pretty high up on the "can never happen" list.

Time goes backwards by one hour[1] at least once a year across most of the
world.

http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time-wisdom



[1] Unless it's less than an hour. Or more than one hour.

 

[Chris Angelico]

NTP is never supposed to move the clock backwards. If your system clock
is fast, it's supposed to reduce the rate your clock runs until it's
back in sync. Well, maybe it only does that for small corrections?

The exact rules are tweakable, but yes, it slews for small corrections
and steps for larger ones.

ChrisA

 

[Chris Angelico]

Time goes backwards by one hour[1] at least once a year across most of the
world.

http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-about-time-wisdom

[1] Unless it's less than an hour. Or more than one hour.

Kinda. That should be accompanied by a change in UTC offset,
indicating that time hasn't gone backwards, just the clock. Anything
that queries the clock in UTC shouldn't see it go backward. But if
there's a time server that's misconfigured, and it actually jumps time
backward, then yes, you could see NTP change the time back by an hour
once a year. It's wrong, but it certainly can happen.

ChrisA

 

[Roy Smith]

How about this one, that I wrote yesterday;

assert second >= self.current_second, "time went backwards"

I think that's pretty high up on the "can never happen" list.

Time goes backwards by one hour[1] at least once a year across most of the
world.

Only if you're stupid enough to run your systems in local time.

http://infiniteundo.com/post/25509354022/more-falsehoods-programmers-believe-a
bout-time-wisdom

Most of those deal with things like daylight saving time and time zones
and calendars, which are all far more complicated (and chaotic) than
just keeping track of UTC timestamps.

There's lists like that for names and genders too. As in, what do you
mean there are people who don't have exactly two names and one gender?

 

[Chris Angelico]

There's lists like that for names and genders too. As in, what do you
mean there are people who don't have exactly two names and one gender?

And addresses, too. The only way to reliably query the user for an
address is with a single field in which s/he may type anything at
all... and that freedom includes the possibility of making a right
hash of it and having delivery fail.

ChrisA

 

[Roy Smith]

And addresses, too. The only way to reliably query the user for an
address is with a single field in which s/he may type anything at
all... and that freedom includes the possibility of making a right
hash of it and having delivery fail.

ChrisA

You're assuming people *have* addresses.

A bunch of years ago (I'm probably dating myself), there was a legal
ruling in New York that having an address was not a requirement to
register to vote. The case in question had to do with a homeless person
who was refused registration because he didn't have an address.

 

[Dennis Lee Bieber]

NTP is never supposed to move the clock backwards. If your system clock
is fast, it's supposed to reduce the rate your clock runs until it's
back in sync. Well, maybe it only does that for small corrections?

Especially likely when one considers that M$ Windows only does a time
synch once a week.