any() and all() on empty list?

[Ron Adam]

Perhaps a practical example will illustrate why all() returns False
better than all this mathematical mumbo-jumbo.

Ok, say you're writing a simple software management/bug tracking
system. It manage another software package that is to have periodic
releases, but the release can only be made when there are no
outstanding bugs. You might have a line of code that looks like this:

if all(bug.status == 'closed' for bug in bugs_filed):
do_release()

As you can see, the release will only happen if all the bugs are marked
closed. But... what if no bugs have been filed? If all() were to
return False on an empty sequence, the software couldn't be fixed until
at least one bug had been filed and closed!

The point is, whenever you have to test that every item in a list is
true, it is almost always correct for the test to pass when the list is
empty. The behavior of all() is correct.


Carl Banks

Yes, But that should be a test for 'not any()'.

if not any(bug.status == 'open' for bug in bugs_filed):
do_release()


So to give a counter example...

Where we are assembling widgets in a manufacturing plant. Where we don't
want to go to the next step until *all* the sub parts are present.

if all(part.status == 'present' for part in unit):
do_release()

Oops! Some empty bins showed up at the next assembly station. ;-)


Cheers,
Ron

 

[Duncan Booth]

Where we are assembling widgets in a manufacturing plant. Where we don't
want to go to the next step until *all* the sub parts are present.

if all(part.status == 'present' for part in unit):
do_release()

Oops! Some empty bins showed up at the next assembly station. ;-)

I don't see the problem. You only get an empty bin if there is some part
assembled from no sub-parts, in which case you wanted an empty bin.

 

[Paul Rubin]

Think of it this way: if all(seq) is true, shouldn't it be the case
that you can point to a specific element in seq that is true?

No, all(seq) is true if you can't point to a specific element in seq
that's false.

 

[Steven D'Aprano]

That would break everything mentioned above. Think of it another way:
if all(seq) is false, shouldn't it be the case that you can point to
a specific element in seq that is false?

Think of it this way: if all(seq) is true, shouldn't it
be the case that you can point to a specific element in
seq that is true?

It may be that all([]) => True is useful more often
than all([]) => False would be, in the same way that it
is useful to define 0! = 1 and other mathematical
identities, but that doesn't imply that, strictly
speaking, there isn't some monkey-business going on there.

Now, I'm happy to admit that this is useful
monkey-business, but it can still lead to unexpected
results, as in my example in a previous post.

 

[Peter Otten]

Okay, Ron's example wasn't the best. How about this
one, from chess:

The intention is to play cautiously if all threatened
pieces are valuable, and daringly otherwise.

Isn't that example even worse? Compare:

- You have one of your valuable pieces threatened. You decide to play
cautiously.

- You have one valuable and one piece of lesser value threatened. You play
daringly.

What is that? The courage of despair?

if any(piece.value > 5 for piece in threatened):
# play cautiously
else:
# play daringly

looks more convincing to the non-chessplaying bystander (read: me) and --
surprise -- is supported by the new builtins just fine.

Peter

 

[Carl Banks]

No, all(seq) is true if every element in seq is true.
Surely that's a more intuitive definition than your
definition by what you can't do.

The question that needs to be answered is, what if
there are no elements at all?

Then every element in seq is true.

(And false.


Carl Banks

 

[Georg Brandl]

No, all(seq) is true if every element in seq is true.
Surely that's a more intuitive definition than your
definition by what you can't do.

The question that needs to be answered is, what if
there are no elements at all? That's an arbitrary
decision. Like the question "what is 0**0?" in
mathematics, some answers are more useful than others.
I can respect that practical answer -- but it isn't the
*only* answer.

(For those who don't see why 0**0 is problematic, 0**x
is equal to 0 for all x, and x**0 is equal to 1 for all
x, so what do you do for 0**0?)

Here's another way of looking at the problem:

all(flying elephants which are pink) => true
all(flying elephants which are not pink) => true

So, these flying elephants -- are they pink or not?

No, you ask two different sets whether they are true.

Georg

 

[Steven D'Aprano]

No, all(seq) is true if you can't point to a specific element in seq
that's false.

No, all(seq) is true if every element in seq is true.
Surely that's a more intuitive definition than your
definition by what you can't do.

The question that needs to be answered is, what if
there are no elements at all? That's an arbitrary
decision. Like the question "what is 0**0?" in
mathematics, some answers are more useful than others.
I can respect that practical answer -- but it isn't the
*only* answer.

(For those who don't see why 0**0 is problematic, 0**x
is equal to 0 for all x, and x**0 is equal to 1 for all
x, so what do you do for 0**0?)

Here's another way of looking at the problem:

all(flying elephants which are pink) => true
all(flying elephants which are not pink) => true

So, these flying elephants -- are they pink or not?

 

[Steven D'Aprano]

Ron Adam wrote:




I don't see the problem. You only get an empty bin if there is some part
assembled from no sub-parts, in which case you wanted an empty bin.

Okay, Ron's example wasn't the best. How about this
one, from chess:

The intention is to play cautiously if all threatened
pieces are valuable, and daringly otherwise. Here is
the obvious, but wrong, code:

if all(piece.value() > 5 for piece in \
threatened_pieces):
play_cautiously()
else:
play_daringly()

It is wrong because it leads to incorrect behaviour
when there are no threatened pieces at all -- the
intention is to play daringly by default, except for
the specific case of there being threatened pieces, all
of which are high value pieces.

So one correct, but more verbose, way to code this
would be:

valuable_danger = [piece.value() > 5 for piece in \
threatened_pieces]
if valuable_danger and all(valuable_danger):
play_cautiously()
else:
play_daringly()


Another obvious way of coding this would be to reverse
the sign of the test like so:

if not any(piece.value() <= 5 for piece in \
threatened_pieces):
play_cautiously()
else:
play_daringly()

In other words, play daringly unless no threatened
piece is low value. Unfortunately, while this is
obvious, it also gives the wrong behaviour when there
are no threatened pieces.



The lesson of this example is that while the standard
behaviour of any() and all(), as implemented in Python,
are often, usually, the Right Way to do things, they do
fail on some occasions, and coders should be aware of
cases where the assumptions break down.

 

[Ron Adam]

I don't see the problem. You only get an empty bin if there is some part
assembled from no sub-parts, in which case you wanted an empty bin.

Hmmm... It wasn't a well though out example. I was thinking in terms
of several processes working at the same time, and not communicating
with each other. Ie.. a bin getter, a part inserter, a part checker, a
bin mover. etc.

But even then if all the parts get marked as 'present' even though they
are aren't there, the bin could be released to the next station. And so
that example is void. I'll try and think of a better one.


Cheers,
Ron

 

[Ron Adam]

No, all(seq) is true if every element in seq is true. Surely that's a
more intuitive definition than your definition by what you can't do.

Yes, they are both valid view points. One is 'is all true' and the
other is 'has all true'.

You can also use either to express the other...

S and isalltrue(S) -> hasalltrue(S)

not S or hasalltrue(S) -> isalltrue(S)



A possibly useful thing to have:

hasall(S, value, test=None)

hasall(S, True) # Test for actual "True" values or 1.
hasall(S, True, bool) # test for true values, not zero or False.
hasall(S, 'ok')
hasall(S, True, lambda n: n=42)

Cheers,
Ron

 

[Ron Adam]

hasall(S, True, lambda n: n=42)

That was suppose to be:

hasall(S, True, lambda n: n==42)

 

[Marcin Ciura]

No, you ask two different sets whether they are true.

No, there is only one empty set.
Relevant to this discussion is stuff
about syllogisms concerning non-existent objects
and what C.S. Peirce did to them in the 19th century.
See e.g. http://www.math.fau.edu/schonbek/mfla/mfla1f01syl.html#tth_sEc5
Marcin

 

[Fredrik Lundh]

No, there is only one empty set.

who said anything about empty sets ?

</F>

 

[Antoon Pardon]

No, all(seq) is true if every element in seq is true.
Surely that's a more intuitive definition than your
definition by what you can't do.

The question that needs to be answered is, what if
there are no elements at all? That's an arbitrary
decision. Like the question "what is 0**0?" in
mathematics, some answers are more useful than others.
I can respect that practical answer -- but it isn't the
*only* answer.

(For those who don't see why 0**0 is problematic, 0**x
is equal to 0 for all x, and x**0 is equal to 1 for all
x, so what do you do for 0**0?)

Here's another way of looking at the problem:

all(flying elephants which are pink) => true
all(flying elephants which are not pink) => true

So, these flying elephants -- are they pink or not?

They are both.

 

[Paul McGuire]

They are both.

That would make them Schrödinger elephants!

-- Paul

 

[Paul Rubin]

No, all(seq) is true if every element in seq is true. Surely that's a
more intuitive definition than your definition by what you can't do.

They are different? I'd say every element in seq is true, unless
there's an element that's false. Do you want to pick a different word
than "all" and suggest renaming the function?

Here's another way of looking at the problem:

all(flying elephants which are pink) => true
all(flying elephants which are not pink) => true

So, these flying elephants -- are they pink or not?

By the definition, "all flying elephants are pink" and "all flying
elephants are non-pink" are both true statements, if that's what
you're asking. There is no contradiction. It's one of those
questions like "have you stopped beating your wife".

I'd say:

def boolify(s): return map(bool, s)

then:

all(S) = reduce(operator.and_, boolify(S), True)
any(S) = reduce(operator.or_, boolify(S), False)

You can see that changing True to False in the above definition of all
would make the result always false.

FWIW, I threw all my TV sets off the roof of my building this morning.
But nobody on the sidewalk needed to worry about getting hit by one ;-).

 

[Steven D'Aprano]

They are different?

Of course they are different -- they differ in the case of an empty
sequence.

I'd say every element in seq is true, unless
there's an element that's false. Do you want to pick a different word
than "all" and suggest renaming the function?

I've already pointed out that I'm satisfied with Tim Peters' explanation
for why the defined behaviour for any() is *most often* the Right Way for
it to be implemented in Python, *but* there are circumstances that this
behaviour is incorrect, therefore the programmer needs to actually
consider carefully what should happen for the empty sequence case. Was I
not clear enough?

By the definition, "all flying elephants are pink" and "all flying
elephants are non-pink" are both true statements, if that's what
you're asking. There is no contradiction.

Of course there is a contradiction. The contradiction is that flying
elephants are simultaneously pink and not pink.

If you don't understand why "Foo is Bar" and "Foo is not Bar" can't both
be true simultaneously, I suggest you spend some time googling on
"noncontradiction logic". To get you started, here's the Wikipedia entry:

http://en.wikipedia.org/wiki/Law_of_noncontradiction

It's one of those
questions like "have you stopped beating your wife".

Think about it: what is the logical value of the boolean "I have stopped
beating my wife" in the case of somebody who never started beating
their wife?

if husband.stopped_beating_wife(): # returns True or False
pay_fine()
else:
go_to_jail()

Pretty hard on the innocent husbands who never even beat their wife at all.

What we're doing is running up to the limitations of Aristotelian
two-value logic. We're trying to pigeon-hole answers into True/False that
really don't fit, so of course there will be the occasional case where
neither True nor False is correct. In hacker culture, the Chinese word
"mu" (literally "without") is sometimes used to mean "I cannot answer that
question because your assumptions are not correct".

In the case of all(seq), the correct answer is "mu". But since we're
stuck with binary logic, the more commonly useful behaviour is True, but
sometimes that leads to problems, such as in my first example of Guido
being punished because he was found guilty of all the terrorist crimes he
committed -- which is an empty list.

 

[Paul Rubin]

Of course they are different -- they differ in the case of an empty
sequence.

I don't think they differ in the case of an empty sequence. If the
sequence is empty, both statements are true.

Of course there is a contradiction. The contradiction is that flying
elephants are simultaneously pink and not pink.

Neither statement asserts the existence of any flying elephants
regardless of color, so neither statement contradicts the other
statement.

If you don't understand why "Foo is Bar" and "Foo is not Bar" can't both
be true simultaneously, I suggest you spend some time googling on
"noncontradiction logic". To get you started, here's the Wikipedia entry:

http://en.wikipedia.org/wiki/Law_of_noncontradiction

"All flying elephants are pink" is not a statement of the form "Foo is
Bar". See <http://en.wikipedia.org/wiki/For_all>, as I've cited
several times. "All flying elephants are pink" simply means "there
are no non-pink flying elephants". "All flying elephants are
non-pink" similarly means "there are no pink flying elephants". The
statements don't contradict, and in fact both statements are true.

if husband.stopped_beating_wife(): # returns True or False
pay_fine()
else:
go_to_jail()

Pretty hard on the innocent husbands who never even beat their wife at all.

Correct. The code should not be written that way.

In hacker culture, the Chinese word
"mu" (literally "without") is sometimes used to mean "I cannot answer that
question because your assumptions are not correct".

In the case of all(seq), the correct answer is "mu".

I don't think it's that bad. We just have to spell out precisely what
the assumptions are, and we've done so.

 

[Carl Banks]

Of course they are different -- they differ in the case of an empty
sequence.

No, they're not.

Look, there are significant differences between natural and computer
languages, and in this case something is happening in the natural
language that isn't happening in this computer language.

In English, if I were to ask you a question like "Have you put all your
books in the truck?" when you have no books, a valid and reasonable
answer is "I don't have any books." I.e., the answer is neither yes
nor no. In fact, yes and no aren't valid answers at all (unless you're
being snarky**), because, in English, the word "all" carries an
assumption of existence. (Or maybe it doesn't for you guys in
Australia; it does in the USA.)

In Python, yes and no are the only possible answers. Probably the only
analogous thing you could do in Python would be for all() to raise
ValueError when passed an empty sequence.


Carl Banks

** - and note that, if you are being snarky, you would say "yes".