PyWart: More surpises via "implict conversion to boolean" (and othersteaming piles!)

[Rick Johnson]

## START CODE ###########################################
def foo():
# foo represents a patternless function
# or method that returns a Boolean value
# based on some internal test.
#
if 1==1:
return True
return False
#
# The fun begins when two tiny chars are forgotten,
# however, since the code is legal, python will happily
# give us the wrong answer.
#
if foo: # <- forgot parenthesis!
print 'implicit conversion to bool bites!'
else:
#
# This block will NEVER execute because foo is
# ALWAYS True!
#
#
# Some introspection to understand why this happened.
#
print 'foo =', foo
print 'bool(foo) ->', bool(foo)
#
## END CODE #############################################

It's obvious i did not follow the syntactical rules of
Python, i understand that, however, there are three design
flaws here that are contributing to this dilemma:

1. Implicit conversion to Boolean is evil

2. Conditionals should never accept parameter-less
functions. If you want to check if a callable is
True or False, then use "if bool(callable)". Any
usage of a bare callable in conditionals should
raise SyntaxError.

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!
Introspection should ALWAYS be explicit!

For a long time i thought Python's idea of a returning the
value of a callable-- who's name is unadorned with "(...)"
--was a good idea, however, i am now wholly convinced that
this design is folly, and the reason is two fold:

1. Parenthesis should not be required for parameter-
less functions. I realize this is a bit more
complicated in languages like Python where
attributes are exposed to the public, but still, not
reason enough to require such onerous typing.

2. Implicit introspection is evil. I would prefer an
explicit method attached to all callables over a
sugar for "callable.call". We should never consume
syntactical sugars UNLESS they can greatly reduce
the density of code (like math operators for
instance!)

 

[Mark Lawrence]

## START CODE ###########################################
def foo():
# foo represents a patternless function
# or method that returns a Boolean value
# based on some internal test.
#
if 1==1:
return True
return False
#
# The fun begins when two tiny chars are forgotten,
# however, since the code is legal, python will happily
# give us the wrong answer.
#
if foo: # <- forgot parenthesis!
print 'implicit conversion to bool bites!'
else:
#
# This block will NEVER execute because foo is
# ALWAYS True!
#
#
# Some introspection to understand why this happened.
#
print 'foo =', foo
print 'bool(foo) ->', bool(foo)
#
## END CODE #############################################

It's obvious i did not follow the syntactical rules of
Python, i understand that, however, there are three design
flaws here that are contributing to this dilemma:

1. Implicit conversion to Boolean is evil

2. Conditionals should never accept parameter-less
functions. If you want to check if a callable is
True or False, then use "if bool(callable)". Any
usage of a bare callable in conditionals should
raise SyntaxError.

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!
Introspection should ALWAYS be explicit!

For a long time i thought Python's idea of a returning the
value of a callable-- who's name is unadorned with "(...)"
--was a good idea, however, i am now wholly convinced that
this design is folly, and the reason is two fold:

1. Parenthesis should not be required for parameter-
less functions. I realize this is a bit more
complicated in languages like Python where
attributes are exposed to the public, but still, not
reason enough to require such onerous typing.

2. Implicit introspection is evil. I would prefer an
explicit method attached to all callables over a
sugar for "callable.call". We should never consume
syntactical sugars UNLESS they can greatly reduce
the density of code (like math operators for
instance!)

This particular PyWart would immediately be caught if another PyWart,
namely the unittest module, were to be used to catch this programming error.

All of your preferences can be met by raising an issue on the bug
tracker and providing a patch that changes code, docs and test suites as
appropriate. Simples

 

[Ned Batchelder]

## START CODE ###########################################
def foo():
# foo represents a patternless function
# or method that returns a Boolean value
# based on some internal test.
#
if 1==1:
return True
return False
#
# The fun begins when two tiny chars are forgotten,
# however, since the code is legal, python will happily
# give us the wrong answer.
#
if foo: # <- forgot parenthesis!
print 'implicit conversion to bool bites!'
else:
#
# This block will NEVER execute because foo is
# ALWAYS True!
#
#
# Some introspection to understand why this happened.
#
print 'foo =', foo
print 'bool(foo) ->', bool(foo)
#
## END CODE #############################################

It's obvious i did not follow the syntactical rules of
Python, i understand that, however, there are three design
flaws here that are contributing to this dilemma:

1. Implicit conversion to Boolean is evil

2. Conditionals should never accept parameter-less
functions. If you want to check if a callable is
True or False, then use "if bool(callable)". Any
usage of a bare callable in conditionals should
raise SyntaxError.

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!
Introspection should ALWAYS be explicit!

For a long time i thought Python's idea of a returning the
value of a callable-- who's name is unadorned with "(...)"
--was a good idea, however, i am now wholly convinced that
this design is folly, and the reason is two fold:

1. Parenthesis should not be required for parameter-
less functions. I realize this is a bit more
complicated in languages like Python where
attributes are exposed to the public, but still, not
reason enough to require such onerous typing.

2. Implicit introspection is evil. I would prefer an
explicit method attached to all callables over a
sugar for "callable.call". We should never consume
syntactical sugars UNLESS they can greatly reduce
the density of code (like math operators for
instance!)

It seems like you are only looking at how to improve the error you just
stumbled over, and not how the full proposal would work out, or even if
you would like it better.

You haven't made the entire idea explicit yet. How would I pass the
function foo to another function? The word "foo" now means, invoke foo.
You mean an explicit method attached to callables, like
"foo.as_callable" ? But why doesn't the word "foo" there invoke foo?
Surely the word "as_callable" isn't special, so is it that
"foo.anything" means direct attribute access on foo? So to perform
attribute access on the result of foo I need "foo().something" ? In
what cases does the word foo invoke the function, and when doesn't it?

It's not possible to make a programming language error-proof. There
will always be mistakes programmers can make.

 

[Rotwang]

[...]

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!

So, for example, none of

isinstance(x, myclass)

map(myfunc, range(10))

x = property(x_get, x_set)

would still work?

 

[Ned Batchelder]

[...]

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!

So, for example, none of

isinstance(x, myclass)

map(myfunc, range(10))

x = property(x_get, x_set)

would still work?

I guess neither would:

except ValueError:

 

[Chris Angelico]

if foo: # <- forgot parenthesis!
print 'implicit conversion to bool bites!'

You also forgot the parentheses on the second line, and that's nothing
to do with boolification

ChrisA

 

[Terry Reedy]

[...]

3. Implicit introspection is evil, i prefer all
references to a callable's names to result in a CALL
to that callable, not an introspection!

So, for example, none of
isinstance(x, myclass)
map(myfunc, range(10))
x = property(x_get, x_set)
would still work?

No. That is what makes this a troll post, for fun, rather than a serious
proposal.

 

[Steven D'Aprano]

## START CODE ###########################################
def foo():
# foo represents a patternless function

Patternless? I have never heard that term before in this context. Do you
mean a parameter-less or argument-less function?

# or method that returns a Boolean value
# based on some internal test.
#
if 1==1:
return True
return False

This always returns True, since 1 always equals 1.

# The fun begins when two tiny chars are forgotten,
# however, since the code is legal, python will happily
# give us the wrong answer.
#
if foo: # <- forgot parenthesis!
print 'implicit conversion to bool bites!'

No it doesn't. It rocks. You have found one tiny little disadvantage,
about the size of a mote of dust floating: if you write functions that
take no arguments (already a code-smell) and forget the brackets
(mistakes will happen...) and have no tests to ensure that both branches
of the `if` are tested (what, are you careless and negligent?), then you
might be bitten by a bug of your own creation.

Compared to that mote, the usefulness and convenience of duck-typing bools
is about the size of Mt Everest.

else:
#
# This block will NEVER execute because foo is
# ALWAYS True!

Correct. And since foo() is also always True, there is no difference.



[...]

It's obvious i did not follow the syntactical rules of Python, i
understand that,

No you don't understand that. You *did* follow the syntactical rules of
Python. `if foo` is perfectly correct syntax, if it were not, you would
have got a SyntaxError exception at compile-time.

You need to understand the difference between syntax and semantics. This
is invalid English syntax:

"Cat mat on sat the."

This is valid syntax, but semantically wrong:

"The mat sat on the cat."

This is both syntactically and semantically correct:

"The cat sat on the mat."

however, there are three design flaws here that are
contributing to this dilemma:

1. Implicit conversion to Boolean is evil

It's not implicit conversion. You're not understanding what is going on.
foo is not converted to a bool, foo remains a function object. Rather,
it's duck-typing truthiness, which is no different from any other duck-
typing. If it swims like a bool and quacks like a bool, it might as well
be a bool.

Duck-typing is a design feature, not a flaw.

2. Conditionals should never accept parameter-less functions.

How is the conditional supposed to know that its term is a function?
What's so special about parameter-less functions? You can forget to call
functions with any number of parameters, especially if they take default
values.

Let's suppose that you get your way. Which of the following if-tests
should be prohibited, and when should they be prohibited? At compile time?

if random.random() > 0.5:
spam = lambda x=23: x
eggs = 42
else:
spam = 42
eggs = lambda x=23: x

if spam:
print "spam is a truthy value"
if eggs:
print "eggs is a truthy value"

If you
want to check if a callable is True or False, then use "if
bool(callable)".

Ewww. That's horrible. bool() should only be used to get a canonical bool
object, e.g. for writing to a database or something. It should never be
used just because you are scared of Python's power.

Any usage of a bare callable in conditionals should
raise SyntaxError.

Ah, so it should happen at compile-time, not run-time? I'm afraid you're
not as clear about how Python operates as you perhaps think you are.

3. Implicit introspection is evil, i prefer all references to a
callable's names to result in a CALL to that callable, not an
introspection! Introspection should ALWAYS be explicit!

Again, this is a complete misunderstanding of Python's execution model.
It is true that callables have names, but they are only used for
displaying error messages in tracebacks. Otherwise, callables are no
different from any other object: they can have zero, one or many names
bound to them, and referring to the name gives you access to the object
itself.

spam = <any object at all, whether a function or not>
spam # this is a reference to the object

Introspection has nothing to do with this.

For a long time i thought Python's idea of a returning the value of a
callable-- who's name is unadorned with "(...)" --was a good idea,
however, i am now wholly convinced that this design is folly,

Oh well, not everyone has the sense to recognise good design. You should
do more Python programming, and less ranting, you might learn something.

and the
reason is two fold:

1. Parenthesis should not be required for parameter- less functions.

Of course they should. Firstly, parameter-less functions are a code-
smell, and ought to be discouraged. Secondly, even if you have a good
reason for using one -- for example, random.random -- then the difference
between referring to the object and calling the object should be clear.

With Python's correct design, we have:

spam # always, without exception, refers to the object
spam() # always, without exception, calls the object

With your suggested design, we would have:

spam # sometimes refers to the object, sometimes calls the object
spam() # always calls the object

Ruby makes this mistake, and is a lessor language for it.

I realize this is a bit more complicated in languages like Python
where attributes are exposed to the public, but still, not reason
enough to require such onerous typing.

What does the presence or absence of attributes have to do with this?

2. Implicit introspection is evil. I would prefer an explicit method
attached to all callables over a sugar for "callable.call". We
should never consume syntactical sugars UNLESS they can greatly
reduce the density of code (like math operators for instance!)

I do not understand what you are trying to say here.

 

[Tim Chase]

You need to understand the difference between syntax and semantics.
This is invalid English syntax:

"Cat mat on sat the."

This is valid syntax, but semantically wrong:

"The mat sat on the cat."

This is both syntactically and semantically correct:

"The cat sat on the mat."

And there are times you *do* want to do unconventional things with
the language, and Python allows that:

http://www.catster.com/files/600px-cat-hiding-under-rug.jpg

because in that particular use case, it *is* semantically correct.

With Python's correct design, we have:

spam # always, without exception, refers to the object
spam() # always, without exception, calls the object

With your suggested design, we would have:

spam # sometimes refers to the object, sometimes calls the object
spam() # always calls the object

Ruby makes this mistake, and is a lessor language for it.

One of the (many) reasons Ruby drives me nuts.

-tkc

 

[Travis Griggs]

Of course they should. Firstly, parameter-less functions are a code-
smell, and ought to be discouraged. Secondly, even if you have a good
reason for using one -- for example, random.random -- then the difference
between referring to the object and calling the object should be clear.

Interesting. Can you clarify or provide some links to the "parameter-less functions are a code-smell” bit?

I agree with your points about consistency. I disagree with the original poster that niladic functions should have a different syntax than the others. I empathize with him, I’ve made the same mistake before (being an ardent Smalltalker in the past, it’s an easy habit to have bite you). But the consistency is more important. And in python, things “happen” when parentheses appear. I just accept that.

OTOH, I’m not sure I’ve heard the parameters-less functions are a code one? Is it just loose functions that you’re referring to? As opposed to methods (which are just bound functions)? I could maybe accept that. But methods with fewer arguments, and even none, are a desirable thing. There are code smells that are the opposite in fact, methods with long parameter lists are generally seen as code smell (“passing a paragraph”).

Anyway, I’d love to understand better what you see as the code smell and why.

 

[Chris Angelico]

OTOH, I’m not sure I’ve heard the parameters-less functions are a code one? Is it just loose functions that you’re referringto? As opposed to methods (which are just bound functions)? I could maybe accept that. But methods with fewer arguments, and even none, are a desirable thing. There are code smells that are the opposite in fact, methods withlong parameter lists are generally seen as code smell (“passing a paragraphâ€).

'self' is, imo, a parameter. When you call a parameter-less method on
an object, it's usually an imperative with a direct object (or
sometimes a subject):

some_file.close() # "Close some_file"
some_list.shuffle() # "Shuffle some_list"
some_file.readline() # "Some_file, read in a line"

There are times when, for convenience, the object is implicit.

print("some text", file=some_file) # Print that text
print(file=some_file) # Print a blank line
print("some text") # Print that text to sys.stdout
print() # Print a blank line to sys.stdout

So in that situation, the no-args call does make sense. Of course,
this is a call to a function that does take args, but it's accepting
all the defaults and providing no additional content. It's quite
different to actually define a function that mandates exactly zero
arguments, and isn't making use of some form of implicit state (eg a
closure, or maybe a module-level function that manipulates
module-level state - random.random() would be an example of the
latter). Syntactically, Python can't tell the difference between
"print()" and "foo()" where foo can never take args.

I'd say that a function taking no args is code smell, unless it's
obviously taking its state from somewhere else (callbacks, for
instance - maybe you pass a bound method, or maybe a closure, but in
either case it has implicit state that's not described by function
args); but _calling_ with no args isn't as smelly. It's certainly less
common than using args, but there are plenty of times when a type is
called without args, for instance[1].

ChrisA

[1] Okay, that was a really abysmal pun.

 

[Jussi Piitulainen]

in fact, methods with long parameter lists are generally seen as

"If you have a predicate with ten arguments, you probably forgot some"
(heard long time ago over in the Prolog world).

 

[Chris Angelico]

"If you have a predicate with ten arguments, you probably forgot some"
(heard long time ago over in the Prolog world).

Conversely:

"Thirteen pushes for each call! Whew. But now we have our files open."

(from an assembly language programming tutorial, on the DosOpen API in OS/2)

ChrisA

 

[Travis Griggs]

'self' is, imo, a parameter. When you call a parameter-less method on
an object, it's usually an imperative with a direct object (or
sometimes a subject):

some_file.close() # "Close some_file"
some_list.shuffle() # "Shuffle some_list"
some_file.readline() # "Some_file, read in a line"

There are times when, for convenience, the object is implicit.

print("some text", file=some_file) # Print that text
print(file=some_file) # Print a blank line
print("some text") # Print that text to sys.stdout
print() # Print a blank line to sys.stdout

So in that situation, the no-args call does make sense. Of course,
this is a call to a function that does take args, but it's accepting
all the defaults and providing no additional content. It's quite
different to actually define a function that mandates exactly zero
arguments, and isn't making use of some form of implicit state (eg a
closure, or maybe a module-level function that manipulates
module-level state - random.random() would be an example of the
latter). Syntactically, Python can't tell the difference between
"print()" and "foo()" where foo can never take args.

So at this point, what I’m reading is that actually making a “no arg function” is difficult, if we widen the semantics. The “arguments” of a function may be bound to its implicit self parameter, or tied to module state.

I'd say that a function taking no args is code smell, unless it's
obviously taking its state from somewhere else (callbacks, for
instance - maybe you pass a bound method, or maybe a closure, but in
either case it has implicit state that's not described by function
args); but _calling_ with no args isn't as smelly. It's certainly less
common than using args, but there are plenty of times when a type is
called without args, for instance[1].

Which leaves me wondering, how would I get my code to smell this way then? What IS an example of a no arg function that doesn’t have an implicit object, that smells? It seems I can escape the smell clause, as long as I find some data that I reason is attached to my function.

This all aside, I don’t think these are what the OP had in mind. A code inspection algorithm is not going to be able to discern when an explicitly parameterless function has implicit parameters. It’s just going to see something like

print vs print()

or

aPoint.transpose vs aPoint.transpose()

 

[Terry Reedy]

So at this point, what I’m reading is that actually making a “no arg function†is difficult, if we widen the semantics.

It is quite easy.

def f(): return 3 # or any other constant.

Chris said that useful functions in Python are (mostly) not really
niladic, which is equivalent to saying that niladic functions in Python
are (mostly) not really useful. They are a mainly a device in pure
function theory to have constants (True, False, 0, 1, ...) while also
having everything be a function (the 'pure' part). Pure set theory uses
its own tricks to make the same constants (and functions) be sets ;-).

 

[Steven D'Aprano]

Interesting. Can you clarify or provide some links to the
"parameter-less functions are a code-smell†bit?

Functions map a value to another value. They can be one-to-one, or many-
to-one. (Mathematically, they cannot be one-to-many or many-to-many,
that's called a relation.) What about zero-to-one?

If the function always returns the same result, e.g.:

def spam():
return "spam spam spam"


why are you using a function? Just create a constant and use that.
Calling a function which always returns the same value is a code smell.
That's not to say it is always wrong, but it smells a bit off.

How about zero-to-many functions? E.g. you have a situation where calling
function() twice might return different values. Okay, here's an example:

give_me_an_even_number()
=> returns 42
give_me_an_even_number()
=> returns 23

Hmmm. There's a bug in give_me_an_even_number(). How do I reproduce that
bug? What arguments do I pass? Oh, the same no-arguments as for the
working call.

Clearly, the function must have *hidden state*. Hidden state (e.g. a
global variable) makes it hard to reason about the function call, since
you don't know what the hidden state is. So that's also a bit smelly.

Hidden state is generally bad, because it makes it hard to reason about
the function call, hard to reproduce results, hard to debug, hard to
test. Think about the difference in difficulty in confirming that
math.sin() of some value x returns the value 0.5, and confirming that
random.random() of some hidden state returns a specific value:

py> assert math.sin(0.5235987755982989) == 0.5

versus:

py> state = random.getstate()
py> random.seed(12345)
py> assert random.random() == 0.41661987254534116
py> random.setstate(state)




[...]

OTOH, I’m not sure I’ve heard the parameters-less functions are a code
one? Is it just loose functions that you’re referring to? As opposed to
methods (which are just bound functions)? I could maybe accept that. But
methods with fewer arguments, and even none, are a desirable thing.

Methods that appear to take zero arguments actually take one argument, it
is just that it is written in a different place:

"some string".upper()

is merely different syntax for:

upper("some string")

with the bonus that str.upper and MyClass.upper live in different
namespaces and so can do different things. So I have no problem with zero-
argument methods, or functions with default values.

Remember that a code smell does not mean the code is bad. Only that it
needs to be looked at a bit more carefully. Perhaps it is bad. Or
perhaps, like durian fruit, it smells pretty awful but tastes really good.

There are code smells that are the opposite in fact, methods with long
parameter lists are generally seen as code smell (“passing a
paragraphâ€).

Absolutely! You'll get no disagreement from me there.

 

[Chris Angelico]

Think about the difference in difficulty in confirming that
math.sin() of some value x returns the value 0.5, and confirming that
random.random() of some hidden state returns a specific value:

py> assert math.sin(0.5235987755982989) == 0.5

versus:

py> state = random.getstate()
py> random.seed(12345)
py> assert random.random() == 0.41661987254534116
py> random.setstate(state)

Really, the assertion just requires the setting of the seed and the
call to random.random(); the other two are to ensure that you don't
fiddle with anything else that's using random.random(). And since
random.random() is actually just a bound method of some module-level
object, you can actually just create the exact same thing with
explicit rather than implicit state:
0.41661987254534116

Which doesn't tamper with the default object's state.

Whether it's a module-level function, a bound method, a closure, or a
callable object, a zero-arg function in Python always has some kind of
implicit state. The question is, what is it doing with it? In the case
of random number generation, maintained state is critical (and making
it implicit is usually sufficient); similar with functions like
input(), where the return value doesn't really depend on the argument
at all [1], and of course anything that iterates over an object is
going to need to change some kind of state (either a pointer, or the
actual data, depending on whether you're looking at eg
iter([1,2,3]).next or [1,2,3].pop). Do you know of any functions in
Python that don't use any implicit state and don't take arguments? I
can't think of any, but of course that proves nothing.

ChrisA

[1] input("Enter your name: ") vs input("What is one plus one? ") will
probably return different values, but that's playing with humans
rather than depending on the value of the argument...

 

[Tim Chase]

give_me_an_even_number()
=> returns 42
give_me_an_even_number()
=> returns 23

Hmmm. There's a bug in give_me_an_even_number(). How do I reproduce
that bug? What arguments do I pass? Oh, the same no-arguments as
for the working call.

Clearly, the function must have *hidden state*. Hidden state (e.g.
a global variable) makes it hard to reason about the function call,
since you don't know what the hidden state is. So that's also a bit
smelly.

I'd even go so far as to claim that this is the primary reason a
zero-argument function is a code-smell. Not because zero-argument
functions smell, but because hidden-state smells and zero-argument
functions imply hidden-state. Date/time functions are a personal pet
peeve for just this reason, and require addressing the hidden-state
of the system clock regardless of parameter-count. Thus instead of
something like

class Person:
def __init__(self, name, dob):
self.name = name
self.dob = dob
def age(self):
return datetime.date.today() - self.dob

I do

def age(self, as_of=None):
if as_of is None:
as_of = datetime.date.today()
return as_of = self.dob

allowing me to test the function with known dates.

Absolutely! You'll get no disagreement from me there.

*coughtkintercough*

-tkc

 

[Tim Chase]

def age(self, as_of=None):
if as_of is None:
as_of = datetime.date.today()
return as_of = self.dob

and of course I mean

return as_of - self.dob

which is what I get for typing in the dark and the "-" and "="
keys are adjacent. :-/

-tkc

 

[Roy Smith]

Whether it's a module-level function, a bound method, a closure, or a
callable object, a zero-arg function in Python always has some kind of
implicit state.

Sometimes, it has a *lot* of implicit state:

os.fork()