A little stricter type checking

[Tongu? Yumruk]

I have a little proposal about type checking in python. I'll be glad
if you read and comment on it. Sorry for my bad english (I'm not a
native English speaker)

A Little Stricter Typing in Python - A Proposal

As we all know, one of the best things about python and other
scripting languages is dynamic typing (yes I know it has advantages
and disadvantages but I will not discuss them now). Dynamic typing
allows us to change types of variables on the fly, and frees us from
the redundant work of defining variables, and their types. On the
other hand in languages like C, java etc... where types are strict ve
have the guarantee that the variables will always be the same type
without any change.

For example think that you are writing a utility function for a
library. In python you just define the function name and the name of
the parameters but you can never know the variables that has been
passed from parameters will always function as you planned. If you
want to be sure, you need to explicitly check if the variable has the
correct type. On the other side for example in C you don't need to
check the types of variables since the compiler will complain if there
are incompatible types. But this approach also limits the flexibility
and destroys the advantages of dynamic typing. What we need is a way
to define types of the parameters in a flexible way. So Here it is...

We can use operators like those we use in boolean operations. For
exampla lets assume that we have a function that take one parameter
which needs to be an instance of class foo, or an instance of a
subclass of foo. In current style it will look like that:
def larry(guido):
<some checks to be sure that guido is an instance of foo or an
instance of a subclass of foo>
<Actual code>

Well always checking the types of the parameters in the start of every
function is too many redundant work. Instead we can define a syntax
that will check the type information at the beginning of function
declaration, and even before executing any code from that function.
That syntax might look like that:

def larry(guido >= foo):
<Just actual code, no checking since the interpreter handles it>

And if we have a function that have much more parameters, this kind of
a syntax will really help us to remove redundant code from our
functions. The syntax can be defined by something like:
Symbol : Definition
== : The parameter is an instance of a given class for example: d ==
dict

= : The parameter is an instance of the given class or one of its

subclasses
<= : The parameter is an instance of the given class or one of its
parent
classes

This list can be expanded to include != or even a more fantastic
operator: "in" so that we can check if the parameter has a specific
attribute before executing our function.

The best thing about this approach is that while keeping flexibility
it adds better type checking and it keeps the compatibility with old
code.

So that's all for now... All comments are welcome.

 

[Andrew Dalke]

> If you want to be sure, you need to explicitly check if the
> variable has the correct type.

Rather, if you want to be sure then it's likely you
haven't fully understood "duck typing" (or "latent typing"
or "dynamic typing", depending on who you are.) See
http://c2.com/cgi/wiki?DuckTyping
and
http://mindview.net/WebLog/log-0052
for more details.

In short, doing explicit type checks is not the Python way.

Well always checking the types of the parameters in the start of every
function is too many redundant work. Instead we can define a syntax
that will check the type information at the beginning of function
declaration, and even before executing any code from that function.
That syntax might look like that:

def larry(guido >= foo):
<Just actual code, no checking since the interpreter handles it>

The few times I've needed explicit type checks is to
do slightly different behaviour based on an type.
For example,

def count_lines(f):
if isinstance(f, str):
f = open(f, "U")
return len(f)

Your proposal doesn't do anything for this use case.

In addition, what you're thinking of could be
done with the new "decorator" proposal. Here's
a sketch of one way to handle it. Ideally
it should propogate the arglist and support
keyword args and default parameters, but this
gives the essense.
.... def __init__(self, *types):
.... self.types = types
.... def __call__(self, func):
.... def f(*args):
.... for arg, basetype in zip(args, self.types):
.... if not isinstance(arg, basetype):
.... raise TypeError("incompatible types")
.... return func(*args)
.... return f
........ def spam(i, s):
.... return s*i
....Traceback (most recent call last):
File "<stdin>", line 1, in ?

(Wow! My first used of the @-thingy and it worked
on the first try.)

And if we have a function that have much more parameters, this kind of
a syntax will really help us to remove redundant code from our
functions. The syntax can be defined by something like:

[
== for exact test

>= for superclass test

<= for subclass test
!= for is neither subclass nor superclass
in for attribute testing
]


But if you look through Python code you'll see that
most of the use cases look like my emulation of
polymorphism and not your strict type checking. Here
are some examples from the stdlib

def __eq__(self, other):
if isinstance(other, BaseSet):
return self._data == other._data
else:
return False



if isinstance(instream, basestring):
instream = StringIO(instream)



elif (not isinstance(self.delimiter, str) or
len(self.delimiter) > 1):
errors.append("delimiter must be one-character string")


if isinstance(longopts, str):
longopts = [longopts]
else:
longopts = list(longopts)


if isinstance(filenames, basestring):
filenames = [filenames]



if isinstance(object, types.TupleType):
argspec = object[0] or argspec
docstring = object[1] or ""
else:
docstring = pydoc.getdoc(object)

The only examples I found along the lines you wanted were
in pickletools.py, copy.py, sets.py, and warnings.py.
It just isn't needed that often in Python programming.


Andrew
(e-mail address removed)

 

[Istvan Albert]

I have a little proposal about type checking in python.

This proposal comes up very often. I've been around in c.l.py
for only about a year now but I saw it proposed what seems like
on a monthly basis.

When I started using python I was worried about ending up with
nasty errors based on inappropriate function calls. In practice
,it turns out, in python I almost never make the kind of errors
that this type-checking would catch...

Istvan.

 

[Peter Otten]

Andrew Dalke wrote:

[examples of isinstance() usage in the standard library]

The only examples I found along the lines you wanted were
in pickletools.py, copy.py, sets.py, and warnings.py.
It just isn't needed that often in Python programming.

There are at least three more idioms that could (sometimes) be replaced
by type-checking:

# "casting"
s = str(s)
it = iter(seq)

# check for an "interface"
try:
lower = s.lower
except AttributeError:
raise TypeError
else:
lower()

# check for an "interface", LBYL-style
if not hasattr(foo, "bar"):
raise TypeError

These are more difficult to hunt and I'm lazy.

However, I don't think you can tell from the structure alone whether an
explicit type declaration would be a superior solution in these cases.

Should Python be extended to allow type declarations, I expect them to
appear in places where they reduce the usefulness of the code while
claiming to make it safe...

Peter

 

[Andrew Dalke]

There are at least three more idioms that could (sometimes) be replaced
by type-checking:

Though from what I can tell the OP wanted "real" type
checking, as found in Java and C. The original text was

] On the other hand in languages like C, java etc...
] where types are strict we have the guarantee that
] the variables will always be the same type without
] any change.

# "casting"

Wouldn't that be type conversion, and not a check?

# check for an "interface"
try:
lower = s.lower
except AttributeError:
raise TypeError
else:
lower()

Mmm, there's a problem with this checking. Consider

class Spam:
lower = 8

Though the odds are low.

# check for an "interface", LBYL-style
if not hasattr(foo, "bar"):
raise TypeError

Aren't both of these the "look before you leap" style?
Well, except for checking for the callable.

These are more difficult to hunt and I'm lazy.

However, I don't think you can tell from the structure alone whether an
explicit type declaration would be a superior solution in these cases.

I do at times use tests like you showed, in order
to get a more meaningful error message. I don't
think they are considered type checks -- perhaps
protocol checks might be a better term?

Should Python be extended to allow type declarations, I expect them to
appear in places where they reduce the usefulness of the code while
claiming to make it safe...

As was mentioned in the @PEP (as I recall) one of the
interesting possibilities is to use type checks with
support for the adapter PEP so that inputs parameters
can get converted to the right type, rather like your
first point.

@require(PILImage)
def draw(image):
...

As new image data types are added, they can be passed to
draw() without need for an explicit convert step, so
long as the adapter is registered.

Andrew
(e-mail address removed)

 

[Peter Otten]

There are at least three more idioms that could (sometimes) be replaced
by type-checking:

Though from what I can tell the OP wanted "real" type
checking, as found in Java and C. The original text was

] On the other hand in languages like C, java etc...
] where types are strict we have the guarantee that
] the variables will always be the same type without
] any change.

# "casting"

Wouldn't that be type conversion, and not a check?

In a way it is both. Consider

def f(s):
s = str(s)

versus

interface Stringifiable:
def __str__(self) -> str:
pass

def f(Stringifiable s):
s = str(s)

In the latter form the compiler can guarantee not that the call succeeds,
but that the interface is there at least, while in the former both steps
(check and creation/conversion) are blurred in the single str() call.
Don't let yourself detract from that by the fact that in Python str() hardly
ever fails.

Mmm, there's a problem with this checking. Consider

class Spam:
lower = 8

Of course it is not a full replacement of a type declaration, but often used
to achieve similar goals. I guess that it would be immediately dismissed by
people experienced only in statically typed languages and most likely
replaced with something like

def f(str s):
s = s.lower()
....

thus (sometimes) unnecessarily excluding every object with a string-like
"duck-type", i. e. a sufficient subset of the str interface for the above
function to work.

Though the odds are low.


Aren't both of these the "look before you leap" style?
Well, except for checking for the callable.

I thought that "look before you leap" is doing something twice, once to see
if it is possible and then for real. Let's concentrate on attribute access
for the sake of the example:

# good style, IMHO
try:
lower = s.lower
except AttributeError:
# handle failure
# use lower, don't access s.lower anymore.

# LBYL, bad IMHO
try:
s.lower
except AttributeError
# handle failure
# use lower, accessing it via attribute access again, e. g.
lower = s.lower # an unexpected failure lurks here

# LBYL, too, but not as easily discernable
# and therefore seen more often in the wild
if hasattr(s, "lower"):
lower = s.lower # an unexpected failure lurks here
else:
# handle failure

And now, just for fun, a class that exposes the difference:
.... def __init__(self):
.... self.lower = 42
.... def __getattribute__(self, name):
.... result = object.__getattribute__(self, name)
.... delattr(self, name)
.... return result
....42
.... print d.lower
....
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<stdin>", line 5, in __getattribute__
AttributeError: 'DestructiveRead' object has no attribute 'lower'

This would of course be pointless if there weren't (sometimes subtle)
real-word variants of the above.

I do at times use tests like you showed, in order
to get a more meaningful error message. I don't
think they are considered type checks -- perhaps
protocol checks might be a better term?

Maybe - I think I'm a bit fuzzy about the terms type, interface and
protocol. In a way I tried to make that fuzziness explicit in my previous
post: In a staticically typed language you may be tempted to ensure the
availability of a certain protocol by specifying a type that does fulfill
it, either because templates are not available or too tedious, and you
cannot make an interface (abstract base class) for every single method, no?

In a dynamically typed language the same conceptual restrictions may be
spread over various constructs in the code (not only isinstance()), and the
translation from one system into the other is far from straightforward.

As was mentioned in the @PEP (as I recall) one of the
interesting possibilities is to use type checks with
support for the adapter PEP so that inputs parameters
can get converted to the right type, rather like your
first point.

@require(PILImage)
def draw(image):
...

why not rely on PILImage's __init__() or __new__() for that:

def draw(image):
image = PILImage(image)

Since the introduction of __new__() this can be almost a noop if the image
is already a PILImage instance.

As new image data types are added, they can be passed to
draw() without need for an explicit convert step, so
long as the adapter is registered.

I had a quick look at the PEP 246 - Object Adaptation, and I'm not sure yet
whether the pythonic way to write e. g.

class EggsSpamAndHam (Spam,KnightsWhoSayNi):
def ham(self): print "ham!"
def __conform__(self,protocol):
if protocol is Ham:
# implements Ham's ham, but does not have a word
return self
if protocol is KnightsWhoSayNi:
# we are no longer the Knights who say Ni!
raise adaptForceFailException
if protocol is Eggs:
# Knows how to create the eggs!
return Eggs()

would rather be along these lines:

class Eggs:
def asEggs(self):
return self

class EggsSpamAndHam(Spam): # no need to inherit form KnightsWhoSayNi
def ham(self): print "ham!"
def asEggs(self):
return Eggs()

That and a class-independent adaptation registry should achieve the same
functionality with much simpler code. The real limits to protocol
complexity are human understanding rather than technical feasibility.

Of course I may change my mind as soon as I see a compelling real world
example...


Peter