"no variable or argument declarations are necessary."

[Diez B. Roggisch]

Now some of the Python-is-perfect crowd seems to suffer from a "Blub

paradox" (http://c2.com/cgi/wiki?BlubParadox). They see annoying,
static typed languages like C and Java, and they see pleasant,
dynamically typed languages like Python, and conclude that static
types = annoying, when in fact they can be orthogonal. So, can there
be a language that's both statically typed, and pleasant? I haven't
used one yet, but lately I've been reading about Haskell and want to
give it a try.

Nobody says that there can't be possibly better languages like python
overall, or for specific tasks. However, this discussion is about
introducing type-checking to python. And as someone who has done his
fair share of FP programming let me assure you that

- all declarations are fully type annotated. The inference only comes
into play on _expressions_. The result in python would be that you'd
have to write

def foo(x:int):int :
return 10

but then could use

x = foo()

which made the inference possible. But it _doesn't figure out that foo
returns an int because there is one returned, and misses the :int in
the declaration! Genericity is reached through solving somewhat more
complicated type equations - but these still require declarations:

def bar(l:list[whatever]):whatever :
return head(l)

x = bar([10])

can be resolved as [] will me a list-constructor that gets passed an
int literal - wich in turn means that whatever as type-variable is
bound to int, and thus x is an int, as that is the return type of bar.

- FPs share their own set of problems - try writing a server. The have
inherent troubles with event-driven programs. Then you need monads, and
that makes things a little bit more ugly...

Still, FP is cool. But python too. And just attaching some
type-inference to python won't work.

Diez

 

[Paul Rubin]

- FPs share their own set of problems - try writing a server. The
have inherent troubles with event-driven programs.
Erlang?

Still, FP is cool. But python too. And just attaching some
type-inference to python won't work.

Yeah, I've figured declarations in Python would be more like Common
Lisp's, i.e. optional, enforced at compile time only when the compiler
can easily figure it out, and at runtime otherwise.

 

[Diez B. Roggisch]

Erlang?

Guess what, worked with that, too And let me assure you - it does
have pretty much runtime type error issues. It's interpreted. I'm not
sure what the compiler/parser gets at loading a source file. And the
Mnesia distributed database lets you easily query the wrong values...
It's interesting, and it's concurrent programming paradigms are great.
But it's far from being perfect, and needs thourough testing before
deploying new code.

Yeah, I've figured declarations in Python would be more like Common
Lisp's, i.e. optional, enforced at compile time only when the compiler
can easily figure it out, and at runtime otherwise.

Easy cases are easy... The thing is: I'm all for typechecking as long
as it doesn't burden me. In FP it doesn't, as the expressional power is
way better. But in JAVA, it does. And just doing wishful-thinking about
that Python should do better that won't help...

Diez

 

[Steve Holden]

Fredrik Lundh a écrit :



Still, it would be great if there were also some automatic type checking
in place. Using assert is hardly automatic and non intrusive.

I mean, why not ? Why does the compiler let me do that when you know
perfectly that that code is incorrect :
def f():
return "a" + 5

Of course the system can't be perfect but it doesn't need to be. It
doesn't need to constrain us in any way but if it can detect some errors
early, then it is worth it.

While this is a perfectly acceptable feature request, we should remember
that Python is developed and maintained by a volunteer team. Do we
*really* want them spending their time adding "features" like this?

regards
Steve

 

[Antoon Pardon]

Well, you didn't tell us what you had in mind.

Indeed I hadn't. It wasn't needed for the question I posed then.

You just said "let's
introduce something like any". I showed you existing implementations of
such a concept that have problems.

But as far as I can see that is a problem of the implementation
not necessarily of the concept.

You say "thats not what _I_ have in
mind, so your criticism doesn't apply." Guess what, I can't read your
mind. But you did not tell me in what your idea is different from
existing concepts.

Indeed you can't read my mind, but what pops up in your mind shows
your preconceptions. You could have just answered the question
as it was posed, instead of filling in the details yourself, maybe
remarking that you didn't see how it would work with current
type systems you know off.

Again - where is your idea layed out in (more) detail, so that one can
discuss them? That was all that I'm asking - which of course you
carefully avoided...

Sure I'm reluctant to give details. I consider this a hostile
environment, for this kind of proposals. I'm sure people will
be able to come up with all kind of problems my idea won't solve
and will see this as a reason to think the idea is useless.

Since I have no intention to

Where exactly come the facts?

About how you filled in the details yourself when all I mentioned
was a type system with an "ANY" type.

All I see is some vague "there should be
something better, by introducing ANY". But no details how typechecking
then would work. I showed you that existing type systems can't properly
cope with ANY so far and allow for much errors. Just saying "but mine
won't" is a little bit thin, don't you think?l

The main idea is that type information would be available at two places.

1) The names, which carry the declared type.

2) The objects which carry the type/class they belong too.

When an object is bound to a name, a check is made that the type of the
object is compatible with the declared type of the name.

A name with type ANY, would be just like any python variable or
instance now. Python could implement this by instead of storing its
variables in dictionaries as (name, value) tuples, by storing them
as (name, declaration, value) tuples and making the necesarry checks
at (re)bind time. Variables that are not declared would get the
ANY declaration so that current scripts would just remain working
as they do now. But as the compiler got more sophisticated some
of these checks might be doable at compile time instead of at
run time.

 

[Paul Rubin]

Guess what, worked with that, too And let me assure you - it does
have pretty much runtime type error issues. It's interpreted.

Yes, it makes no attempt at being statically typed. It's like Python
that way, AFAIK.

Easy cases are easy... The thing is: I'm all for typechecking as long
as it doesn't burden me. In FP it doesn't, as the expressional power is
way better. But in JAVA, it does. And just doing wishful-thinking about
that Python should do better that won't help...

How about Lisp? It seems to do some good there, without getting in
the way.

 

[Diez B. Roggisch]

But as far as I can see that is a problem of the implementation
not necessarily of the concept.

Without any concept, sure there can't be problems with that concept.

Indeed you can't read my mind, but what pops up in your mind shows
your preconceptions. You could have just answered the question
as it was posed, instead of filling in the details yourself, maybe
remarking that you didn't see how it would work with current
type systems you know off.

How can one answer a question without what he knows? Sure I fill in the
context. If I wouldn't know that this whole NG is about python, I
wouldn't know how to answer most of the questions that arise here.
Which is a general thing about communication.

But I certainly have had more contact with type systems than you had -
so my "filling-out" was by no means unreasonable - as you imply.
Without further details, one can only guess. And debuking guesswork by
saying "but _that_ wasn't what I meant" as you permanently do is easy -
but doesn't make _your_ point valid.

Sure I'm reluctant to give details. I consider this a hostile
environment, for this kind of proposals. I'm sure people will
be able to come up with all kind of problems my idea won't solve
and will see this as a reason to think the idea is useless.

Why hostile? Because people will possibly destroy your wishful thinking
by providing counter examples - if they exist? I doubt that a serious
proposal would get suppressed here - as the static compilers that have
shown up recently prove, as they have been greeted and met with
reasonable criticism where it was in order.

You didn't come up with an Idea so far, as I'm concerned. Just some
random thoughts.

But then again, here we go:

The main idea is that type information would be available at two places.

1) The names, which carry the declared type.

JAVA.

2) The objects which carry the type/class they belong too.

That already is the case.

When an object is bound to a name, a check is made that the type of the
object is compatible with the declared type of the name.

JAVA exactly does that.

A name with type ANY, would be just like any python variable or
instance now. Python could implement this by instead of storing its
variables in dictionaries as (name, value) tuples, by storing them
as (name, declaration, value) tuples and making the necesarry checks
at (re)bind time. Variables that are not declared would get the
ANY declaration so that current scripts would just remain working
as they do now.

So far, this is exactly what java does - plus a more dynamic approach
to method/function invocation on ANY. So it seems my assumptions about
what you had in mind weren't so false after all, eh?

But as the compiler got more sophisticated some
of these checks might be doable at compile time instead of at
run time.

Well, that exactly is the point where we make the transition from "this
is how things work" to pure speculation. Can't say that there won't be
a solution someday - but certainly it requires much more, and from
above nobody can say that this would solve _any_ problem.

What you propose above is what JAVA does - plus more dynamicity. Well,
given that even the "non-dynamic, everything has to be annotated" JAVA
fails to deal with ANY (called Object there), I can't see how a more
dynamic environment will do _better_ in that respect. So unless you lay
out some more detailed ideas how that works, this surely won't do much
if any better than JAVA does today - and JAVA sucks _precisely_ because
of the half-static-half-dynamic nature. It gives you both troubles -
runtime errors like in python, together with compile-time limitations.

Let's face it: you don't know much about type-systems. I do know a bit
more - but don't claim to possess the holy grail. And I don't say that
more powerful analyzing isn't possible. However, all you do so far is
fantasizing and "fail to see why not". Well, that failure might be
because of limited sight on your side - not necessarily on our, which
you constantly claim.

Visions are a nice thing - but actually, in the scientific domain not
so much a vision, but proofs are what is needed. And if you consider it
hostile that nobody buys your ideas because so far they aren't more
than marketing/whishful thinking, I'm sorry that I can't help you.

Diez

 

[Fredrik Lundh]

I mean, why not ? Why does the compiler let me do that when you know
perfectly that that code is incorrect :

def f():
return "a" + 5

probably because the following set is rather small:

bugs caused by invalid operations involving only literals, that are not
discovered during initial testing

</F>

 

[Paul Rubin]

Excep that's not a decleration, that's a binding. That's identical to
the Python fragment:

a = 3
return a + 1

except for the creation of the new scope. Not a variable decleration
in site.

Really not though, the scope is what makes it a declaration. In
Python you can say:

for i in 1,2:
if i == 2: print x # x is unbound the first time through the loop
x = 9 # but bound the second time

and the print doesn't raise an error. Further, 'x' is still in scope
even after the loop exits. Scheme lets you say something like

(define (counter)
(let ((k 0))
(define (f)
(set! k (+ 1 k))
k)
f))

Yeah, I know you'd write it a bit more concisely in Scheme, but I
wanted to make it look like the Python equivalent:

def counter():
k = 0
def f():
k += 1
return k
return f

The trouble is, in f, k is neither local nor global, so Python throws
up its hands and raises an error. There's no way to tell it where to
find k. This could be solved with a declaration, if Python understood it.

 

[Diez B. Roggisch]

How about Lisp? It seems to do some good there, without getting in

the way.

I don't know much about lisp. But the thing is that one of the most
important data structures in python (and basically the only one in
LISP), lists, are a big problem to type-checking if they aren't
homogenous. So I guess one can write a compiler that generates
specialized code for lists of a certain homogenous kind, like lists of
int, float and maybe string - and as long as you ensure that code is
called with such a list, you're ok. But I think that is what psyco
does, too (no idea how these two compare)

Basically, all type-inferencing boils down to collecting constraints on
variable values. Like in this expression

z = x + y

It will be internall represented by this:

z:C = x:A + y:B

which introduces type-variables A,Band C. That tells us that all
operations __plus__ and their respective types could be meant, thus A B
C are constrained by these types. Which, in an overloading-allowing
language, can get pretty much. The trick is to find a solution for the
variables that satisfy all the constraints. And a solution are actual
types, not ad-hoc sets of types - otherwise, you can't create any
specialized code, and the compiler will puke on you.

Saying

z += 10

now creates a constraint that binds C to int, and subsequently A and B.

Now the problem is someting like this:

if x is None:
x = 10
else:
x = x * 10

Lets assume A as type-var for x. Then we have the two contstraints A ->
int and A -> None. Now - is there a type that satisfies both
constraints? No. And even worse, A shall be int by x = 10 - but that
collides with A->None. So the only thing you can do is resort to A=ANY
- and interpret that code above

Diez

 

[Alex Martelli]

No it is not. You may not have had any use for this
kind of code, but unfamiliary with certain types
of problems, doesn't make something contrived.

It's so contrived it will raise a SyntaxError due to the spurious extra
colon on the first line;-).

Or, consider, once the stray extra colon is fixed:

Helen:/tmp alex$ cat ap.py
def ap():
egold = 0
while egold < 10:
if test():
ego1d = egold + 1

Helen:/tmp alex$ pychecker ap.py
Processing ap...

Warnings...

ap.py:4: No global (test) found
ap.py:5: Local variable (ego1d) not used
Helen:/tmp alex$

If you're so typo-prone and averse to unittests that you consider this
kind of issue to be a serious problem, just use pychecker and get
informed about any such typo, just as above.

Incessant whining about the non-existent advantages of declarations,
rather than the simple use of tools that can diagnose such spelling
mistakes without any need for declarations, would qualify you as a troll
even if you didn't have a long history of trolling this group...

Names do get misspelled and sometimes that misspelling is hard to spot.

It's totally trivial, of course, as shown above, and there is no need to
pervert and distort the language for the purpose, as you, troll, have
kept whining about for years. I'm partial to pychecker -- that's what
we use at Google, and we also, incidentally, recently had the good
fortune to hire Neal Norwitz, pychecker's author; but there are several
other free tools that perform similar tasks, albeit with very different
philosophy, such as Logilab's pylint...:

Helen:/tmp alex$ pylint ap.py
No config file found, using default configuration
************* Module ap
W: 2: Bad indentation. Found 3 spaces, expected 4
W: 3: Bad indentation. Found 3 spaces, expected 4
W: 4: Bad indentation. Found 5 spaces, expected 8
W: 5: Bad indentation. Found 7 spaces, expected 12
C: 0: Too short name "ap"
W: 0: Missing docstring
W: 0: Missing required attribute "__revision__"
C: 1:ap: Too short name "ap"
W: 1:ap: Missing docstring
E: 4:ap: Undefined variable 'test'
W: 5:ap: Unused variable 'ego1d'

[rest of long critique of ap.py snipped]

Again, unused variables (typos...) get easily diagnosed without any need
for declarations. (Similar tools, of course, apply to languages
requiring declaration, to diagnose a variable that's declared but
unused, which is a very bad code smell typical of such languages). Of
course, pylint is about enforcing all sort of code rules, such as, by
default, indentation by multiples of 4 spaces, name length, docstrings,
and so on; while pychecker is much simpler and more narrowly aimed at
diagnosing likely mistakes and serious code smells.

But, with either tool or any of many others, there is no need at all for
declarations in order to catch typos (of course, unittests are still a
VERY good idea -- catching all typos and even coding rules violations is
NO guarantee that your code is any good, testing is A MUST).

I don't think anyone with a little bit of experience will be so naive.

Heh, right. After all, _I_, for example, cannot have even "a little bit
of experience" -- after all, I've been programming for just 30 years
(starting with my freshman year in university), and anyway all I have to
show for that is a couple of best-selling books, and a stellar career
culminating (so far) with my present job as Uber Technical Lead for
Google, Inc, right here in Silicon Valley... no doubt Google's reaching
over the Atlantic to come hire me from Italy, and the US government's
decision to grant me a visa under the O-1 category (for "Aliens with
Outstanding Skills"), were mere oversights on their part that,
obviously, I cannot have even "a little bit of experience", given that I
(like great authors such as Bruce Eckel and Robert Martin) entirely
agree with the opinion you deem "so naive"... that any automatic
catching of misspellings can never be a substitute for unit-testing!


Ah well -- my good old iBook's settings had killfiles for newsreaders,
with not many entries, but yours, Antoon, quite prominent and permanent;
unfortunately, that beautiful little iBook was stolen
(http://www.papd.org/press_releases/8_17_05_fix_macs_211.html), so I got
myself a brand new one (I would deem it incorrect to use for personal
purposes the nice 15" Powerbook that Google assigned me), and it takes
some time to reconstruct all the settings. But, I gotta get started
sometime -- so, welcome, o troll, as the very first entry in my
brand-new killfile.

In other words: *PLONK*, troll!-)


Alex

 

[Paul Rubin]

ap.py:4: No global (test) found
ap.py:5: Local variable (ego1d) not used
Helen:/tmp alex$

If you're so typo-prone and averse to unittests that you consider this
kind of issue to be a serious problem, just use pychecker and get
informed about any such typo, just as above.

That's very helpful, but why isn't it built into Python?

 

[Alex Martelli]

That's very helpful, but why isn't it built into Python?

Because some users will prefer to use a different approach to checking,
for example, such as pylint (much more thorough in enforcing coding
rules and checking for all sort of things) or nothing (much faster than
pychecker, which in turn is faster than pylint). Just as for other
programming tools, such as, say, an editor, I think it's wise to avoid
excessive and premature standardization on one specific tool to the
detriment of others. (IDLE is "bundled with" Python, but not _built
into_ it -- indeed some would claim that the bundling was too much).

Not all tools need evolve at the same speed as the core language, which
currently follows a wise policy of "major" releases (2.3, 2.4, etc)
about 18 to 24 months apart, and NO feature changes for "point" release
(2.4.2 has exactly the same features as 2.4.1 -- it just fixes more
bugs). Any tool which gets built into python (or, less strictly but
still problematically, is separate but bundled with it) must get on
exactly the same schedule and policy as Python itself, and that is
definitely not something that's necessarily appropriate.

If you're worried about the end-users which can't be bothered to
download tools (and, for that matter, libraries) separately from the
main language, the solution is "sumo releases" -- Enthought Python (from
Enthought) being an extreme example, but Active Python (from
ActiveState) comes with quite a few bundled add-ons, too. I believe
that Linux has proven the validity of this general model: having the
"core" (mostly the kernel, in Linux's case; the language and standard
library, in Python's) evolve and get released as its own speed, and
having _distributions_ bundling the core with different set of tools and
add-ons get released on THEIR preferred schedules, independently.


Alex

 

[Antoon Pardon]

Well, that exactly is the point where we make the transition from "this
is how things work" to pure speculation.

Everything starts with pure speculation. I had no intention of
introducing the big type system here. I just think people who
oppose type systems, limit themselves too much to how things
currently work and from there oppose anything that resembles
a type system instead of thinking of what they would like
a type system to do even if it is not implemtable now.

Can't say that there won't be
a solution someday - but certainly it requires much more, and from
above nobody can say that this would solve _any_ problem.

What you propose above is what JAVA does - plus more dynamicity.

So it isn't exactly JAVA afterall.

Well,
given that even the "non-dynamic, everything has to be annotated" JAVA
fails to deal with ANY (called Object there),

So please explain how my system fails with the ANY.

I can't see how a more
dynamic environment will do _better_ in that respect.

You first argued that a type system had to limit the coder and you
gave the example of the homegeneous list. I suggested an ANY type
and asked how a homegeneous list of ANY's would limit the coder.
Your respons was that this was like a JAVA Object or C void* hack,
that can be used to circumvent the type system and cause all kinds
of problems, presumebly python was not vulnerable to.

So I would either like you to explain how my idea can be used to
circumvent the type system and cause problems, which don't
concern python or explain how this system will limit the coder
with respect what can be done in python.

So unless you lay
out some more detailed ideas how that works, this surely won't do much
if any better than JAVA does today - and JAVA sucks _precisely_ because
of the half-static-half-dynamic nature. It gives you both troubles -
runtime errors like in python, together with compile-time limitations.

I have no need to convince you. It was you who made this claim how
a type system had to limit the coder. That you can't see how it
could be done is not an argument. Surely if a typesystem *must* limit
the user or else cause all kinds of problems as more general argument
can be given that doesn't depend on the specific implementation.

Let's face it: you don't know much about type-systems. I do know a bit
more - but don't claim to possess the holy grail. And I don't say that
more powerful analyzing isn't possible. However, all you do so far is
fantasizing and "fail to see why not". Well, that failure might be
because of limited sight on your side - not necessarily on our, which
you constantly claim.

Well, I could accept that if you would have explained what the problem
would be with my system instead of just saying you couldn't see how
it would be more usefull than JAVA Objects.

Visions are a nice thing - but actually, in the scientific domain not
so much a vision, but proofs are what is needed.

Well you made the claim that a type system had to limit the coder.
Now prove your claim. Untill this is done I see no problem thinking
about type systems that don't limit the coder.

And if you consider it
hostile that nobody buys your ideas because so far they aren't more
than marketing/whishful thinking, I'm sorry that I can't help you.

I don't consider it hostile that nobody buys my ideas. I considered
the environment here hostile to type systems long before I brought
my two cents to the discussions about this subject here.

 

[Bengt Richter]

It's so contrived it will raise a SyntaxError due to the spurious extra
colon on the first line;-).

Glad to see a smiley ;-)

Or, consider, once the stray extra colon is fixed:

[... demonstration of effective tool use to diagnose contrived[1] snippet's problems ...]

[1] the code snippet certainly seems contrived to me too. Not sure whether entire class of code
that it may have been intended to represent is contrived, but I'm willing to give the benefit
of the doubt if I sense no ill will ;-)
[...]This strikes me as a somewhat provocative and unfortunately ambiguous statement ;-)

The way I read it was to assume that Antoon was agreeing with the judgement that
the 'sense of security' would be false, and that he was saying that an experienced
programmer would not be so naive as to feel secure about the correctness of his
code merely on the basis of a compiler's static checks (and thus skip run-time testing).

Heh, right. After all, _I_, for example, cannot have even "a little bit
of experience" -- after all, I've been programming for just 30 years
(starting with my freshman year in university), and anyway all I have to
show for that is a couple of best-selling books, and a stellar career
culminating (so far) with my present job as Uber Technical Lead for
Google, Inc, right here in Silicon Valley... no doubt Google's reaching
over the Atlantic to come hire me from Italy, and the US government's
decision to grant me a visa under the O-1 category (for "Aliens with
Outstanding Skills"), were mere oversights on their part that,
obviously, I cannot have even "a little bit of experience", given that I
(like great authors such as Bruce Eckel and Robert Martin) entirely
agree with the opinion you deem "so naive"... that any automatic
catching of misspellings can never be a substitute for unit-testing!

I somehow don't think Antoon was really disagreeing with that (maybe because after
45+ years of programming and debugging I think it would be too absurd ;-)

Ah well -- my good old iBook's settings had killfiles for newsreaders,
with not many entries, but yours, Antoon, quite prominent and permanent;
unfortunately, that beautiful little iBook was stolen
(http://www.papd.org/press_releases/8_17_05_fix_macs_211.html), so I got

Ugh, bad luck ... but if it had to happen, better that it wasn't from your home.

myself a brand new one (I would deem it incorrect to use for personal
purposes the nice 15" Powerbook that Google assigned me), and it takes
some time to reconstruct all the settings. But, I gotta get started
sometime -- so, welcome, o troll, as the very first entry in my
brand-new killfile.

I'd urge you to reconsider, and see if you really see trollish _intent_
in Antoon ;-)

In other words: *PLONK*, troll!-)

IMO that's a bit harsh, especially coming from a molto certified heavyweight ;-)

Antoon doesn't strike me as having the desire to provoke for the sake of provoking,
which seems to me to be the sine qua non hallmark of trolls. Of course anyone
with any ego is likely to be feel like posting defensive tit-for-tat to "correct"
any inadequate appreciation of worth coming from the other side, and that can degenerate
into something that looks like pure troll postings, but I think that is normal succumbing
to ego temptations, not signs true trollishness. E.g., ISTM Antoon and Diez were managing
rather well, both being frustrated in getting points across, but both displaying patience
and a certain civility. To me, the "winning" posts are the ones that further the development
of the "truth" about the topic at hand, and avoid straying into ad hominem irrelevancies.
OTOH, I think everyone is entitled at least to ask if a perceived innuendo was real and
intentional (and should be encouraged to do so before launching a counter-offence).
Sometimes endless niggling and nitpicking gets tiresome, but I don't think that is necessarily
troll scat either. And one can always tune out ;-)

Anyway, thanks for the pychecker and pylint demos. And I'm glad that we can enjoy your posts again,
even if for a limited time.

-- Martellibot admirer offering his .02USD for peace ... ;-)

Regards,
Bengt Richter