Using the Python Interpreter as a Reference

[Ian Kelly]

My language combines generators and collection initializers, instead of
creating a whole new syntax for comprehensions.

[| for i in 0..10: for j in 0.10: yield return i * j |]

Are we supposed to intuit what that means?

Is | a token, or are the delimiters [| and |] ?

Is there a difference between iterating over 0..10 and iterating over
what looks like a float 0.10?

What is "yield return"?

I would assume that "yield return" is borrowed from C#, where it is
basically equivalent to Python's yield statement. The advantage of
using two keywords like that is that you can compare the statements
"yield return foo" and "yield break", which is a bit clearer than
comparing the equivalent "yield foo" and "return".

Having to type out "yield return" in every comprehension seems a bit
painful to me, but I can understand the approach: what is shown above
is a full generator, not a single "generator expression" like we use
in Python, so the statement keywords can't be omitted. It's trading
off convenience for expressiveness (a bad trade-off IMO -- complex
generators should be named, not anonymous).

That does not follow. Lambdas and def functions are the same thing in
Python, but Python requires a special keyword.

I think the implication is that Unit has only one syntax for creating
functions, which is lambda-style. In any case, why does Python
require a special keyword? def is only used in a statement context,
and lambda is only used in an expression context. Why not use the
same keyword for both? I think the answer is historical: def came
first, and when anonymous functions were added it didn't make sense to
use the keyword "def" for them, because "def" implies a name being
defined.

Cheers,
Ian

 

[Neil Cerutti]

I think the implication is that Unit has only one syntax for
creating functions, which is lambda-style. In any case, why
does Python require a special keyword? def is only used in a
statement context, and lambda is only used in an expression
context. Why not use the same keyword for both? I think the
answer is historical: def came first, and when anonymous
functions were added it didn't make sense to use the keyword
"def" for them, because "def" implies a name being defined.

I've always held with the "anti-functional style conspiracy"
interpretation of Python's lambda expressions. They were added
but grudgingingly, made weak on purpose to discourage their use.

 

[Gregory Ewing]

I thinking tabs are
out-of-date. Even the MAKE community wishes that the need for tabs
would go away

The situation with make is a bit different, because it
*requires* tabs in certain places -- spaces won't do.
Python lets you choose which to use as long as you don't
mix them up, and I like it that way.

let Parse = public static method (value: String)
throws(FormatException UnderflowException OverflowException)

Checked exceptions? I fear you're repeating a huge mistake
going down that route...

 

[Gregory Ewing]

I've always held with the "anti-functional style conspiracy"
interpretation of Python's lambda expressions. They were added
but grudgingingly, made weak on purpose to discourage their use.

Seems to me that Python's lambdas are about as powerful
as they can be given the statement/expression distinction.
No conspiracy is needed, just an understandable desire on
Guido's part not to do violence to the overall syntactic
style of the language.

 

[Neil Cerutti]

Seems to me that Python's lambdas are about as powerful as they
can be given the statement/expression distinction. No
conspiracy is needed, just an understandable desire on Guido's
part not to do violence to the overall syntactic style of the
language.

It's true. Most conspiracy theories do fall apart once facts and
clear thinking are applied. But we love them anyway.

 

[Travis Parks]

The situation with make is a bit different, because it
*requires* tabs in certain places -- spaces won't do.
Python lets you choose which to use as long as you don't
mix them up, and I like it that way.


Checked exceptions? I fear you're repeating a huge mistake
going down that route...

Exception handling is one of those subjects few understand and fewer
can implement properly in modern code. Languages that don't support
exceptions as part of their signature lead to capturing generic
Exception all throughout code. It is one of those features I wish .NET
had. At the same time, with my limited experience with Java, it has
been a massive annoyance. Perhaps something to provide or just shut
off via a command line parameter. What indications have there been
that this has been a flaw? I can see it alienating a large group of up-
and-coming developers.

 

[Travis Parks]

My language combines generators and collection initializers, instead of
creating a whole new syntax for comprehensions.
[| for i in 0..10: for j in 0.10: yield return i * j |]

Are we supposed to intuit what that means?

Is | a token, or are the delimiters [| and |] ?

Is there a difference between iterating over 0..10 and iterating over
what looks like a float 0.10?

What is "yield return"?

I would assume that "yield return" is borrowed from C#, where it is
basically equivalent to Python's yield statement.  The advantage of
using two keywords like that is that you can compare the statements
"yield return foo" and "yield break", which is a bit clearer than
comparing the equivalent "yield foo" and "return".

Having to type out "yield return" in every comprehension seems a bit
painful to me, but I can understand the approach: what is shown above
is a full generator, not a single "generator expression" like we use
in Python, so the statement keywords can't be omitted.  It's trading
off convenience for expressiveness (a bad trade-off IMO -- complex
generators should be named, not anonymous).

That does not follow. Lambdas and def functions are the same thing in
Python, but Python requires a special keyword.

I think the implication is that Unit has only one syntax for creating
functions, which is lambda-style.  In any case, why does Python
require a special keyword?  def is only used in a statement context,
and lambda is only used in an expression context.  Why not use the
same keyword for both?  I think the answer is historical:  def came
first, and when anonymous functions were added it didn't make sense to
use the keyword "def" for them, because "def" implies a name being
defined.

Cheers,
Ian

Most languages I have worked with have a "lambda" syntax and a
function syntax. It has always been a historical artifact. Languages
start out avoiding functional features and then eventually adopt them.
It seems that eventually, convenient high-order functions become a
must-have (most standard algorithm packages). It is a conflict between
old C-style programming and the need for functional code. As soon as
functions can be assigned to variables, the code starts looking oddly
like JavaScript.

 

[Chris Angelico]

Languages that don't support
exceptions as part of their signature lead to capturing generic
Exception all throughout code. It is one of those features I wish .NET
had. At the same time, with my limited experience with Java, it has
been a massive annoyance.

In Java, it mainly feels like syntactic salt. There's still a class of
RuntimeExceptions that aren't listed in the signature, so you still
have to concern yourself with the possibility that unexpected
exceptions will propagate; and you're forced to decorate every method
with the list of what it might propagate up, other than that.

It's like const-decorating a series of functions in C++, only far less
consequential and requiring far more typing.

ChrisA

 

[Steven D'Aprano]

On Sun, Nov 27, 2011 at 4:55 PM, Steven D'Aprano


I think the implication is that Unit has only one syntax for creating
functions, which is lambda-style. In any case, why does Python require
a special keyword? def is only used in a statement context, and lambda
is only used in an expression context. Why not use the same keyword for
both?

Because the syntax is completely different. One is a statement, and
stands alone, the other is an expression. Even putting aside the fact
that lambda's body is an expression, and a def's body is a block, def
also requires a name. Using the same keyword for both would require
special case reasoning: sometimes def is followed by a name, sometimes
without a name. That's ugly.

def name(args): block # okay

funcs = [def args: expr, # okay so far
def name(args): expr, # not okay
def: expr, # okay
]

def: expr # also okay

def: expr
expr # but not okay

x = def x: expr # okay
x = def x(x): expr # not okay

Using the same keyword for named and unnamed functions is, in my opinion,
one of those foolish consistencies we are warned about. When deciding on
syntax, the difference between anonymous and named functions are more
significant than their similarities.

I think the answer is historical: def came first, and when
anonymous functions were added it didn't make sense to use the keyword
"def" for them, because "def" implies a name being defined.

That reasoning still applies even if they were added simultaneously.

Lambda is pretty old: it certainly exists in Python 1.5 and almost
certainly in 1.4. While it doesn't exist as a keyword in Python 0.9.1,
there is a module called "lambda" with a function "lambda" that uses more
or less the same syntax. Instead of lambda x: x+1, you would instead
write lambda("x", "x+1"). So the idea of including anonymous functions
was around in Python circles before the syntax was locked in.

 

[Chris Angelico]

Because the syntax is completely different. One is a statement, and
stands alone, the other is an expression. Even putting aside the fact
that lambda's body is an expression, and a def's body is a block, def
also requires a name. Using the same keyword for both would require
special case reasoning: sometimes def is followed by a name, sometimes
without a name. That's ugly.

All you need to do is define that a block of code is an object (and
thus suitable material for an expression), and you have easy
commonality.

fn = def(args):
block
of
code

Now def is an expression that takes an optional name (omitted in the
above), an arg list, and a block of code... and there's minimal
difference between named and anonymous functions. (If it's given a
name, then it sets __name__ and also binds to that name, being
convenient for the common case. The above code is a silly way to do
almost the default.)

ChrisA

 

[Steven D'Aprano]

Exception handling is one of those subjects few understand and fewer can
implement properly in modern code. Languages that don't support
exceptions as part of their signature lead to capturing generic
Exception all throughout code. It is one of those features I wish .NET
had. At the same time, with my limited experience with Java, it has been
a massive annoyance. Perhaps something to provide or just shut off via a
command line parameter. What indications have there been that this has
been a flaw? I can see it alienating a large group of up- and-coming
developers.

http://www.ibm.com/developerworks/java/library/j-jtp05254/index.html

Note also that Bruce Eckel repeats a rumour that checked exceptions were
*literally* an experiment snuck into the Java language while James
Gosling was away on holiday.

http://www.mindview.net/Etc/Discussions/UnCheckedExceptionComments

Even if that is not true, checked exceptions are a feature that *in
practice* seems to lead to poor exception handling and cruft needed only
to satisfy the compiler:

http://www.alittlemadness.com/2008/03/12/checked-exceptions-failed-experiment/#comment-219143

and other annoyances. It's main appeal, it seems to me, is to give a
false sense of security to Java developers who fail to realise that under
certain circumstances Java will raise certain checked exceptions *even if
they are not declared*. E.g. null pointer exceptions.

See also:

http://java.dzone.com/articles/checked-exceptions-i-love-you

and note especially the comment from the coder who says that he simply
declares his functions to throw Exception (the most generic checked
exception), thus defeating the whole point of checked exceptions.

 

[DevPlayer]

Tabs have every theoretical advantage and only one practical
disadvantage: the common toolsets used by Unix programmers are crap in
their handling of tabs, and instead of fixing the toolsets, they blame
the tabs.

The use of spaces as indentation is a clear case of a technically worse
solution winning over a better solution.


Why on earth should your language dictate the width of an indentation? I
can understand that you might care that indents are consistent within a
single source code unit (a file?), but anything more than that is just
obnoxious.

I do not understand why the interpreter preprocesses each logical line
of source code using something as simple as this:

def reindent( line, spaces_per_tab=os.spaces_per_tab):

index = 0 # index into line of text
indent = 0 # increase 1 when in next tab column
spaces = 0 # index into current column

for c in line:
if c == ' ':
spaces +=1
if spaces >= spaces_per_tab:
indent += 1
spaces = 0
if c == '\t': # jump to next tab column
indent += 1
spaces = 0
if c <> ' ' and c <> '\t':
break
index += 1
rest_of_line = line[index:]
new_indent = ' ' * indent * spaces_per_tab + ' ' * spaces
newline = new_indent + rest_of_line

return newline

or even some regex equivelent.

That function would need to be run on each line of source code, after
processing the line continuation character and single/double/triple
quote pairs are processed but before the code block tokenizer (INDENT/
DEDENT) if possible. Unless some of you expert parser/compiler writers
know fancier tricks.

To me, I would think the interpreter finding the coder's intended
indent wouldn't be that hard. And just make the need for consistant
spaces or tabs irrevelent simply by reformatting the indent as
expected. Pretty much all my text editors can.

 

[DevPlayer]

I do not understand why the interpreter preprocesses each logical line

of source code using something as simple as this:

correction:

I do not understand why the interpreter - does not- preprocess each
logical line
of source code using something as simple as this:

 

[Chris Angelico]

To me, I would think the interpreter finding the coder's intended
indent wouldn't be that hard. And just make the need for consistant
spaces or tabs irrevelent simply by reformatting the indent as
expected. Pretty much all my text editors can.

The trouble with having a language declaration that "a tab is
equivalent to X spaces" is that there's no consensus as to what X
should be. Historically X has always been 8, and quite a few programs
still assume this. I personally like 4. Some keep things narrow with
2. You can even go 1 - a strict substitution of \t with \x20. Once you
declare it in your language, you immediately break everyone who uses
anything different.

ChrisA

 

[Travis Parks]

On Sun, Nov 27, 2011 at 4:55 PM, Steven D'Aprano

I think the implication is that Unit has only one syntax for creating
functions, which is lambda-style.  In any case, why does Python require
a special keyword?  def is only used in a statement context, and lambda
is only used in an expression context.  Why not use the same keyword for
both?

Because the syntax is completely different. One is a statement, and
stands alone, the other is an expression. Even putting aside the fact
that lambda's body is an expression, and a def's body is a block, def
also requires a name. Using the same keyword for both would require
special case reasoning: sometimes def is followed by a name, sometimes
without a name. That's ugly.

def name(args): block  # okay

funcs = [def args: expr,  # okay so far
         def name(args): expr,  # not okay
         def: expr,  # okay
         ]

def: expr  # also okay

def: expr
    expr  # but not okay

x = def x: expr  # okay
x = def x(x): expr  # not okay

Using the same keyword for named and unnamed functions is, in my opinion,
one of those foolish consistencies we are warned about. When deciding on
syntax, the difference between anonymous and named functions are more
significant than their similarities.

A good example I have run into is recursion. When a local function
calls itself, the name of the function may not be part of scope (non-
local). Languages that support tail-end recursion optimization can't
optimize. In order to support this, a function in Unit will have
access to its own name and type. In other words, special scoping rules
are in place in Unit to allow treating a function as an expression.

That reasoning still applies even if they were added simultaneously.

Lambda is pretty old: it certainly exists in Python 1.5 and almost
certainly in 1.4. While it doesn't exist as a keyword in Python 0.9.1,
there is a module called "lambda" with a function "lambda" that uses more
or less the same syntax. Instead of lambda x: x+1, you would instead
write lambda("x", "x+1"). So the idea of including anonymous functions
was around in Python circles before the syntax was locked in.

I find that interesting. I also find it interesting that the common
functional methods (all, any, map, filter) are basically built into
Python core language. That is unusual for most imperative programming
languages early-on.

 

[Chris Angelico]

A good example I have run into is recursion. When a local function
calls itself, the name of the function may not be part of scope (non-
local). Languages that support tail-end recursion optimization can't
optimize. In order to support this, a function in Unit will have
access to its own name and type. In other words, special scoping rules
are in place in Unit to allow treating a function as an expression.

I'm inclined toward an alternative: explicit recursion. Either a
different syntax, or a special-case on the use of the function's own
name, but whichever syntax you use, it compiles in a "recurse" opcode.
That way, if name bindings change, it's still going to recurse -
something few languages guarantee, and therefore few languages can
optimize.

ChrisA

 

[Travis Parks]

http://www.ibm.com/developerworks/java/library/j-jtp05254/index.html

Note also that Bruce Eckel repeats a rumour that checked exceptions were
*literally* an experiment snuck into the Java language while James
Gosling was away on holiday.

http://www.mindview.net/Etc/Discussions/UnCheckedExceptionComments

Even if that is not true, checked exceptions are a feature that *in
practice* seems to lead to poor exception handling and cruft needed only
to satisfy the compiler:

http://www.alittlemadness.com/2008/03/12/checked-exceptions-failed-ex...

and other annoyances. It's main appeal, it seems to me, is to give a
false sense of security to Java developers who fail to realise that under
certain circumstances Java will raise certain checked exceptions *even if
they are not declared*. E.g. null pointer exceptions.

See also:

http://java.dzone.com/articles/checked-exceptions-i-love-you

and note especially the comment from the coder who says that he simply
declares his functions to throw Exception (the most generic checked
exception), thus defeating the whole point of checked exceptions.

I think all of the examples you listed referred specifically to most
programmers finding ways around the annoyance. I have heard about
throwing generic Exception or inheriting all custom exception types
from RuntimeException. I did this quite often myself.

In general, unchecked exceptions shouldn't be caught. They occur
because of bad code and insufficient testing. Checked exceptions occur
because of downed databases, missing files, network problems - things
that may become available later without code changes.

One day, I went through about 10,000 lines of code and moved argument
checking code outside of try blocks because I realized I was handling
some of them by accident. Here is the program: if me == idiot: exit().

People don't think about this, but the exceptions thrown by a module
are part of that module's interface. Being able to make sure only what
you expect to come out is important. Unlike Java, Unit requires you to
opt in to using throws clauses. If you don't list one, one is
generated for you automatically. The benefit: you can see what a
function throws and protect yourself without all the babysitting.

A lack of exception handling is big problem in .NET. I have had
libraries from big names including Novell and Oracle throw
NullReferenceExceptions because _they_ didn't know what would happen
in cases where a DLL is missing or a dependency isn't installed. They
could have done better testing, but if the biggest names in
development can't manage to figure it, I say leave it up to the
compiler. Returning nulls or special value in cases of failures takes
us back to the days of C and Fortran.

 

[Travis Parks]

The trouble with having a language declaration that "a tab is
equivalent to X spaces" is that there's no consensus as to what X
should be. Historically X has always been 8, and quite a few programs
still assume this. I personally like 4. Some keep things narrow with
2. You can even go 1 - a strict substitution of \t with \x20. Once you
declare it in your language, you immediately break everyone who uses
anything different.

ChrisA

Yeah. We must remember the Unix users, espcially those who don't know
how to hack Vim or bash. I've decided not to require a specific number
of spaces. I am still teetering on whether to allow tabs.

 

[Ian Kelly]

I find that interesting. I also find it interesting that the common
functional methods (all, any, map, filter) are basically built into
Python core language. That is unusual for most imperative programming
languages early-on.

all and any are actually quite recent additions. Guido added them to
placate users of "reduce(lambda x, y: x and y, foo)" back when the
plan was to remove reduce in Python 3.

Cheers,
Ian

 

[Steven D'Aprano]

The trouble with having a language declaration that "a tab is equivalent
to X spaces" is that there's no consensus as to what X should be.
Historically X has always been 8, and quite a few programs still assume
this. I personally like 4. Some keep things narrow with 2. You can even
go 1 - a strict substitution of \t with \x20. Once you declare it in
your language, you immediately break everyone who uses anything
different.

Why should we enforce how many spaces a tab is set to? That is literally
only visible to the editor, not the compiler. (Unless the parser is
particularly stupid and merely substitutes X spaces for a tab every time
it sees one.)

Mixed spaces and tabs in an indent are ambiguous, and so raises
SyntaxError (or at least it *should* raise SyntaxError). But separate
code blocks can use different absolute indent levels without ambiguity,
so long as the relative indentation is consistent:

def ham(): # tab stops at 4 and 8
do(a)
do(b)
if c:
do(c)
else:
do(d)


def spam(): # tab stops at 8 and 12
do(a)
do(b)
if c:
do(c)
else:
do(d)

There is no meaningful difference in indentation between ham and spam
above. In either case, I could replace spaces with tabs to get the same
relative indents regardless of the absolute indentation.

I can appreciate the aesthetic argument that *within a single file*,
indentation should be consistent. But it's not logically necessary for
the parser: it need only be consistent within a single code unit
(function or class usually). In other words: syntax should only care
about relative indentation, absolute indentation belongs as a coding
standard.