Closures in leu of pointers?

[Chris Angelico]

On a related note, I think that generator functions should in some way
be explicitly marked as such in the declaration, rather than needing
to scan the entire function body for a yield statement to determine
whether it's a generator or not.

Most functions are:

def func(args):
body
return result

Generators are:

class func(args): # okay, you can't shortcut it like that, give it an
__init__ method
def do_stuff(self):
body
yield results one by one

I don't know that anything would be gained by having a different
function declaration statement/attribute, but maybe this is something
that would benefit from a code comment (which, as far as I'm
concerned, is as much a part of the function signature as the coded
parts are).

ChrisA

 

[Terry Reedy]

On a related note, I think that generator functions should in some way
be explicitly marked as such in the declaration, rather than needing
to scan the entire function body for a yield statement to determine
whether it's a generator or not.

I agree that one should not have to scan. The doc string, which should
be present should start 'Return a generator that yields ...' or even
'Generate ...'. Of course, then non-generator functions should not start
the same way. The first option should be non-ambiguous.

 

[Ian Kelly]

I agree that one should not have to scan. The doc string, which should be
present should start 'Return a generator that yields ...' or even 'Generate
...'. Of course, then non-generator functions should not start the same way.
The first option should be non-ambiguous.

I don't deny that properly written comments help, but consider that
Python already enforces proper indentation for the sake of
readability. I don't think it would be a great harm if the syntax
enforced easily readable generator

 

[rusi]

This is strictly a matter of opinion, and one on which mine differs
from yours. I think explicit declarations are better than implicit
"you've assigned to this name" local creations; the declarations help
to catch typos. Also, I like the consistency of C-style declarations -
where-ever you declare something, it's valid from there "in", and not
"out". Without declarations, there's a magical scope boundary at a
function definition that's different from the non-boundary at a loop,
for instance.


But that's just my opinion. and as Teresa says, I'm only one, and
possibly I'm wrong.

Getting scoping right has been -- history-wise -- exceptionally difficult.

Lisp started with dynamic scoping -- the word and the consequences of that hardly being conceivable in 1960. It took 25 years to correct half that error in scheme and common-lisp viz. dynamic to static scoping. The other half still persists in common lisp though not scheme
http://en.wikipedia.org/wiki/Lisp-1_vs._Lisp-2#The_function_namespace.

Its taken another 25 years to get the mistake out of the most ubiquitous lisp -- Emacs-lisp
https://en.wikipedia.org/wiki/Emacs_Lisp#From_dynamic_to_lexical_scoping

I believe perl started with 'local' and later added 'my', repeating the same mistake.

Even python needed to modify the LEGB rule (was it 2.2??) for similar reasons,
got comprehension scoping wrong by having variables leak out -- corrected from 2 -> 3, and still gets it wrong from one element to the next.

And finally, the founders of lambda-calculus -- Church, Curry, Rosser -- screwed up (I believe) in the initial definitions of free and bound variables..

 

[rusi]

If, in the general case, the compiler requires two passes to understand
a function body, then *so do people*#. This requirement is what trips up
people who are either not used to the idea of two-pass compilation or do
not cognize that the language requires it.

# The alternative for either program or people is a 1-pass +
backtracking process where all understandings are kept provisional until
the end of the body and revised as required. 2 passes are simpler.

Well languages like C have half a dozen such passes!

They are called, preprocessing, compiling, assembling, linking, dynamic linking etc.

And then we have that a variable at one stage becomes a constant at another, eg a macro.

All of which adds up to making scoping/variables an arcane craft.

Now having such passes is one thing. Defining the language in terms of them quite another...

(Python) is compiled, and the compilation of functions is a two pass
affair. The first pass classifies all names. The second pass generates
code based on the classification of the first pass. This is not part of
the grammar, but implied (required) by the semantic description.

So if understanding a language in terms of machine states -- a so called operational definition -- is unsatisfactory, understanding it in terms of the machine-states of the implementation is doubly so -- we may call it a meta-operational definition.

Now, thanks to Joel Spolsky we know that abstractions in general will leak including our primary abstractions viz programming languages. So, no completely banning operational understanding is generally impossible.

However when it is avoidable it should be avoided.

I documented the hell that teachers have teaching operational/meta-operational langauges like C 25 years ago http://www.the-magus.in/Publications/chor.pdf
And I would wish that python at least tries not to repeat C's mistakes!

 

[Antoon Pardon]

Op 29-06-13 21:23, Ian Kelly schreef:

Perhaps because that is the terminology used by the language documentation:

http://docs.python.org/3/reference/executionmodel.html#naming-and-binding

I don't think this reference is as strong as you think it is. Here is
a paragraph somewhat lower:

] If a name is bound in a block, it is a local variable of that block,
] unless declared as nonlocal. If a name is bound at the module level, ]
it is a global variable. (The variables of the module code block are ]
local and global.) If a variable is used in a code block but not
] defined there, it is a free variable.

So the language documentation mentions these names as being variables.

 

[Steven D'Aprano]

On Sun, 30 Jun 2013 01:56:25 -0700, rusi wrote:

[...]

All of which adds up to making scoping/variables an arcane craft.

Now having such passes is one thing. Defining the language in terms of
them quite another...

I don't believe that Python's behaviour is defined in terms of the number
of passes. It is defined in terms of a condition "if you assign to a
name, it is local unless declared global". Parsing the function twice
just happens to be the simplest way for the compiler to implement that
rule, but it's not strictly necessary. A skilled human reader, for
example, may read the function once, and re-interprete what's already
seen on the basis of what they've just seen. For a human reader, such
backtracking is probably easier than a two-pass process explicitly
memorizing which variables are local and which are global.

 

[rusi]

I don't believe that Python's behaviour is defined in terms of the number
of passes. It is defined in terms of a condition "if you assign to a

Yeah sure, I dont believe that python is nearly as bad as C where for example one commonly has to run the preprocessor standalone to debug f---ed up macros.

I was responding to Terry's

This is not part of the grammar, but implied (required) by the semantic
description.

Now in the case of C one would not be able to give a non-operational semantics even with a battalion of denotational semantics PhDs. Python is clearlynot so bad, though in some cases the praxis of a certain aspect may be better dealt with operationally than the language spec even if that is non-operational.

A clear case of this is that of mutable default parameters. Understanding that the def statement is an imperative statement in python and the consequences of that understanding, solves that issue. However the cost implicit in this is that the semantics has been operationalized.

 

[Ian Kelly]

I don't think this reference is as strong as you think it is. Here is
a paragraph somewhat lower:

] If a name is bound in a block, it is a local variable of that block,
] unless declared as nonlocal. If a name is bound at the module level, ] it
is a global variable. (The variables of the module code block are ] local
and global.) If a variable is used in a code block but not
] defined there, it is a free variable.

So the language documentation mentions these names as being variables.

It seems to refer to "local" and "global" variables as a short hand
for talking about specific types of name binding, which is the
dominant nomenclature of the documentation. You asked why people talk
about Python binding names instead of assigning variables, and I think
the reference material is a clear source for that, even if it does
also use the word "variable". There is also the section on assignment
statements, where it again refers to names being bound, not variables
being assigned:

http://docs.python.org/3/reference/simple_stmts.html#assignment-statements

 

[Ian Kelly]

I don't think this reference is as strong as you think it is. Here is
a paragraph somewhat lower:

] If a name is bound in a block, it is a local variable of that block,
] unless declared as nonlocal. If a name is bound at the module level, ] it
is a global variable. (The variables of the module code block are ] local
and global.) If a variable is used in a code block but not
] defined there, it is a free variable.

So the language documentation mentions these names as being variables.

It seems to refer to "local" and "global" variables as a short hand
for talking about specific types of name binding, which is the
dominant nomenclature of the documentation. You asked why people talk
about Python binding names instead of assigning variables, and I think
the reference material is a clear source for that, even if it does
also use the word "variable". There is also the section on assignment
statements, where it again refers to names being bound, not variables
being assigned:

http://docs.python.org/3/reference/simple_stmts.html#assignment-statements

Actually, looking back at your earlier post, I think I misunderstood
your question. You weren't asking "why do people talk about binding
names instead of assigning variables in Python", but more strongly
"why do people claim there are no variables in Python"? If the latter
is a common claim, then I haven't really taken notice of it, but I'll
take your word that you have. I don't agree with that -- saying that
Python has no variables is just an argument of semantics.

 

[alex23]

In general I agree, although when reading code I would definitely
prefer if the locals were declared.

If you import the code into the interpreter as an adjunct to reading it
you can see the locals with:

On a related note, I think that generator functions should in some way
be explicitly marked as such in the declaration, rather than needing
to scan the entire function body for a yield statement to determine
whether it's a generator or not.

However, a dedicated code reader built around the inspect module could
be a handy tool.

 

[Steven D'Aprano]

On a related note, I think that generator functions should in some way
be explicitly marked as such in the declaration, rather than needing to
scan the entire function body for a yield statement to determine whether
it's a generator or not.

That was considered when generators were introduced in Python 2.2.
Guido's rationale for preferring to keep "def" for both generator
functions and normal functions is given in the PEP:


Issue: Introduce another new keyword (say, "gen" or "generator") in
place of "def", or otherwise alter the syntax, to distinguish
generator-functions from non-generator functions.

Con: In practice (how you think about them), generators *are*
functions, but with the twist that they're resumable. The mechanics
of how they're set up is a comparatively minor technical issue, and
introducing a new keyword would unhelpfully overemphasize the
mechanics of how generators get started (a vital but tiny part of a
generator's life).

Pro: In reality (how you think about them), generator-functions are
actually factory functions that produce generator-iterators as if by
magic. In this respect they're radically different from non-generator
functions, acting more like a constructor than a function, so reusing
"def" is at best confusing. A "yield" statement buried in the body is
not enough warning that the semantics are so different.

BDFL: "def" it stays. No argument on either side is totally
convincing, so I have consulted my language designer's intuition. It
tells me that the syntax proposed in the PEP is exactly right - not
too hot, not too cold. But, like the Oracle at Delphi in Greek
mythology, it doesn't tell me why, so I don't have a rebuttal for the
arguments against the PEP syntax. The best I can come up with (apart
from agreeing with the rebuttals ... already made) is "FUD". If this
had been part of the language from day one, I very much doubt it
would have made Andrew Kuchling's "Python Warts" page.


http://www.python.org/dev/peps/pep-0255/


5+ versions later, I think that Guido has been shown to be correct. Even
if you believe that generator functions would have been better with
different syntax, there is no evidence that re-using def is actively
harmful.