elements of decorator syntax suggestions

[Steven Bethard]

I think one of the biggest reasons we're having such problems coming
to any agreement on decorator syntax is that each proposal makes a
number of syntax decisions, not just one. For decorators, I see the
following decisions that must be made:

1) Indicator
Proposals differ on how some sort of indicator of "decoratorhood" is
use. These include:
* none (e.g. just the list, as in the "list-after-def" suggestion)
* the '@' character
* a new keyword (with, using, decorate, etc.)

2) Location
Proposals also differ on exactly where the decorator declaration
should be made. These include:
* before def
* between def and function name
* between function name and argument list
* between argument list and colon
* after colon (first line in body)

3) List notation
As I understand it, it's already decided that any implementation of
decorators must allow for a list of decorators to be applied. So
proposals that might be accepted must explain how this can be done.
The options I've seen:
* one per line (as currently)
* commas only
* current list syntax (or some similar variant)

4) Indentation
Proposals also differ on whether or not decorators should introduce a
new block. What I've seen:
* no block
* block

I thought the summary above might put some things into perspective so
that when you're arguing for one syntax or another, you can argue
separately for the points that matter most to you. It might be that
we mostly agree on, say, the need for a keyword, or the block
indentation (though I'm certainly not holding my breath). But I'd
rather hear arguments for the different sections separately because,
to some degree, they're pretty much orthogonal.


In case you were just asking yourself, "well, gee, Steve, what do you
think?", =) here's what I think for each of the points above:

1) Indicator
I want a new keyword. A list alone (no matter where it's placed)
doesn't look any more like a decorator to me than '@' does. I want
some sort of cue that reads more clearly. I'm not picky on what this
keyword is -- 'decorate' seemed like a reasonable proposal, but I'd
even be happy with 'with' or 'using' -- but I don't just want
arbitrary symbols. I think this is most important from the
perspective of new Python users, who will have to guess the meaning of
such syntax from someone else's code. Having a keyword that makes the
usage clear makes this reading much easier.

2) Location
I don't like "between def and function name" or "between function name
and argument list" because I think they make it harder to see the
basic signature of the function. Remember that we have to support
lists, so just sticking a in single word, like staticmethod or
classmethod, isn't an option -- otherwise, I could have perhaps seen
the merit of "between def and function name".

I'm not a big fan of "after colon" (like a doc-string) because I'm
more likely to read this as part of the function definition. (In
fact, that's what I'd do now, so this is a semantic change, one of the
things I'd rather avoid.)

As far as "before def" or "betwen argument list and colon" go (and I
believe these are the major contenders), I could go either way -- and
I'm really not particular here; I'd rather have /some/ decorator
syntax than argue over these forever. "before def" seems to deal
better with functions that have long names or argument lists (Guido's
example convinced me that I'd be confused where arguments ended and
decorators started), but "between argument list and colon" avoids the
two-lines-define-a-single-function problem. Eh... Like I say, I'm
not going to argue this one -- I could be happy with either.

3) List notation
I don't really have a preference here. I can see that commas only
might be a pain if you need to use multiple lines perhaps. But "one
per line" or "list syntax" would both be fine with me.

4) Indentation
I'd prefer that decorators didn't start a new block mainly because
that makes, for example, an instance method with an 'accepts'
decorator on a different block level than an instance method without
one. Still, I could live with indented decorators.


So in summary
-> yay keywords!
-> before def or before colon, and
-> whatever list notation and indentation floats your boat

=)

Steve

 

[Peter Hansen]

So in summary
-> yay keywords!
-> before def or before colon, and
-> whatever list notation and indentation floats your boat

Good post, Steven, and I agree with all your own opinions on the
matter as expressed above.

Note one addition item that most of us are glossing over in the
debate for/against the pie (@) syntax. Order of operation.

@ syntax has reversed order from the list-after-def. I think
the order should be readily apparent from the syntax, and I
think the list syntax does provide that benefit over the @ syntax.

I think I'd like a "decorate" keyword (or equivalent), *before*
the definition to avoid the problems with obscuration, and a
mandatory list of some kind that clearly shows the order.

Something like, but not necessarily exactly the same as, this:

from __future__ import decoration

decorate [
foo,
bar,
bletch(spam='baz')
]
def func(): ...

The only problem is that it still has the defect of @ wherein
the decorate doesn't seem to "bind" to the function def very
well, especially if blank lines are allowed etc.

I'm unclear on what I think it best... but then I don't mind
the current approach much at all, where you just stick the
stuff after the entire function block, so who cares about my
opinion? ;-)

-Peter

 

[Anthony Baxter]

This is an excellent list! Thanks for doing it. I've made a couple of notes
inline about the particular points.

I think one of the biggest reasons we're having such problems coming
to any agreement on decorator syntax is that each proposal makes a
number of syntax decisions, not just one. For decorators, I see the
following decisions that must be made:

1) Indicator
Proposals differ on how some sort of indicator of "decoratorhood" is
use. These include:
* none (e.g. just the list, as in the "list-after-def" suggestion)
* the '@' character
* a new keyword (with, using, decorate, etc.)

This is the biggy, it seems. Current (as of a couple of hours ago)
discussions on python-dev are discussing other alternatives instead
of @, that will hopefully make it easier for IPython or Leo to cope
for now (but note that in the future, some other use for @ might be
found, so anyone relying on it at the moment might want to think
about that). One current suggestion is to use the | character, instead.

2) Location
Proposals also differ on exactly where the decorator declaration
should be made. These include:
* before def

Short of someone producing a _very_ strong argument in favour of
another form, this is almost certainly going to be the choice.

* between def and function name

Breaks too many tools, as well as stupid humans who are used to seeing
def functionname.

* between function name and argument list

I'm not sure that this was ever popular, with anyone

* between argument list and colon

Guido's ruled this out (see previous posts, I put a link to his post
outlining why.

* after colon (first line in body)

A problem here is interaction with docstrings?

3) List notation
As I understand it, it's already decided that any implementation of
decorators must allow for a list of decorators to be applied. So
proposals that might be accepted must explain how this can be done.
The options I've seen:
* one per line (as currently)

There's some confusion as to how the one-per-line thing is ordered. I find
it quite obvious, but I've been playing with this for a week now, so it might
just be that I know the answer now. Simply -

@decA
@decB
@decC
def func():

is equivalent to

func = decA(decB(decC(func)))

* commas only

Too much ambiguity in the parsing, I suspect.

* current list syntax (or some similar variant)

4) Indentation
Proposals also differ on whether or not decorators should introduce a
new block. What I've seen:
* no block
* block

Guido pointed out that this would lead to the default indentation level
for a decorated method being 3 tab stops. I also don't like it visually -
I tried reformatting test_decorators using it, and it looked... strange.

I thought the summary above might put some things into perspective so
that when you're arguing for one syntax or another, you can argue
separately for the points that matter most to you. It might be that
we mostly agree on, say, the need for a keyword, or the block
indentation (though I'm certainly not holding my breath). But I'd
rather hear arguments for the different sections separately because,
to some degree, they're pretty much orthogonal.

An excellent list! If you don't mind, I might steal this format for the PEP.
It allows for a lot more alternatives to be covered off in a smaller space
(since many proposals are minor variations on an existing proposal, and
share the same problems).

 

[Steven Bethard]

An excellent list! If you don't mind, I might steal this format for the PEP.
It allows for a lot more alternatives to be covered off in a smaller space
(since many proposals are minor variations on an existing proposal, and
share the same problems).

Please feel free to -- glad it was helpful. =) Thanks so much for all
your work already!

Steve

 

[John Marshall]

I think one of the biggest reasons we're having such problems coming
to any agreement on decorator syntax is that each proposal makes a
number of syntax decisions, not just one. For decorators, I see the
following decisions that must be made:

1) Indicator

2) Location

3) List notation

4) Indentation

I think you have done a good job identifying and
isolating what the issues are regarding decorator
options. I think another thing that should perhaps
be made clearer (maybe for me only?) is to identify,
in order of importance/signfigance, the relationship
between decorators and the function:
1) is the function/method signature more important
than the decorators (to the programmer)?
2) do the decorators belong or are affected by the function?
3) what other constructs in Python say/imply "look ahead
for what I am about to affect/define"?

I understand that designing programming languages is
sometimes an art rather than a science (natural languages
a chock full of exceptions to the rule). But if the
priority of functions and decorators are established,
then I think this would help.

In response to my own questions above:
1) From my perspective, it seems that almost everybody
would say that the function (class also) signature is
more important than decorators. For example, how many
people would say "I wonder where the classmethod
function is?" Instead, people would say, "Where is my
function xyz()". This seems to argue for the decorators
following (somewhere) after the signature.

Of course highlighting editors can help by marking off
function names and parameters, but this argument only
serves to confirm that the signature is the KEY thing
programmers are interested in. Apart from aesthetics,
which are subjective and could be debated endlessly, I
cannot see how one could argue otherwise.

2) Perhaps I am not understanding fully. Can someone explain
how decorators before the function logically belong outside
of the function definition block if they are intended to
affect the function? Are the decorators meant to affect
something other than the function? It seems to me that
in every other situation in Python (please correct me if I
am wrong) blocks are the way we are able to identify what
belongs to what (e.g., local variable to a function).
The current decorator approach says to me, "I belong to
the class" (if a class is involved), or "I belong to the
module" (if no class is involved).

3) In a sense, this point is somewhat related to #2. Does
Python support an implicit "I am doing something now, on
something I haven't defined, but which is coming up next"
anywhere else than the decorator before function
construction? Even in classes, Python forces the use of
self.xxx (explicit).

Hope this contributes to the discussion.

John

 

[Jeff Shannon]

There's some confusion as to how the one-per-line thing is ordered. I find
it quite obvious, but I've been playing with this for a week now, so it might
just be that I know the answer now. Simply -

@decA
@decB
@decC
def func():

is equivalent to

func = decA(decB(decC(func)))

So one way of looking at this, then, is that the @ is (very loosely)
semantically equivalent to an opening paren, with the closing paren
being implied?

This makes sense if one thinks of decorators as a wrapper function that
contains the true function. It doesn't make quite so much sense when
one is talking about decorators as meta_data_ which is attached to the
function. Clearly, decorators *can* be used in either sense, and
equally clearly people who want metadata _will_ use decorators for that
purpose. But I suspect that at least part of the objection to the
current syntax is that it doesn't fit with the model that would be
expected for metadata, for which use-case function attributes seem a
better fit. There's a fundamental difference between the decorator
being a (possibly recursive) container for a function, and functions
being a container for a set of associated data (which might be other
callables), and that distinction hasn't been explicitly addressed so far
as I've seen. (But note that I've only been reading discussions here,
and haven't followed up on the py-dev archives.)

(I *still* don't like the two-or-more-lines-per-def, but it does make
decorators-before-def seem more reasonable.)

Jeff Shannon
Technician/Programmer
Credit International

 

[Bengt Richter]

This is an excellent list! Thanks for doing it. I've made a couple of notes
inline about the particular points.

+1 on the info format of the post
what about a builtin name? or does that count as a keyword?

This is the biggy, it seems. Current (as of a couple of hours ago)
discussions on python-dev are discussing other alternatives instead
of @, that will hopefully make it easier for IPython or Leo to cope
for now (but note that in the future, some other use for @ might be
found, so anyone relying on it at the moment might want to think
about that). One current suggestion is to use the | character, instead.


Short of someone producing a _very_ strong argument in favour of
another form, this is almost certainly going to be the choice.


Breaks too many tools, as well as stupid humans who are used to seeing
def functionname.


I'm not sure that this was ever popular, with anyone


Guido's ruled this out (see previous posts, I put a link to his post
outlining why.


A problem here is interaction with docstrings?



There's some confusion as to how the one-per-line thing is ordered. I find
it quite obvious, but I've been playing with this for a week now, so it might
just be that I know the answer now. Simply -

@decA
@decB
@decC
def func():

is equivalent to

func = decA(decB(decC(func)))

Looks to me like the semantics is

__magic_internal_list.append(decA)
__magic_internal_list.append(decB)
__magic_internal_list.append(decC)
def func(): pass
while _magic_internal_list: func = __magic_internal_list.pop()(func)

why not go with a little sugar like
__builtins__.decorate = __magic_internal_list.append

and then
decorate(decA)
decorate(decB)
decorate(decC)
def func(): pass
# (automatic trigger of the while statement above, after the def)

Of course, I think more interesting things can be done along with that bare minimum,
but even this minimal implementation at least avoids such a narrow commitment for '@'.
BTW, even if __magic_internal_list were per-thread, wouldn't one need guidelines at
least for writing wrappers safely? What does current @decorator do?

An excellent list! If you don't mind, I might steal this format for the PEP.
It allows for a lot more alternatives to be covered off in a smaller space
(since many proposals are minor variations on an existing proposal, and
share the same problems).

+1 on that ;-)

Regards,
Bengt Richter

 

[Michele Simionato]

This is the biggy, it seems. Current (as of a couple of hours ago)
discussions on python-dev are discussing other alternatives instead
of @, that will hopefully make it easier for IPython or Leo to cope
for now (but note that in the future, some other use for @ might be
found, so anyone relying on it at the moment might want to think
about that). One current suggestion is to use the | character, instead.

Aha! These are good news!!
I like the current proposal better than the list syntax (too many parens
and commas) *except* for the "@". The "@" is terrible! But fortunately
we have plenty of alternative punctuation to choose upon: the colon,
the dot, the "|", even the caret, the tilde, etc. All stuff which is
already in current Python. At the moment I like this:

- sincronyzed
- classmethod
def f(cls, *args):
pass

Michele Simionato

 

[Michele Simionato]

This is the biggy, it seems. Current (as of a couple of hours ago)
discussions on python-dev are discussing other alternatives instead
of @, that will hopefully make it easier for IPython or Leo to cope
for now (but note that in the future, some other use for @ might be
found, so anyone relying on it at the moment might want to think
about that). One current suggestion is to use the | character, instead.

What about "-" ?


- syncronized
- classmethod
def f(cls, *args):
pass


Any punctuation already used in current Python
would go for me (., :, -, +, *, /, |, \, ^, etc. etc.) !



Michele Simionato

 

[Tim Peters]

[Michele Simionato]

What about "-" ?

- syncronized

That's already (pre-2.4) valid Python, so could change the meaning of
existing code -- won't happen.

- classmethod
def f(cls, *args):
pass


Any punctuation already used in current Python
would go for me (., :, -, +, *, /, |, \, ^, etc. etc.) !

Overloading a unary prefix operator is out. As Anthony said, vertical
bar seemed to be the leading contender on python-dev Friday. You can
find tedious arguments there about most of the others that would go
for you.

 

[Erik Max Francis]

Overloading a unary prefix operator is out. As Anthony said, vertical
bar seemed to be the leading contender on python-dev Friday. You can
find tedious arguments there about most of the others that would go
for you.

Why is | superior to @? | already has a meaning in Python, and it has
nothing to do with decorators ... at least @ has the virtue of currently
being unused in the language, and having the precedent of being used for
a similar feature in Java.

 

[Jarek Zgoda]

Why is | superior to @? | already has a meaning in Python, and it has
nothing to do with decorators ... at least @ has the virtue of currently
being unused in the language, and having the precedent of being used for
a similar feature in Java.

Sure, use all characters, that are still not used. Fine. I propose "^".
If this fails, please consider "~" and "`".

Oh, and ";" is still not used in Python, what about some special meaning
for that one?

 

[Michele Simionato]

Why is | superior to @? | already has a meaning in Python, and it has
nothing to do with decorators ... at least @ has the virtue of currently
being unused in the language, and having the precedent of being used for
a similar feature in Java.

1. @ is already used in Leo and ipython, "|" is not.
2. "|" stands visually more than other characters.
3. Many people have an irrational repulsion for "@" and "$".


Michele Simionato

 

[Dan Bishop]

Sure, use all characters, that are still not used. Fine. I propose "^".

Already used (bitwise exclusive OR).

If this fails, please consider "~"

Already used (bitwise inversion).

and "`".

Already used (`x` is a shorthand for repr(x))

Oh, and ";" is still not used in Python,
what about some special meaning for that one?

Already used (statement separator).

The only ASCII characters that aren't used in Python are '@', '$', and '?'.

 

[Erik Max Francis]

1. @ is already used in Leo and ipython, "|" is not.

It's also used in EmPy, but you don't see me complaining.

2. "|" stands visually more than other characters.

@ sure takes up more visual space and is easier to spot in text than |,
which is precisely why I used it in EmPy as the token prefix (also
because @ at the time is neither commonly used in English text or legal
in Python, though the latter will be changing).

3. Many people have an irrational repulsion for "@" and "$".

Evidently!

 

[Anthony Baxter]

@ sure takes up more visual space and is easier to spot in text than |,
which is precisely why I used it in EmPy as the token prefix (also
because @ at the time is neither commonly used in English text or legal
in Python, though the latter will be changing).

The far-more-obvious-in-code is my reason for preferring @ over |.

The other point that's been raised is that | looks similar to l and I
in a bunch of fonts, and I is a common prefix (for interfaces).

 

[Hallvard B Furuseth]

The far-more-obvious-in-code is my reason for preferring @ over |.

In that respect I think '=' would be pretty good too.
And possibly '!'.

However,

if (long-expression
| more-expresson):
|decorator-function(blah blah)
def foo(): pass

Could do the same with != and '==', but at least that's different from
'!' and '='. One could write

if bar(long_variable_name
=value):
=decorator-function(blah blah)
def foo(): pass

though that seems a bit contrived.

 

[Terry Reedy]

The other point that's been raised is that | looks similar to l and I
in a bunch of fonts, and I is a common prefix (for interfaces).

With a mandatory space after the |, it should be much harder to see it as
anything else, since anything else would be a syntax error. Both 1 deco
and I deco are nonesense, so | deco should be visually interpreted as such.

Terry J. Reedy

 

[Dan Christensen]

I wonder what people think of the following crazy idea, which has two
parts.

1) Allow anonymous multi-line functions. For definiteness, I'll use
the following syntax, but lambda or something else could be used
too:

f = def (a,b,c):
d = a*b
return d + c

2) Define a *binary* operator @ which is just a shorthand for
function application: f @ x = f(x)

Note that f(x) is sometimes pronounced "f at x", so @ is a
reasonable symbol to use. But it could also be something else.

This @ is not associative; make the rule that it associates
from right to left, so g @ f @ x = g(f(x))

Presto, you have decorators:

f = decorator1 @
decorator2 @
def (a,b,c):
d = a*b
return d + c

And the function name is at the top, like some people prefer.

Notes:

- Multi-line anonymous functions are something that have been
requested a lot already.

- With 1) alone, you already can do:

f = decorator1(
decorator2(
def (a,b,c):
d = a*b
return d + c
))

The sole purpose of @ is to avoid all the closing parens.

- To avoid trailing backslashes, the parser would have to
automatically continue to the next line when a line ends in @.

- Since g @ f @ x = g(f(x)), @ is a bit like function composition,
usually written as a small circle, again suggesting that @ is a
reasonable symbol. (But note that g @ f = g(f) which is not the
same as g composed with f...)

- This @ could be useful in other contexts to avoid deeply
nested parentheses.

- If x is a tuple, f @ *x could mean f(*x), which allows
@ to be used with functions of several arguments. (Not
very relevant here.)

Dan

 

[Peter Hansen]

I wonder what people think of the following crazy idea, which has two
parts.

1) Allow anonymous multi-line functions. For definiteness, I'll use
the following syntax, but lambda or something else could be used
too:

f = def (a,b,c):
d = a*b
return d + c

I think this is identical to the following code, isn't it?

def f(a, b, c):
d = a * b
return d + c

[...]

Presto, you have decorators:

f = decorator1 @
decorator2 @
def (a,b,c):
d = a*b
return d + c

And the function name is at the top, like some people prefer.

I think if we asked them, they would clarify that it is not
the name *alone* which is important, but the *definition*.
Roughly the code that says this:

i-am-defining-a-function this-is-the-name other-stuff:

I believe those of us who prefer to see the name first really
prefer to see something we might call the _definition_ (and
"def" is well-named here) which includes a sign that we are
defining a function, very near the name, and preferably with
the argument list also very close (since it is generally quite
important to a reader). Nothing should be more important than
the name, so it should be first, then the arguments and other
meta stuff, then the body. Languages like C which put the
return types first are unfortunate in that the return type is
not as important as the name, yet usually obscures it. (I
used to write the return type on the previous line, sometimes
indented once, to avoid this problem, but that's still not
the best solution.)

- To avoid trailing backslashes, the parser would have to
automatically continue to the next line when a line ends in @.

Sounds like another special case, as I'm not sure Python does
this for anything right now except when there are matched
opening and closing symbols.

-Peter