Docorator Disected

[Ron_Adam]

I was having some difficulty figuring out just what was going on with
decorators. So after a considerable amount of experimenting I was
able to take one apart in a way. It required me to take a closer look
at function def's and call's, which is something I tend to take for
granted.

I'm not sure this is 100%, or if there are other ways to view it, but
it seems to make sense when viewed this way.

Is there a way to do this same thing in a more direct way? Like
taking values off the function stack directly. How much of it get's
optimized out by the compiler?


#
# Follow the numbers starting with zero.
#

# (0) Read defined functions into memory

def decorator(d_arg): # (7) Get 'Goodbye' off stack

def get_function(function): # (8) Get func object off stack

def wrapper(f_arg): # (9) Get 'Hello' off stack

new_arg = f_arg+'-'+d_arg
result = function(new_arg) # (10) Put new_arg on stack
# (11) Call func object

return result # (14) Return result to wrapper

return wrapper # (15) Return result to get_function

return get_function # (16) Return result to caller of func



@decorator('Goodbye') # (5) Put 'Goodbye' on stack
# (6) Do decorator

def func(s): # (12) Get new_arg off stack

return s # (13) Return s to result

# (1) Done Reading definitions


print func('Hello') # (2) Put 'Hello' on stack
# (3) Put func object on stack
# (4) Do @decorator
# (17) print 'Hello-Goodbye'

# Hello-Goodbye

 

[El Pitonero]

# (0) Read defined functions into memory

def decorator(d_arg): # (7) Get 'Goodbye' off stack

def get_function(function): # (8) Get func object off stack

def wrapper(f_arg): # (9) Get 'Hello' off stack

new_arg = f_arg+'-'+d_arg
result = function(new_arg) # (10) Put new_arg on stack
# (11) Call func object

return result # (14) Return result to wrapper

return wrapper # (15) Return result to get_function

return get_function # (16) Return result to caller of func



@decorator('Goodbye') # (5) Put 'Goodbye' on stack
# (6) Do decorator

def func(s): # (12) Get new_arg off stack

return s # (13) Return s to result

# (1) Done Reading definitions


print func('Hello') # (2) Put 'Hello' on stack
# (3) Put func object on stack
# (4) Do @decorator
# (17) print 'Hello-Goodbye'

# Hello-Goodbye

Is it possible that you mistakenly believe your @decorator() is being
executed at the line "func('Hello')"?

Please add a print statement to your code:

def decorator(d_arg):
def get_function(function):
print 'decorator invoked'
def wrapper(f_arg):
new_arg = f_arg+'-'+d_arg
result = function(new_arg)
return result
return wrapper
return get_function

When you run the program, you will see that the comment "decorator
invoked" is printed out at the moment when you finish defining:

@decorator('Goodbye')
def func(s):
return s

That is, decorator is invoked before you run the line "func('Hello')".

Decorator feature is a metaprogramming feature. Not a programming
feature. By metaprogramming I mean you are taking a function/code
object, and try to do something with it (e.g., wrap it around.) By the
time you finish defining the function "func(s)", the decorator
"get_function()" was already invoked and will never be invoked again.

It's better to view functions as individual objects. And try to think
who holds reference to these objects. If no one holds reference to an
object, it will be garbage collected and will be gone. After you define
the function "func()" and before you execute "func('Hello')", this is
the situation:

decorator() <--- held by the module
get_function() <--- temporary object, garbage collected
wrapper() <--- held by the module, under the name "func"
func() <--- held by wrapper(), under the name "function"

'Goodbye' <--- string object, held by the wrapper function object,
under the name d_arg

Objects can be rebound to different names. In your code you have
rebound the original wrapper() and func() function objects to different
names.

I think the confusing part is that, for function name binding, Python
does not use the = operator, but instead relies on the "def" keyword.
Maybe this is something to be considered for Python 3K. Anonymous
function or codeblock objects are good to have, when you are doing
metaprogramming.

 

[Kay Schluehr]

def decorator(d_arg): # (7) Get 'Goodbye' off stack

def get_function(function): # (8) Get func object off stack

def wrapper(f_arg): # (9) Get 'Hello' off stack

new_arg = f_arg+'-'+d_arg
result = function(new_arg) # (10) Put new_arg on stack
# (11) Call func object

return result # (14) Return result to wrapper

return wrapper # (15) Return result to get_function

return get_function # (16) Return result to caller of func

@decorator('Goodbye') # (5) Put 'Goodbye' on stack
# (6) Do decorator

def func(s): # (12) Get new_arg off stack

return s # (13) Return s to result

There is actually nothing mysterious about decorators. It is nothing
more than ordinary function composition, executed when the decorated
function is defined. In case of Your definition it, the composition
rules are:

decorator("Goodbye")(func)(s) = get_function(func)(s) = wrapper(s),
where wrapper stores "Goodbye" in the local d_arg.

Or a bit more formally we state the composition principle:

Args x Func -> Func, where decorator() is a function of Args, that
returns a function Func -> Func. As Guido had shown recently in his
Artima blog, Func need not be an instance of an ordinary function but
can be a function-object like his MultiMethod :

http://www.artima.com/weblogs/viewpost.jsp?thread=101605

It is also possible to extend this view by "chaining" decorators.

decorator : Args(2) x (Args(1) x Func - > Func ) -> Func.

To understand decorator chains it is very helpfull to accept the
functional view instead of arguing in a procedural picture i.e. pushing
and popping arguments onto and from the stack.

Someone asked once for a solution of the following problem that is
similar in character to Guidos multimethod but some more general.

def mul(m1,m2):
def default(m1,m2):
return "default",1+m1*m2
def mul_dec(m1,m2):
return "mul_dec",Decimal(str(m1))*Decimal(str(m2))
def mul_float(m1,m2):
return "mul_float",m1*m2
return (default,mul_dec,mul_float)

The function mul defines the inner functions default, mul_float and
mul_dec. What we want is a unified access to this functions by means of
mul. Guidos solution would decompose mul in three different versions of
mul:

@multimethod(int,float)
def mul(m1,m2):
return m1*m2

@multimethod(float,float)
def mul(m1,m2):
return m1*m2


@multimethod(Decimal,Decimal)
def mul(m1,m2):
return m1*m2

but it is hard to tell, what should be done if no argument tuple
matches.

An attempt like:

@multimethod(object,object)
def mul(m1,m2):
return 1+m1*m2

would be useless, because there is no concrete match of argument types
onto (object,object).

So I introduced an "external switch" over argument tuples, using a
decorator chain:

@case(None,"default")
@case((float,float),'mul_float')
@case((int,float),'mul_float')
@case((Decimal,Decimal),'mul_dec')

def mul(m1,m2):
def default(m1,m2):
return "default",1+m1*m2
def mul_dec(m1,m2):
return "mul_dec",Decimal(str(m1))*Decimal(str(m2))
def mul_float(m1,m2):
return "mul_float",m1*m2
return (default,mul_dec,mul_float)

Can You imagine how "case" works internally?

Regards,
Kay

 

[Ron_Adam]

Is it possible that you mistakenly believe your @decorator() is being
executed at the line "func('Hello')"?

Please add a print statement to your code:

def decorator(d_arg):
def get_function(function):
print 'decorator invoked'
def wrapper(f_arg):
new_arg = f_arg+'-'+d_arg
result = function(new_arg)
return result
return wrapper
return get_function

Thanks, you are correct. I'll post a revised dissection with print
statements documenting the flow in a few minutes. I'm still a bit
fuzzy on how the arguments are stored and passed.

Regards,
Ron_Adam

 

[Diez B. Roggisch]

statements documenting the flow in a few minutes. I'm still a bit

fuzzy on how the arguments are stored and passed.

The arguments are part of the outer scope of the function returned, and thus
they ar kept around. That's standart python,too:

def foo():
a = 10
def bar():
return a*a
return bar

print foo()()


No decorator-specific magic here - just references kept to outer frames
which form the scope for the inner function.

 

[Ron_Adam]

The arguments are part of the outer scope of the function returned, and thus
they ar kept around. That's standart python,too:

def foo():
a = 10
def bar():
return a*a
return bar

print foo()()


No decorator-specific magic here - just references kept to outer frames
which form the scope for the inner function.

I followed that part. The part that I'm having problems with is the
first nested function get's the argument for the function name without
a previous reference to the argument name in the outer frames. So, a
function call to it is being made with the function name as the
argument, and that isn't visable so it looks as if it's magic.

Ok, Since I was using the wrong model the first time, probably due to
not sleeping well and mixing past language experience in improperly,
we will try again.

In the below model, the @decorator, (object or the interpreter
executing the @decorator statement?), calls nested functions in the
function of the same name until it reaches the inner loop which is
then attached to the function name. Is this correct now?

Cheers,
Ron


### Decorator Dissection V.2 ###

print "\n(0) Start reading decorator defs"
def decorator(d_arg):
print "(3) decorator: gets '"+d_arg+"'"

def get_function(function):
print "(6) get_function: gets 'func' object"

def wrapper(f_arg):
print "(10) wrapper: gets '"+f_arg+"'"
new_arg = f_arg+'-'+d_arg

print "(11) wrapper: calls func('"+new_arg+"')"
result = function(new_arg)

print "(13) wrapper: returns '"+result+"'"
return result

print "(7) get_function: returns 'wrapper' object"
return wrapper

w = get_function
print "(4) decorator: return 'get_function' object"
print '(5) @decorator: calls get_function(func)'
# Need to print this here, done at *(5)
return w

print "(1) Done reading decorator defs\n"


print "(2) @decorator: calls decorator('goodbye')"
# *(5) @decorator: call get_funtion(func)
@decorator('Goodbye')
def func(s):
print '(12) func returns:', s
return s
print "(8) @decorator: func = wrapper\n"


print "(9) Call func('Hello') which is now wrapper object:"
result = func('Hello')
print "(14) result gets '"+result+"'\n"

print result


#---output---

(0) Start reading decorator defs
(1) Done reading decorator defs

(2) @decorator: calls decorator('Goodbye')
(3) decorator: gets 'Goodbye'
(4) decorator: return 'get_function' object
(5) @decorator: calls get_function(func)
(6) get_function: gets 'func' object
(7) get_function: returns 'wrapper' object
(8) @decorator: func = wrapper

(9) Call func('Hello') which is now wrapper object:
(10) wrapper: gets 'Hello'
(11) wrapper: calls func('Hello-Goodbye')
(12) func returns: Hello-Goodbye
(13) wrapper: returns 'Hello-Goodbye'
(14) result gets 'Hello-Goodbye'

Hello-Goodbye

 

[Diez B. Roggisch]

I followed that part. The part that I'm having problems with is the

first nested function get's the argument for the function name without
a previous reference to the argument name in the outer frames. So, a
function call to it is being made with the function name as the
argument, and that isn't visable so it looks as if it's magic.

No, its not - but I stepped into that trap before - and thought its magic

The trick is to know that

- a decorator is a callable
- get passed a callable
- has to return a callable

So this is the simplest decorator imaginable is:

def identity(f):
return f

And the decorator _syntax_ is just a python expression that has to be
_evaluated_ to a yield decorator. So

@identity
def foo(self):
pass

the @identity is just the expression evaluated - to the function reference
to identity, which is callable and follows the decorator protocol - and the
_result_ of that evaluation is called with the callable in question.

So if you want to have _parametrized_ decorators, that expression is
_evaluated_ and has to yield a decorator. Like this:

def arg_decorator(arg):
def real_decorator(f):
return f
return real_decorator

So, this works

@arg_decorator('fooobar')
def foo(self):
pass

@arg_decorator('fooobar') is evaluated to real_decorator (which a scope
containing arg), and _that_ gets called with foo.

HTH - bit me the first time too

 

[Ron_Adam]

There is actually nothing mysterious about decorators.

I've heard this quite a few times now, but *is* quite mysterious if
you are not already familiar with how they work. Or instead of
mysterious, you could say complex, as they can be used in quite
complex ways.

What is missing most with them is some really good documentation. I
got the basic idea and syntax of decorators down right away, but ran
into problems implementing them because, the structure of the
functions being used for the decorators wasn't clear.

It is nothing
more than ordinary function composition, executed when the decorated
function is defined. In case of Your definition it, the composition
rules are:

decorator("Goodbye")(func)(s) = get_function(func)(s) = wrapper(s),
where wrapper stores "Goodbye" in the local d_arg.

It worked as a model, but I mixed in concepts from cstacks and
function calls, which apparently isn't correct. I posted another
model, it should be a bit closer. (with the Subject line spelled
correctly, continue this thread there.

Or a bit more formally we state the composition principle:

Args x Func -> Func, where decorator() is a function of Args, that
returns a function Func -> Func. As Guido had shown recently in his
Artima blog, Func need not be an instance of an ordinary function but
can be a function-object like his MultiMethod :

http://www.artima.com/weblogs/viewpost.jsp?thread=101605

I read this, this morning it was very interesting.

It is also possible to extend this view by "chaining" decorators.

decorator : Args(2) x (Args(1) x Func - > Func ) -> Func.

To understand decorator chains it is very helpfull to accept the
functional view instead of arguing in a procedural picture i.e. pushing
and popping arguments onto and from the stack.

Understanding chains is next on my list.

Someone asked once for a solution of the following problem that is
similar in character to Guidos multimethod but some more general.

def mul(m1,m2):
def default(m1,m2):
return "default",1+m1*m2
def mul_dec(m1,m2):
return "mul_dec",Decimal(str(m1))*Decimal(str(m2))
def mul_float(m1,m2):
return "mul_float",m1*m2
return (default,mul_dec,mul_float)

The function mul defines the inner functions default, mul_float and
mul_dec. What we want is a unified access to this functions by means of
mul. Guidos solution would decompose mul in three different versions of
mul:

This is similar to c++'s polymorphism which I've played with nearly 10
years ago. I generally found it useful only in small doses even then.
I seem to think now that c++'s version of it was implemented at
compile time, with each function call being matched up with the
correct function, by the argument types. Where as Guido's version, is
dynamic and handles the situation at run time. I may not be correct
in this, it's been a while.

@multimethod(int,float)
def mul(m1,m2):
return m1*m2

@multimethod(float,float)
def mul(m1,m2):
return m1*m2


@multimethod(Decimal,Decimal)
def mul(m1,m2):
return m1*m2

but it is hard to tell, what should be done if no argument tuple
matches.

It could then invoke the adapt() function to determine if a possible
single way to continue is available. But with that you could run into
some very subtle bugs. Or just annoying windows like behavior, such
as a word processor auto correcting a word when don't want it to.

An attempt like:

@multimethod(object,object)
def mul(m1,m2):
return 1+m1*m2

would be useless, because there is no concrete match of argument types
onto (object,object).

So I introduced an "external switch" over argument tuples, using a
decorator chain:

@case(None,"default")
@case((float,float),'mul_float')
@case((int,float),'mul_float')
@case((Decimal,Decimal),'mul_dec')

def mul(m1,m2):
def default(m1,m2):
return "default",1+m1*m2
def mul_dec(m1,m2):
return "mul_dec",Decimal(str(m1))*Decimal(str(m2))
def mul_float(m1,m2):
return "mul_float",m1*m2
return (default,mul_dec,mul_float)

Can You imagine how "case" works internally?

Regards,
Kay

Sure, That should be fairly straight forward. Although I can imagine
several ways of implementing it at the moment. I think after I play
with decorator chains, one way will probably stand out as being
cleaner than the others.

Cheers,
Ron

 

[El Pitonero]

I've heard this quite a few times now, but *is* quite mysterious if
you are not already familiar with how they work. Or instead of
mysterious, you could say complex, as they can be used in quite
complex ways.

If the syntax were like:

decorator = function(d_arg) {
return function(f) {
return function(f_arg) {
new_arg = f_arg+'-'+d_arg;
return f(new_arg);
}
}
}

func = decorator('Goodbye') function(s) {
return s;
}

Would you think it would be more easily understandable? Here,
"function()" is a metafunction (or function factory) whose role is to
manufacture a function given a parameter spec and a code body. And in
the expression

func = decorator('Goodbye')(function(s){return s;})

one pair of outter parenthesis have been omitted. Sure, it's not as
readable as Python's "def", but with today's syntax highlighters, the
special word "function" can be highlighted easily.

If the decorator does not have parameters, one has:

func = decorator function(s) {
....
}

or in the general case:

func = deco1 deco2 deco3 function(s) {
....
}

 

[Ron_Adam]

No, its not - but I stepped into that trap before - and thought its magic

It's magic until we understand it.

I get the feeling that those who have gotten to know decorators find
them easy, and those who haven't, find them nearly impossible to
understand. Which means there is a fairly large first few steps to
get over, then it gets easy. There *is* some underlying processes
being made, which is also the reason that makes them attractive. Less
type/declare/organize/etc... but that is also what causes the
difficulty in understanding and using them at first.

The trick is to know that

- a decorator is a callable
- get passed a callable
- has to return a callable

So this is the simplest decorator imaginable is:

def identity(f):
return f

And the decorator _syntax_ is just a python expression that has to be
_evaluated_ to a yield decorator. So

@identity
def foo(self):
pass

This much I understand.

the @identity is just the expression evaluated - to the function reference
to identity, which is callable and follows the decorator protocol - and the
_result_ of that evaluation is called with the callable in question.

This tells me what it is, and what it does, but not how it works. How
is the ***expression evaluated***, what is the ***decorator
protocol***.

Those are the parts I'm trying to understand at this point. I know
this is the equivalent of looking behind the curtains to reveal the
little man who is the wizard. But I can't resist.

So if you want to have _parametrized_ decorators, that expression is
_evaluated_ and has to yield a decorator. Like this:

There's that word again... **evaluated**. How?

def arg_decorator(arg):
def real_decorator(f):
return f
return real_decorator

So, this works

@arg_decorator('fooobar')
def foo(self):
pass

@arg_decorator('fooobar') is evaluated to real_decorator (which a scope
containing arg), and _that_ gets called with foo.

So if I'm following you right?

When the interpreter gets to the line @arge_decorator('fooobar')

it does the following?

foo = arg_decorator('fooobar')(foo)() #?

(experiment with idle a bit...)

Ok I got it.

I wasn't aware that the form:

result = function(args)(args)

Was a legal python statement.

So python has a built in mechanism for passing multiple argument sets
to nested defined functions! (click) Which means this is a decorator
without the decorator syntax.

def arg_decorator(arg1):
def real_decorator(function):
def wrapper(arg2)
return f(arg2)
return real_decorator

def foo(arg2):
pass
foo = arg_decorator('fooobar')(foo)(2arg)

The apparent magic is the silent passing of the second two arguments.

So this isn't a decorator question any more. Each argument gets
passed to the next inner defined function, via... a stack(?)

Somehow I think I've completed a circle. LOL

Cheers,
Ron

 

[Ron_Adam]

Thanks Kay, I wasn't aware of pythons ability to pass arguments to
nested functions in this way. I missed it the first time.

Cheers,
Ron

 

[Bengt Richter]

I was having some difficulty figuring out just what was going on with
decorators. So after a considerable amount of experimenting I was
able to take one apart in a way. It required me to take a closer look
at function def's and call's, which is something I tend to take for
granted.

I think it might help you to start out with very plain decorators rather than
decorators as factory functions that return decorator functions that wrap the
decorated function in a wrapper function. E.g., (this could obviously be
parameterized as a single decorator factory, but I wanted to show the simplest level
of decorator functionality)
... f.decstr = getattr(f, 'decstr', '') + 'a'
... return f
... ... f.decstr = getattr(f, 'decstr', '') + 'b'
... return f
... ... f.decstr = getattr(f, 'decstr', '') + 'c'
... return f
... ... @decoc
... @decob
... @decob
... def foo(): pass
... 'bbca'

I.e.,

@decoa
@decoc
def foo(): pass

is almost[1] exactly equal to (note calling order c then a)

def foo(): pass
foo = decoc(foo)
foo = decoa(foo)

[1] One difference is that foo = deco(foo) is a RE-binding of foo,
and the binding wouldn't happen at all if the @deco version
raised an exception in deco. E.g.,
...

foo not yet defined: Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'foo' is not defined

Try the bad decorator: ... def foo(): pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in deco
NotImplementedError

No go, and foo still undefined: Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'foo' is not defined

But the other way around:

bar undefined to start: Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'bar' is not defined

Define it sucessfully: <function bar at 0x02EE8B54>

Try to decorate the old-fashioned way: Traceback (most recent call last):
File "<stdin>", line 1, in ?
<function bar at 0x02EE8B54>

Still there, defined as before (well, strictly speaking, not
necessarily as before: with bar already defined, deco could
have messed with the existing bar and THEN raised the exception).
Which would also happen with @deco, just that the new binding to bar
wouln't happen.
... f.bomb = 'bombed'
... raise Exception, 'Did it bomb the existing function?'
...

>>> def foo(): pass ...
>>> vars(foo).keys() []
>>> @decobomb

... def foo(): pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
[]

Nope, seems that was a new foo that got bombed and then not
bound to replace the old foo.
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'function' object has no attribute 'bomb'

I'm not sure this is 100%, or if there are other ways to view it, but
it seems to make sense when viewed this way.

I like annotated code walk-throughs. But as others have pointed out,
it's still a bit buggy ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

No, its not - but I stepped into that trap before - and thought its magic

The trick is to know that

- a decorator is a callable

Strictly speaking, UIAM "deco" in @deco is a non-general
very-limited-syntax expression that should evaluate to a callable.

- get passed a callable
- has to return a callable

That last is a "should" ;-) Actually, you can abuse the def just
to provide a binding name whose associated function is otherwise ignored
(other than syntax check and compilation ), e.g.,
... def foo(): pass
... 'maverick'

Or
... def foo(): pass
... '<function foo at 0x02EE8D14>'

Note the quotes ;-)
<type 'str'>

Just being picky about absolute statements ;-)

Regards,
Bengt Richter

 

[M.E.Farmer]

Hello Ron ,
You have many good explanations already, but I thought that this
__might__ help others.
Like you I was confused by the decorator syntax. till I realized it was
shorthand for ...

def identity(f):
return f

def foo():
pass

# this is the 'old way'
foo = identity(foo)

It just rebinds foo to the return value of the decorator function.
With the new syntax it becomes.

def identity(f):
return f

@identity
def foo(self):
pass
This is the same as above but now the function is
passed and rebound behind the scenes.
Also note that decorators don't have to be a nested function, it really
depends on what you are trying to achieve.

hth,
M.E.Farmer

 

[Ron_Adam]

I think it might help you to start out with very plain decorators rather than
decorators as factory functions that return decorator functions that wrap the
decorated function in a wrapper function. E.g., (this could obviously be
parameterized as a single decorator factory, but I wanted to show the simplest level
of decorator functionality)

<clipped interesting examples>

Thanks for the examples of stacked decorators!

I think I pretty much got it now, I had never needed to pass arguments
to nested "defined" functions before and none of the documentation I
have, ever mentioned that alternative.

So I didn't know I could do this:


def foo(a1):
def fee(a2):
return a1+a2
return fee

fum = foo(2)(6) <------ !!!

# fum is 8


The interesting thing about this is the 'return fee' statement gets
the (6) apparently appended to it. So it becomes 'return fee(6).

That subtle action is confusing if you don't already know about it,
which I didn't.

In this example.


def foo(a1):
def fee(a2):
return a1+a2
return fee

fum = foo(2)


There is no second set of arguments to append to 'return fee', so the
name fum is pointed to object fee instead and fee is not evaluated.

This second subtle action, is also confusing if you aren't aware of
it. Since the two look the same when you examine the def statements.
So there is no reason to think they would not act the same, both
returning an function object.

Now, add in the @decorator syntax to the mix. Which hides the extra
argument sets that are passed to the nested defined functions and the
obscuration is complete. There then is no visual indication of where
the function calls get their arguments from, and this is what I
believe caused me to have so much trouble with this.

Another inconsistency, although not a bad one, is that nested
'defined' function share scope, but nested function calls do not.

Now what this means, is it will be very difficult for some people to
put it all together. I would have gotten it sooner or later, but I
really happy to have help from comp.lang.python. on this one.

I like annotated code walk-throughs. But as others have pointed out,
it's still a bit buggy ;-)

It helped a lot, but notice that it took me several tries. That's a
strong indicator that decorators are more implicit than explicit and
that goes against the "Explicit is better than Implicit" guideline
that python tries to follow.

Maybe there are ways to make decorators -and- nested function calls a
bit more explicit?

I think a having indicators on the return statements that are meant to
return a value vs object would help readability and take some of the
mystery out as far as the un initiated are concerned.

def foo(a1):
def fee(a2):
def fiddle(a3):
pass
return a3
return fee # Always return a function object.
# Error, if argument is passed to it.
# and

return fee(a2) # always require an argument,
# error if none is passed to it.

Or some other way if this breaks something. But it will make it more
apparent what nested function should do. And give clearer feed back
when trying to use or write decorators.

I'm not sure what might make @decorator more explicit. Maybe allowing
all the function to be specified as an option. Maybe it is already(?)

@decorator(a1)(foo)
def foo():
pass


So we will have:

def foo(a1):
def fee(a2):
def fiddle(a3):
pass
return a3
return fee # Object always returned here or
# or error if argument is received.

@decorator(a1)(fum) # Last argument optional.
def fum(a3):
return a3

These I think are small changes that might be acceptable.

A little more aggressive alterations would be: Requiring the
'function' argument may have a use when using stacked decorators. Then
it could be inserted into a sequence?

@deco3(postcalc)
@deco2(fum)
@deco1(precalc)
def fum(pointxyz):
return translatepoint(pointxyz)

.... and that reversed order... (yuck!), is it really necessary?
Readability is important, and it is a big reason people don't jump
ship for some other language. Why the exceptions here?

Ok, don't mean to grip. I'm sure there's been plenty of that in
past discussions.

Cheers,
Ron

 

[El Pitonero]

So I didn't know I could do this:

def foo(a1):
def fee(a2):
return a1+a2
return fee

fum = foo(2)(6) <------ !!!

Ah, so you did not know functions are objects just like numbers,
strings or dictionaries. I think you may have been influenced by other
languages where there is a concept of static declaration of functions.

The last line can be better visualized as:

fum = (foo(2)) (6)

where foo(2) is a callable.

-----------

Since a function is an object, they can be assigned (rebound) to other
names, pass as parameters to other functions, returned as a value
inside another function, etc. E.g.:

def g(x):
return x+3

h = g # <-- have you done this before? assignment of function

print h(1) # prints 4

def f(p):
return p # <-- function as return value

p = f(h) # <-- passing a function object

print p(5) # prints 8

Python's use of "def" keyword instead of the "=" assignment operator
makes it less clear that functions are indeed objects. As I said
before, this is something to think about for Python 3K (the future
version of Python.)

------------

Function modifiers exist in other languages. Java particularly is
loaded with them.

public static synchronized double random() {
....
}

So your new syntax:

@decorator(a1)(foo)
def foo():
pass

is a bit out of the line with other languages.

 

[Ron_Adam]

Ah, so you did not know functions are objects just like numbers,
strings or dictionaries. I think you may have been influenced by other
languages where there is a concept of static declaration of functions.

No, I did not know that you could pass multiple sets of arguments to
nested defined functions in that manner. Just haven't ran acrossed it
in the two years I've been playing around with python. I haven't had a
reason to try it either. But maybe now that I'm aware of it, I'll
find more uses for it.

The last line can be better visualized as:

fum = (foo(2)) (6)

where foo(2) is a callable.

-----------

Since a function is an object, they can be assigned (rebound) to other
names, pass as parameters to other functions, returned as a value
inside another function, etc. E.g.:

def g(x):
return x+3

h = g # <-- have you done this before? assignment of function

Sure, I have no problem with that. Been doing it for quite a while.

print h(1) # prints 4

def f(p):
return p # <-- function as return value

p = f(h) # <-- passing a function object

print p(5) # prints 8

Python's use of "def" keyword instead of the "=" assignment operator
makes it less clear that functions are indeed objects. As I said
before, this is something to think about for Python 3K (the future
version of Python.)

I've always equated 'def' as if it were 'make', or in Python its just
a variation of 'class' for a subset of objects of type 'function'.

------------

Function modifiers exist in other languages. Java particularly is
loaded with them.

public static synchronized double random() {
...
}

So your new syntax:

@decorator(a1)(foo)
def foo():
pass

is a bit out of the line with other languages.

So? Why would it need to be the same as other languages? I like
Python because it's not the same.

The above syntax suggestion, just matches the already existing
behavior,

Thanks for helping BTW, I think I have it down pretty good now.

Cheers,
Ron

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I wasn't aware that the form:

result = function(args)(args)

Was a legal python statement.

So python has a built in mechanism for passing multiple argument sets
to nested defined functions! (click) Which means this is a decorator
without the decorator syntax.

No. There is no mechanism for passing multiple argument sets to
nested functions. Instead, functions are objects, which can be
assigned to variables, passed as arguments to other functions,
and returned:

>>> len

>>> other=len
>>> other([1,2,3]) 3
>>> other

<built-in function len>

Here, len is *an object* that gets *assigned to* the variable other.
The grammatical construct

<something>(<list of something>)

is evaluated as follows:

1. evaluate <something>. Evaluation always returns an object,
be it 1+2, other, or f(x).
2. evaluate <list of something>, from left to right.
3. call the object returned in step 1, with the arguments
computed in step 2.

Because the something before the parentheses can be any expression,
you can also write things like

>>> items=[]
>>> items.append(len)
>>> items[0](range(10))

10

Functions as parameters and return values are not much different:
.... return x
....10

Now, a nested function is locally defined, but then the function
is returned
.... def lenfun(o):
.... return 2*len(o)
.... return lenfun
....20

Here, first getlenfun() is evaluated, returning a function lenfun.
Then, range(10) is evaluated, returning a list. Then, lenfun is
invoked with this list.

So far, this has nothing to do with decorators.

So this isn't a decorator question any more. Each argument gets
passed to the next inner defined function, via... a stack(?)

No, functions are objects. Notice that in step 1, the object returned
doesn't have to be a function - other things are callable, too, like
types, classes, and objects implementing __call__.

Regards,
Martin

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

No, I did not know that you could pass multiple sets of arguments to
nested defined functions in that manner.

Please read the statements carefully, and try to understand the mental
model behind them. He did not say that you can pass around multiple
sets of arguments. He said that functions (not function calls, but
the functions themselves) are objects just like numbers. There is
a way of "truly" understanding this notion, and I would encourage
you to try doing so.

Regards,
Martin

 

[Bengt Richter]

No, I did not know that you could pass multiple sets of arguments to

That phraseology doesn't sound to me like your concept space is quite isomorphic
with reality yet, sorry ;-) It sounds like you are thinking of "multiple sets of arguments"
as an aggregate that is passed as such, and that isn't happening, as I believe El Pitonero
is trying to indicate with his parenthesized visualization below.

What is happening is that an expression "foo(2)(6)" is being evaluated left to right.
First foo as a name evaluates to whatever it is bound to, which is the foo function.
Then () is the calling operator, which says evaluate the list inside the parens left to right
and call the thing you had so far, which was foo here. The arg list was just 2, so foo is called
with 2, and foo returns something, with which we will do the next operation if there is one.

If the next operation was "." (i.e., attribute getting) the next thing following would have had
to be an attribute name, e.g. like func_name. foo(2).func_name would evaluate to the same as fee.func_name
for the fee returned by foo(2). But we are not doing .func_name, we are doing (6) as the next operation
in the left-to-right evaluation of the expression. And whatever we have at the foo(2) stage, the (6) means
we should take it and call it with 6 as an argument.

So if you are seeing (2)(6) as something to pass, as opposed to a sequence of operations, I think there's
a misconception involved. Perhaps I am taking your words askew ;-)

nested defined functions in that manner. Just haven't ran acrossed it
in the two years I've been playing around with python. I haven't had a
reason to try it either. But maybe now that I'm aware of it, I'll
find more uses for it.

That's clear to me, anyway ;-)

The code shows it too:
1 0 LOAD_NAME 0 (foo)
3 LOAD_CONST 1 (2)
6 CALL_FUNCTION 1
9 RETURN_VALUE

The (6) just calls whatever the result of the preceding was
1 0 LOAD_NAME 0 (foo)
3 LOAD_CONST 1 (2)
6 CALL_FUNCTION 1
9 LOAD_CONST 2 (6)
12 CALL_FUNCTION 1
15 RETURN_VALUE

HTH

Regards,
Bengt Richter