I love the decorator in Python!!!

8

88888 Dihedral

I use the @ decorator to behave exactly like a c macro that
does have fewer side effects.

I am wondering is there other interesting methods to do the
jobs in Python?

A lot people complained that no macro in Python.

Cheers to the rule of Python :
If there's none then just go ahead and build one!
 
C

Chris Rebert

I use the @ decorator to behave exactly like a c macro that
does have fewer side effects.

I am wondering is there other interesting methods to do the
jobs in Python?

* Class decorators (http://www.python.org/dev/peps/pep-3129/ ); i.e.
using decorators on a class instead of a function/method
* Metaclasses (http://docs.python.org/dev/reference/datamodel.html#customizing-class-creation
); a rather more complicated feature capable of serious black magic
* Descriptors (http://docs.python.org/dev/reference/datamodel.html#implementing-descriptors
); probably the most rarely directly used of the three

Cheers,
Chris
 
T

Thomas Rachel

Am 08.12.2011 08:18 schrieb 88888 Dihedral:
I use the @ decorator to behave exactly like a c macro that
does have fewer side effects.

I am wondering is there other interesting methods to do the
jobs in Python?

In combination with a generator, you can do many funny things.


For example, you can build up a string:

def mkstring(f):
"""Turns a string generator into a string,
joining with ", ".
"""
return ", ".join(f())

def create_answer():
@mkstring
def people():
yield "Anna"
yield "John"
yield "Theo"

return "The following people were here: " + people


Many other things are thinkable...


Thomas
 
K

K.-Michael Aye

Am 08.12.2011 08:18 schrieb 88888 Dihedral:

In combination with a generator, you can do many funny things.


For example, you can build up a string:

def mkstring(f):
"""Turns a string generator into a string,
joining with ", ".
"""
return ", ".join(f())

def create_answer():
@mkstring
def people():
yield "Anna"
yield "John"
yield "Theo"

return "The following people were here: " + people


Many other things are thinkable...


Thomas

I am still perplexed about decorators though, am happily using Python
for many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each
other with a comma, why wouldn't I just create a function doing exactly
this? Why would I first write a single name generator and then decorate
it so that I never can get single names anymore (this is the case,
isn't it? Once decorated, I can not get the original behaviour of the
function anymore.
So, above, why not
def mkstring(mylist):
with the same function declaration and then just call it with a list of
names that I generate elsewhere in my program?
I just can't identify the use-case for decorators, but as I said, maybe
I am missing something.

Michael
 
C

Chris Angelico

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA
 
K

K.-Michael Aye

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more
sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA

I understand this one, it seems really useful. And maybe i start to
sense some more applicability. Like this, with extra flags that could
be set at run time, I could influence the way a function is executed
without designing the function too complex, but by decorating it, which
at the end could be easier to read than complicated if-then statements
in the function.
Thanks for your example.
Michael
 
C

Chris Angelico

I understand this one, it seems really useful. And maybe i start to sense
some more applicability. Like this, with extra flags that could be set at
run time, I could influence the way a function is executed without designing
the function too complex, but by decorating it, which at the end could be
easier to read than complicated if-then statements in the function.

Right, and less duplication usually results in less bugs, too.
Obviously if you go to any greater level of complexity than this you'd
want to break the lambda out into a proper function, and possibly play
with an indent level (increment on call, decrement on return). You can
also easily add the run-time check, and anything else you want to do,
too. It'll still all be buried away in the definition of trace(), and
on functions that you want to trace, you need still only have "@trace"
and nothing else.

One piece of sophistication that I would rather like to see, but don't
know how to do. Instead of *args,**kwargs, is it possible to somehow
copy in the function's actual signature? I was testing this out in
IDLE, and the fly help for the function no longer gave useful info
about its argument list.

This is why I say that this has almost certainly been done before in a
much better way.

ChrisA
 
A

Andrew Berg

One piece of sophistication that I would rather like to see, but don't
know how to do. Instead of *args,**kwargs, is it possible to somehow
copy in the function's actual signature?
I remember seeing this in a PEP that is planned to be implemented in 3.3.
 
A

Andrew Berg

Decorators are great for adding common functionality to several
functions without duplicating code. For example, I have one for my IRC
bot that checks that the person sending the command is authorized to use
the command. It's only "if mask in owner list then execute function else
say access denied", but it simplifies the code and the intent of each
function that uses it becomes more obvious. I have another that puts
functions in a common try/except clause, which again simplifies the
functions and makes them easier to understand.
 
E

Ethan Furman

Chris said:
One piece of sophistication that I would rather like to see, but don't
know how to do. Instead of *args,**kwargs, is it possible to somehow
copy in the function's actual signature? I was testing this out in
IDLE, and the fly help for the function no longer gave useful info
about its argument list.

If you want to roll your own, it looks something like this:

2.x code
----------------------------------------------------------------------
def trace(func, debugmode=debugmode):
if debugmode:
name = func.__name__
argspec = inspect.getargspec(func)
signature = inspect.formatargspec(
formatvalue=lambda val: "", *argspec)[1:-1] # trim parens
new_func = (
'def _wrapper_(%(signature)s):\n'
' print(">"+func.__name__)\n'
' result = func(%(signature)s)\n'
' print("<"+func.__name__)\n'
' return result\n'
% {'signature':signature}
)
evaldict = {'func':func}
exec new_func in evaldict
wrapped = evaldict['_wrapper_']
wrapped.__name__ = name
wrapped.__doc__ = func.__doc__
wrapped.__module__ = func.__module__
wrapped.__dict__ = func.__dict__
wrapped.func_defaults = func.func_defaults
return wrapped
return func
----------------------------------------------------------------------

The downside (which you get even with Michele's decorator module) is
that tracebacks aren't quite as clean.

~Ethan~
 
E

Ethan Furman

Chris said:
One piece of sophistication that I would rather like to see, but don't
know how to do. Instead of *args,**kwargs, is it possible to somehow
copy in the function's actual signature? I was testing this out in
IDLE, and the fly help for the function no longer gave useful info
about its argument list.

And here it is with Michele's decorator module:

2.x code (probably the same in 3.x, but haven't tested)
----------------------------------------------------------------
from decorator import decorator

debugmode = True

def trace(func):
if debugmode:
@decorator
def traced(func, *a, **ka):
print(">", func.__name__, a, ka)
result = func(*a, **ka)
print("<", func.__name__)
return result
return traced(func)
return func

@trace
def test(x):
"a simple test"
print("Test! "+x)
return 5
 
A

alex23

One piece of sophistication that I would rather like to see, but don't
know how to do. Instead of *args,**kwargs, is it possible to somehow
copy in the function's actual signature? I was testing this out in
IDLE, and the fly help for the function no longer gave useful info
about its argument list.

The 3rd party 'decorator' module takes care of issues like docstrings
& function signatures. I'd really like to see some of that
functionality in the stdlib though.
 
T

Thomas Rachel

Am 08.12.2011 12:43 schrieb Chris Angelico:
The example given is a toy. It's hardly useful.

Right. It was supposed to be an example.

In my case, I work with a script used to build a XML file. I change this
script from time to time in order to match the requirements.
Here I find it useful just to add some more yield statements for adding
entries.

But now that I think again about it, it's more an example for
generators, not so much for decorators - which I like as well.

*****

But some useful examples for decorators include

1. "Threadifying" a function, i.e. turning it into a Thread object, or
into a callable which in turn starts a thread according to the given
parameters.

2. Automatically calling a function if the given module is executed as a
script, a kind of replacement for the "if __name__ == '__main__':" stuff.

3. Meta decorators:

I find it annoying to have to wrap the function given to the decorator
into another one, modifying its properties and returning that in turn.

def wrapfunction(decorated):
"""Wrap a function taking (f, *a, **k) and replace it with a
function taking (f) and returning a function taking (*a, **k) which
calls our decorated function.
"""
from functools import wraps
@wraps(decorated)
def wrapped_outer(f):
@wraps(f)
def wrapped_inner(*a, **k):
return decorated(f, *a, **k)
return wrapped_inner
return wrapped_outer

makes it much easier to create decorators which just wrap a function
into another, extending its funtionality:

@wrapfunction
def return_list(f, *a, **k)
return list(f(*a, **k))

is much easier and IMHO much better to read than

def return_list(f):
"""Wrap a function taking (f, *a, **k) and replace it with a
function taking (f) and returning a function taking (*a, **k) which
calls our decorated function.
"""
from functools import wraps
@wraps(f)
def wrapped(*a, **k):
return list(f, *a, **k)
return wrapped

- especially if used multiple times.

3a. This is a modified case of my first example: If you want a function
to assemble and return a list instead of a generator object, but prefer
"yield" over "ret=[]; ret.append();...", you can do that with this
@return_list.

4. So-called "indirect decorators":

@spam(eggs)
def foo(bar):
pass

are as well quite tricky to build when taking

def indirdeco(ind):
from functools import update_wrapper, wraps
upd=wraps(ind)
# outer wrapper: replaces a call with *a, **k with an updated
# lambda, getting the function to be wrapped and applying it and
# *a, **k to ind.
outerwrapper=lambda *a, **k: upd(lambda f: ind(f, *a, **k))
# We update this as well:
return upd(outerwrapper)
# We don't update f nor the result of ind() - it is the callee's
# business.

It is kind of reverse to 3.

@indirdeco
def addingdeco(f, offset):
return lambda *a, **k: f(*a, **k) + offset
# Here should maybe be wrapped - it is just supposed to be an
# example.

5. Creating a __all__ for a module. Instead of maintaining it somewhere
centrally, you can take a

class AllList(list):
"""list which can be called in order to be used as a __all__-adding
decorator"""
def __call__(self, obj):
"""for decorators"""
self.append(obj.__name__)
return obj

, do a __all__ = AllList()

and subsequently decorate each function with

@__all__

6. Re-use a generator:

A generator object is creted upon calling the generator function with
parameters and can be used only once. A object wrapping this generator
might be useful.

# Turn a generator into a iterable object calling the generator.
class GeneratorIterable(object):
"""Take a parameterless generator function and call it on every
iteration."""
def __init__(self, gen):
# Set object attribute.
self.gen = gen
def __iter__(self):
# Class attribute calls object attribute in order to keep
# namespace variety small.
return self.gen()


@GeneratorIterable
def mygen():
yield 1
yield 2

list(mygen) -> [1, 2]
list(mygen) -> [1, 2] # again, without the ()

Might be useful if the object is to be transferred to somewhere else.

*****

Some of these decorators are more useful, some less if seen standalone,
but very handy if creating other decorators.

HTH nevertheless,


Thomas
 
R

Robert Kern

The 3rd party 'decorator' module takes care of issues like docstrings
& function signatures. I'd really like to see some of that
functionality in the stdlib though.

Much of it is:

http://docs.python.org/library/functools#functools.update_wrapper

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
 
F

Francesco Bochicchio

I am still perplexed about decorators though, am happily using Python
for many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each
other with a comma, why wouldn't I just create a function doing exactly
this? Why would I first write a single name generator and then decorate
it so that I never can get single names anymore (this is the case,
isn't it? Once decorated, I can not get the original behaviour of the
function anymore.
So, above, why not
def mkstring(mylist):
with the same function declaration and then just call it with a list of
names that I generate elsewhere in my program?
I just can't identify the use-case for decorators, but as I said, maybe
I am missing something.

Michael

I had/have similar feelings. For instance, this is something that I
tought useful, but then I never used in real code.
The idea was to find a way to automate this code pattern, which I do a
lot:

class SomeClass:
def __init__(self, some, attribute, here ):
self.some, self.attribute, self.here = some, attribute, here


In other words, I often define classes in which the constructor list
of arguments corresponds one-to-one to class attributes.
So I thought of this (it uses class decorators so it only works with
Python 3.x ) :


class FieldsDecorator:
def __init__(self, *names):
self.names = names

def __call__(self, cls):
def constructor(instance, **kwds):
for n,v in kwds.items():
if n in self.names:
setattr(instance, n, v)
else: raise TypeError("%s is not a valid field" % s )
setattr(cls, '__init__', constructor )
return cls


@FieldsDecorator("uno", "due")
class Prova:
pass

p = Prova(uno=12, due=9)
print (p.uno, p.due )


It works and it is nice, but I don't find it compelling enough to use
it. I keep assigning directly the attributes, which is more readable.

Decorators are really useful when you have lot of repetitive
boilercode that you _want_ to hide, since it has little to do with
the problem logic and more to to with the technicalities of the
programming language or of some framework that you are using. It is
called "separating of concerns" I think, and is one of the principles
of Aspect-Oriented Programming (and with decorators you can do some
nice AOP exercises ... ).

Ciao
 
8

88888 Dihedral

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA

I did use decorators to turn functions into iterables to be traced.
 
8

88888 Dihedral

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA

I did use decorators to turn functions into iterables to be traced.
 
8

88888 Dihedral

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
test("asdf")
test
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA

I did use decorators to turn functions into iterables to be traced.

It is easy to use decorators in python to mimic those programs in Erlang.
 
8

88888 Dihedral

I am still perplexed about decorators though, am happily using Python for
many years without them, but maybe i am missing something?
For example in the above case, if I want the names attached to each other
with a comma, why wouldn't I just create a function doing exactly this? Why
would I first write a single name generator and then decorate it so that I
never can get single names anymore (this is the case, isn't it? Once
decorated, I can not get the original behaviour of the function anymore.

The example given is a toy. It's hardly useful. However, there are a
number of handy uses for decorators; mostly, they consist of giving a
single simple keyword to a complicated set of logic. One example is
the @classmethod and @staticmethod decorators - the code to implement
them could be uglier than nested inline assembly, but you don't have
to care, because you just type "@staticmethod" in front of your def
statement and it does its magic.

Here's a handy trick that I'm sure someone has done in a more sophisticated way:

def trace(func):
if debugmode:
return lambda *a,**ka:
(print(">"+func.__name__),func(*a,**ka),print("<"+func.__name__))[1]
return func

Then you put @trace in front of all your functions, and if debugmode
is False, nothing will be done - but set it to true, and you get
console output at the entry and exit of each function.
def test(x):
print("Test! "+x)
return 5
test("asdf")
test
Test! asdf
<test
5

Again, it's helpful because it condenses all the logic (including the
'debugmode' flag) down to a single high level directive: "Trace this
function".

ChrisA

I did use decorators to turn functions into iterables to be traced.

It is easy to use decorators in python to mimic those programs in Erlang.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,156
Messages
2,570,878
Members
47,405
Latest member
DavidCex

Latest Threads

Top