Decorater inside a function? Is there a way?

R

Ron_Adam

I'm trying to figure out how to test function arguments by adding a
decorator.

@decorate
def func( x):
# do something
return x

This allows me to wrap and replace the arguments with my own, but not
get the arguments that the original function received.

To do that I would need to put the decorator inside the function.

def func( x):
@decorate
# doc something
return x

Then I could use @decorators to check the function input for
condition, ranges, and or types.

Is there a equivalent way to do that?

Also can I use @decorate with assert?


Ron
 
J

Jeremy Bowers

I'm trying to figure out how to test function arguments by adding a
decorator.

The rest of your message then goes on to vividly demonstrate why
decorators make for a poor test technique.

Is this an April Fools gag? If so, it's not a very good one as it's quite
in line with the sort of question I've seen many times before. "I have
a hammer, how do I use it to inflate my tire?"

Assuming you're serious, why not use one of the many testing technologies
actually designed for it, and tap into the associated body of knowledge on
how to accomplish various tasks? Start with what you're trying to do, then
work on how to do it.
 
R

Ron_Adam

The rest of your message then goes on to vividly demonstrate why
decorators make for a poor test technique.

So it's not possible to do. Ok, thanks.
Is this an April Fools gag? If so, it's not a very good one as it's quite
in line with the sort of question I've seen many times before. "I have
a hammer, how do I use it to inflate my tire?"

Not an April fools gag, I'm just new to decorators and google brings
up lots of discussions from the past on how they may be implemented in
the future, but not much in actually how they work or how to use them.
They don't seem to be documented well at the present, possibly because
the syntax and or function of them isn't completely decided on. I've
been able to figure out the basic principle from the examples I've
found, but that doesn't mean there isn't more possibilities I haven't
found yet.
Assuming you're serious, why not use one of the many testing technologies
actually designed for it, and tap into the associated body of knowledge on
how to accomplish various tasks? Start with what you're trying to do, then
work on how to do it.

I'm trying to understand the use's, limits, and possibilities of
decorators.

It just occurred to me that wrapping the contents of a function vs
wrapping the function it's self, could be useful.

Ron
 
J

Jeremy Bowers

Not an April fools gag, I'm just new to decorators and google brings
up lots of discussions from the past on how they may be implemented in
the future, but not much in actually how they work or how to use them.

OK, just checking :)

A decorator is completely equivalent in principle to

def function():
pass
function = decorator(function)

That's a simplified form; decorators can themselves be an expression which
returns a callable that can be applied to a function and the rule for
applying several in sequence work as you'd expect (pipelining earlier
results into later ones, making for a great Obfuscated Python entry or
two based on the "function name misdirection" trick), but this simplified
form captures the essense, which is what I think you're looking for. In
particular, it's just "syntax sugar", not a "special feature".
I'm trying to understand the use's, limits, and possibilities of
decorators.

It just occurred to me that wrapping the contents of a function vs
wrapping the function it's self, could be useful.

Decorators, literally, can only wrap functions. You can write a wrapper
then that does something to the arguments, which people sometimes do, but
you can't directly "wrap" the arguments.

Note, having shown you how decorators work, you can "manually" apply the
decorator yourself:

Python 2.3.5 (#1, Mar 3 2005, 17:32:12)
[GCC 3.4.3 (Gentoo Linux 3.4.3, ssp-3.4.3-0, pie-8.7.6.6)] on linux2
Type "help", "copyright", "credits" or "license" for more information..... print args, kwargs
.... return "My Wrapper", string._join(*args, **kwargs)
....
string.join = joinWrap
string.join(["1","2","3"], "|")
My Wrapper (['1', '2', '3'], '|') {}
'1|2|3'
So, whatever it is you are trying can do can still be done without the
decorator syntax, and *this* is not unheard of, though managing the
references correctly can be tricky the first few times if you're not used
to it. (Note the replaced function (join in this example) can go anywhere
the wrapper can get at it, I just stick it back in the original module for
simplicity.) It's not the first thing I reach for, in fact in all my
testing code I don't think I ever do this, but it is in the toolbox.

Do this instead of abusing the decorator syntax; you could write a
decorator that tries to figure out if it's being run in a testing
environment and conditionally affects the function, but that's probably a
bad idea.

Feeling-like-I-owed-you-an-answer-after-the-april-fool-accusation-ly yrs,
Jeremy Bowers
:)
 
R

Ron_Adam

OK, just checking :)

A decorator is completely equivalent in principle to

def function():
pass
function = decorator(function)

This helped some. Thanks.
That's a simplified form; decorators can themselves be an expression which
returns a callable that can be applied to a function and the rule for
applying several in sequence work as you'd expect (pipelining earlier
results into later ones, making for a great Obfuscated Python entry or
two based on the "function name misdirection" trick), but this simplified
form captures the essense, which is what I think you're looking for. In
particular, it's just "syntax sugar", not a "special feature".

Are you sure? There appears to be some magic involved with these,
things happening under the hood with argument passing.

def decorate(function):
def wrapper(args):
print 'args' = args
return function(args)
return wrapper

@decorate
def func(s):
print s

func('hello')

In this example, how does wrapper get the correct arguments? This
leads me to believe what I'm looking for is possible, yet in this case
there isn't any way to pass, new arguments to the wrapper without
loosing the original ones.

Wait a min, hold the phone.. Eureka! :) I just figured how to do it.

(after trying it in idle)

def append_arg(n_args):
def get_function(function):
def wrapper(args):
return function(args+'-'+n_args)
return wrapper
return get_function

@append_arg('goodbye')
def func(s):
print s

func('hello')

prints:

hello-goodbye

Ok, this isn't a very useful example, but it demonstrates something
important. That, there seems to be a stack involved in the argument
passing of nested defined functions. Any arguments passed in the
decorators get puts on top of the stack. And nested functions pull
them back off. Does this sound right?

I still feel it can be simplified a bit. These aren't easy to
understand, and having to nest functions like this adds to the
confusion. possibly being able to get the argument "stack", as it
appears to be, directly in the first level could make things a lot
easier.

Feeling-like-I-owed-you-an-answer-after-the-april-fool-accusation-ly yrs,
Jeremy Bowers
:)

Thanks, it helped. :)
 
G

George Sakkis

It turns out it's not a "how to inflate tires with a hammer" request;
I've actually written an optional type checking module using
decorators. The implementation details are not easy to grok, but the
usage is straightforward:

from typecheck import *
@returns(listOf(int, size=3))
@expects(x=str, y=containerOf(int))
def foo(x,y):
return [len(x)] + list(y)
foo('1',[2,3]) [1, 2, 3]
foo('1',(2,3)) [1, 2, 3]
foo(1,[2,3])
Traceback (most recent call last):
....
TypeError: str expected (int given)
Traceback (most recent call last):
....
TypeError: container said:
Traceback (most recent call last):
....
TypeError: container of size 3 expected ([1, 2, 3, 4] given)


George
 
R

Ron_Adam

It turns out it's not a "how to inflate tires with a hammer" request;
I've actually written an optional type checking module using
decorators. The implementation details are not easy to grok, but the
usage is straightforward:

from typecheck import *
@returns(listOf(int, size=3))
@expects(x=str, y=containerOf(int))
def foo(x,y):
return [len(x)] + list(y)

Hi George,

I wrote one like that too yesterday once I figured out how to pass the
arguments, except not with 'conainerof' emplemented. That's a nice
touch.
# some stuff
return an_int, an_list

It checks both the inputs and returns for types and number of items.
And gives error appropriate for both.

Next, would be to surround the 'def define' statements with an
"if __debug__: " statement so it can be turned off for the final
version. I wonder if a decorator, that just passes values straight
though, gets optimized out or not? Or if there's another way to turn
of a decorator?

I also have a test_type function that can be put inline that tries to
adapt the value before it gives an error.

These are interesting problems to solve and go into my tool box,
although I don't have a need for them at the moment. :)

Cheers,
Ron
 
G

George Sakkis

Yes, it is possible to turn off type checking at runtime; just add this
in the beginning of your define:

def define(func):
if not ENABLE_TYPECHECKING:
return lambda func: func
# else decorate func

where ENABLE_TYPECHECKING is a module level variable that can be
exposed to the module's clients. In my module, the default is
ENABLE_TYPECHECKING = __debug__.


George
 
R

Ron_Adam

Yes, it is possible to turn off type checking at runtime; just add this
in the beginning of your define:

def define(func):
if not ENABLE_TYPECHECKING:
return lambda func: func
# else decorate func

where ENABLE_TYPECHECKING is a module level variable that can be
exposed to the module's clients. In my module, the default is
ENABLE_TYPECHECKING = __debug__.


George

Cool, I'll try that.

Thanks,
Ron
 
G

George Sakkis

def define(func):
if not ENABLE_TYPECHECKING:
return lambda func: func
# else decorate func

A small correction: The argument of the decorator is not 'func' but the
parameter checks you want to enforce. A template for define would be:

def define(inputTypes, outputType):
if not ENABLE_TYPECHECKING:
return lambda func: func
def decorate(func):
def typecheckedFunc(*args,**kwds):
##### TYPECHECK *args, **kwds HERE #####
r = func(*args,**kwds)
##### TYPECHECK r HERE #####
return r
return typecheckedFunc
return decorate


Depending on how much flexibility you allow in inputTypes, filling in
the typechecking logic can be from easy to challenging. For example,
does typechecking have to be applied in all arguments or you allow
non-typechecked aruments ? Can it handle *varargs and **kwdargs in the
original function ? An orthogonal extension is to support 'templated
types' (ala C++), so that you can check if something is 'a dict with
string keys and lists of integers for values'. I would post my module
here or the cookbook but at 560 (commented) lines it's a bit long to
qualify for a recipe :)

George
 
R

Ron_Adam

A small correction: The argument of the decorator is not 'func' but the
parameter checks you want to enforce. A template for define would be:

def define(inputTypes, outputType):
if not ENABLE_TYPECHECKING:
return lambda func: func
def decorate(func):
def typecheckedFunc(*args,**kwds):
##### TYPECHECK *args, **kwds HERE #####
r = func(*args,**kwds)
##### TYPECHECK r HERE #####
return r
return typecheckedFunc
return decorate

This is the same pattern I used except without the enable/disable at
the top.

The inline type check function also checks for TYPECHECK == True, and
TYPESTRICT == False, as default to determine the strictness of the
type checking wanted. Where TYPESTRICT == True, causes it to give an
error if they are not the correct type, even if they are the exact
value. TYPESTRICT == False, result in it trying to convert the object,
then checks it, by converting it back to the original type. If it's
still equal it returns the converted object in the specified type.
Depending on how much flexibility you allow in inputTypes, filling in
the typechecking logic can be from easy to challenging. For example,
does typechecking have to be applied in all arguments or you allow
non-typechecked aruments ? Can it handle *varargs and **kwdargs in the
original function ? An orthogonal extension is to support 'templated
types' (ala C++), so that you can check if something is 'a dict with
string keys and lists of integers for values'. I would post my module
here or the cookbook but at 560 (commented) lines it's a bit long to
qualify for a recipe :)

George

Sounds like your version does quite a bit more than my little test
functions. :)

I question how far type checking should go before you are better off
with a confirmtypes() function that can do a deep type check. And then
how much flexibility should that have?

My view point is that type checking should be available to the
singleton types, with conversions only if data integrity can be
insured. ie.. the conversion is reversible with an "identical" result
returned.

def type_convert( a, t):
b = t(a)
aa = type(a)(b)
if a == aa:
return b
else:
raise TypeError

In cases where a conversion is wanted, but type checking gives an
error, an explicit conversion function or method should be used.

In containers, and more complex objects, deep type checking should be
available through a general function which can compare an object to a
template of types, specific to that object. It's important to use a
template instead of a sample, because a sample could have been
changed.

It's all about protecting the data content with a high degree of
confidence. In general, 98% of the time the current python way would
be adequate, but those remaining 2% are important enough to warrant
the additional effort that type checking takes.

On another note, there's the possibility that type checking in python
source code could make writing a compiler easier.

Another idea is that of assigning a name a type preference. And then
overload the assign operators to check for that first before changing
a name to point to a new object. It could probably be done with a
second name dictionary in name space with {name:type} pairs. With that
approach you only need to give key variables a type, then they keep
that type preference until it's assigned a new type, or removed from
the list. The down side to this is that it could slow things down.

Cheers,
Ron
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,232
Messages
2,571,168
Members
47,803
Latest member
ShaunaSode

Latest Threads

Top