Skipping decorators in unit tests

G

Gilles Lenfant

Hi,

(explaining the title) : my app has functions and methods (and maybe classes in the future) that are decorated by decorators provided by the standard library or 3rd party packages.

But I need to test "undecorated" functions and methods in my unit tests, preferably without adding "special stuffs" in my target tested modules.

Can someone point out good practices or dedicated tools that "remove temporarily" the decorations.

I pasted a small example of what I heed at http://pastebin.com/20CmHQ7Y

Many thanks in advance
 
C

Cameron Simpson

(explaining the title) : my app has functions and methods (and
maybe classes in the future) that are decorated by decorators
provided by the standard library or 3rd party packages.

But I need to test "undecorated" functions and methods in my unit tests, preferably without adding "special stuffs" in my target tested modules.

Can someone point out good practices or dedicated tools that "remove temporarily" the decorations.
I pasted a small example of what I heed at http://pastebin.com/20CmHQ7Y

Speaking for myself, I would be include to recast this code:

@absolutize
def addition(a, b):
return a + b

into:

def _addition(a, b):
return a + b

addition = absolutize(_addition)

Then you can unit test both _addition() and addition().

And so forth.

Cheers,
--
Cameron Simpson <[email protected]>

1st Law Economists: For every economist there exists an equal and opposite
economist.
2nd Law Economists: They're both always wrong!
 
N

Ned Batchelder

Speaking for myself, I would be include to recast this code:

@absolutize
def addition(a, b):
return a + b

into:

def _addition(a, b):
return a + b

addition = absolutize(_addition)

Then you can unit test both _addition() and addition().

And so forth.

Cheers,

I have to admit I'm having a hard time understanding why you'd need to
test the undecorated functions. After all, the undecorated functions
aren't available to anyone. All that matters is how they behave with
the decorators.

But my imagination is weak: do you mind explaining more about what the
functions do, what the decorators do, and why you need to test the
undecorated functions? I'll learn something, and with more information,
we might be able to find a better solution.

--Ned.
 
T

Terry Reedy

On 10/10/2013 10:00 AM, Gilles Lenfant wrote:

To add to the other two responses so far...
(explaining the title) : my app has functions and methods (and maybe classes in the future) that are decorated by decorators provided by the standard library or 3rd party packages.

But I need to test "undecorated" functions and methods in my unit tests, preferably without adding "special stuffs" in my target tested modules.

Let's assume that the decorator wraps the function in a way that the
wrapper has a reference to the original function, so it does not disappear.
Can someone point out good practices or dedicated tools that "remove temporarily" the decorations.

The easiest thing would be to have the decorator add the original
function as an attribute .wrapped to the wrapper. Then test foo.wrapped.
If you do not like this 'special stuff', then you would have to
introspect the wrapper to access the wrapped function. How to do that
depends on the wrapper.
 
S

Steven D'Aprano

Speaking for myself, I would be include to recast this code:

@absolutize
def addition(a, b):
return a + b

into:

def _addition(a, b):
return a + b

addition = absolutize(_addition)

Then you can unit test both _addition() and addition().

*shudders*

Ew ew ew ew.


I would much rather do something like this:


def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

def decorate(func):
def inner(arg):
return func(arg) + 1
return inner

@decorate
def f(x):
return 2*x



And in use:


py> f(100)
201
py> undecorate(f)(100)
200
 
C

Cameron Simpson

*shudders*
Ew ew ew ew.

Care to provide some technical discourse here? Aside from losing the neat
and evocative @decorator syntax, the above is simple and overt.
I would much rather do something like this:

def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black magic.
And in use:

py> f(100)
201
py> undecorate(f)(100)
200

All lovely, provided you can convince me that undecorate() is robust.
(And if you can, I'll certainly be filing it away in my funcutils
module for later use.)

Cheers,
--
Cameron Simpson <[email protected]>

DRM doesn't inconvenience pirates ¿ indeed, over time it trains
law-abiding users to become pirates out of sheer frustration.
- Charles Stross
 
S

Steven D'Aprano

Care to provide some technical discourse here? Aside from losing the
neat and evocative @decorator syntax, the above is simple and overt.


What part of "Ew ew ew ew" was not technical enough for you? Would it
help if I add a few extra "ew"s?

*wink*

But seriously, I don't like doubling the number of names in the namespace
just for the sake of white-box testing. That means you have to somehow
document each and every one of the private functions that they aren't for
using, just for testing.

For the avoidance of doubt, I understand you flagged them as private. But
even for private use within the module, they're not supposed to be used.
They are only for testing. So now you have to have a naming convention
and/or documentation to ensure that they aren't used internally.

If there really is a good use-case for using both the decorated and
undecorated version of the function (whether internally, or as part of
the public API), then I'm completely with you. We don't have to use
decorator syntax if we have good reason to keep the wrapped and unwrapped
functions separate, just bind them to separate names.

I also like Terry Reedy's suggestion of having the decorator
automatically add the unwrapped function to the wrapped function as an
attribute:

def decorate(func):
@functools.wraps(func)
def inner(arg):
blah blah
inner._unwrapped = func # make it public if you prefer
return inner

which makes it all nice and clean and above board. (I seem to recall a
proposal to have functools.wraps do this automatically...)

I would much rather do something like this:

def undecorate(f):
"""Return the undecorated inner function from function f.""" return
f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black
magic.

Not every one-line function needs to be in the std lib :)

To go into the std lib, it would need to be a tad more bullet-proof. For
instance, it should have better error checking for the case where
func_closure is None, rather than just raise the cryptic error message:

TypeError: 'NoneType' object is not subscriptable

If would also need to deal with arbitrary closures, where item 0 is not
necessarily a function, or where there might be multiple functions, or no
functions at all.

But for purely internal use within a test suite, we can afford to be a
little more ad hoc and just deal with those cases when and if they occur.
Since we're white-box testing, we presumably know which functions are
decorated and which ones aren't (we can read the source code!), and will
only call undecorate on those which are decorated.
All lovely, provided you can convince me that undecorate() is robust.
(And if you can, I'll certainly be filing it away in my funcutils module
for later use.)

It needs error handling. It assumes that the closure only references a
single object, namely the function being wrapped. In that sense, it's not
ready for production as a public utility function. But as a private
function for use only in testing, under controlled conditions, I think it
is robust enough, it works in CPython 2.5 through 2.7 and IronPython 2.6.
In Python 3.x, you have to change func_closure to __closure__, but
otherwise it works in 3.2 and 3.3. Jython 2.5 seems to choke on the
decorator syntax:

File "<stdin>", line 1
@decorate
^
SyntaxError: mismatched input '<EOF>' expecting CLASS


which surely is a bug in Jython. If I apply the decorator manually, it
works.
 
T

Terry Reedy

def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black magic.
And in use:

py> f(100)
201
py> undecorate(f)(100)
200

All lovely, provided you can convince me that undecorate() is robust.
(And if you can, I'll certainly be filing it away in my funcutils
module for later use.)

It only works if the decorator returns a closure with the original
function as the first member (of func_closure). Often true, but not at
all a requirement.
 
T

Terry Reedy

I also like Terry Reedy's suggestion of having the decorator
automatically add the unwrapped function to the wrapped function as an
attribute:

def decorate(func):
@functools.wraps(func)
def inner(arg):
blah blah
inner._unwrapped = func # make it public if you prefer
return inner

which makes it all nice and clean and above board. (I seem to recall a
proposal to have functools.wraps do this automatically...)

The explicit attribute can also be set by class rather than closure
based decorators.
 
T

Terry Reedy

On 11Oct2013 02:55, Steven D'Aprano
def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black
magic.
And in use:

py> f(100)
201
py> undecorate(f)(100)
200

All lovely, provided you can convince me that undecorate() is robust.
(And if you can, I'll certainly be filing it away in my funcutils
module for later use.)

It only works if the decorator returns a closure with the original
function as the first member (of func_closure). Often true, but not at
all a requirement.
 
T

Terry Reedy

On 11Oct2013 02:55, Steven D'Aprano
def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black
magic.
And in use:

py> f(100)
201
py> undecorate(f)(100)
200

All lovely, provided you can convince me that undecorate() is robust.
(And if you can, I'll certainly be filing it away in my funcutils
module for later use.)

It only works if the decorator returns a closure with the original
function as the first member (of func_closure). Often true, but not at
all a requirement.

Another standard decorator method is to write a class with a .__call__
method and attach the original function to instances as an attribute.
(Indeed, decorators were borrowed from class-happy Java ;-). But there
is no standard as to what the function attribute of instances is called.
The OP's request for accessing the function without modifying the tested
code cannot be met in general. One must have access to the tested code.
 
G

Gilles Lenfant

Cameron, Steven, Ben, Ned, Terry, Roy. Many thanks for this interesting discussion.

I ended up... mixing some solutions provided by your hints :

* Adding an "__original__" attribute to the wrapper func in the decorators of my own
* Playing with "func_closure" to test functions/methods provided by 3rd party tools

Cheers and thanks again for taking time to help me.
 
E

Ethan Furman

Care to provide some technical discourse here? Aside from losing the neat
and evocative @decorator syntax, the above is simple and overt.

And completely dismisses the whole point of adding @decorator to the
language: easy to use, easy to see == folks will actually use it.
I would much rather do something like this:

def undecorate(f):
"""Return the undecorated inner function from function f."""
return f.func_closure[0].cell_contents

Whereas this feels like black magic. Is this portable to any decorated
function? If so, I'd have hoped it was in the stdlib. If not: black magic.

Probably black magic. But you can go with the decorator.wrapped route;
after all, you're testing your own stuff so you should have control of
your own decorators (okay, you may have to adapt a few others ;) .
 
C

Cameron Simpson

* Adding an "__original__" attribute to the wrapper func in the decorators of my own

Just one remark: Call this __original or _original (or even original).
The __x__ names are reserved for python operations (like __add__, supporting "+").

Cheers,
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,982
Messages
2,570,190
Members
46,740
Latest member
AdolphBig6

Latest Threads

Top