doctest, unittest, or if __name__='__main__'

[john_sips_tea]

For writing testcode, it looks like there's three ways that it's
typically done:

(1). using the doctest module,

(2). using the unittest module (i.e. "pyunit"), or else

(3). just putting an "if __name__ = '__main__':" at the bottom of your
module containing code to manually run your class through its paces.

So, which way is preferred? Seems to me that unittest is the way to go,
at least because it has you separate your test code from the code being
tested.

If unittest is the standard way to write test code, why do we still
have doctest? (I notice there's no mention in PEP 3000 of deprecating
the doctest module).

 

[Fredrik Lundh]

(e-mail address removed) wrote

For writing testcode, it looks like there's three ways that it's
typically done:

(1). using the doctest module,

(2). using the unittest module (i.e. "pyunit"), or else

(3). just putting an "if __name__ = '__main__':" at the bottom of your
module containing code to manually run your class through its paces.

So, which way is preferred? Seems to me that unittest is the way to go,
at least because it has you separate your test code from the code being
tested.

If unittest is the standard way to write test code, why do we still
have doctest?

because it's a vastly superior way to write tests ?

(I notice there's no mention in PEP 3000 of deprecating
the doctest module).

why is it that comp.lang.python is suddenly full of folks who
want to deprecate anything they don't understand ? don't
you have better things to do with your time ?

</F>

 

[Christoph Zwerschke]

If unittest is the standard way to write test code, why do we still
have doctest? (I notice there's no mention in PEP 3000 of deprecating
the doctest module).

Because both have their pros and cons and their right to exist. Doctest
is really easy to use and you can kill two birds with one stone.

Rather than removing one of the two, I would like to see yet another
alternatise such as py.test in the standard lib, because unittest is
indeed standard, but clumsy and un-pythonic.

-- Christoph

 

[john_sips_tea]

because it's a vastly superior way to write tests ?

Well, since this is a mailing list where this sort of thing is
discussed, and since I was was asking about which way
is preferred, could you tell us why you (presumably) think
doctest is vastly superior to unittest?

why is it that comp.lang.python is suddenly full of folks who
want to deprecate anything they don't understand ?

I didn't say I wanted to deprecate it. I wrote that I noticed
there was no mention in PEP 3k about deprecating it.

don't you have better things to do with your time ?

That wasn't very friendly.

Anyhow, I'm not attacking doctest, but rather, just trying
to understand why there seems to be two very similar ways
of testing your code built into the system (considering that
one motto around here is "There should be one -- and
preferably only one -- obvious way to do it.").

 

[john_sips_tea]

and you can kill two birds with one stone.

By that, do you mean you can write your tests and your
docstrings in one shot with doctest?

I would like to see yet another alternatise such as py.test
in the standard lib,

Ah: http://codespeak.net/py/current/doc/home.html

 

[Tim Peters]

For writing testcode, it looks like there's three ways that it's
typically done:

(1). using the doctest module,

(2). using the unittest module (i.e. "pyunit"), or else

(3). just putting an "if __name__ = '__main__':" at the bottom of your
module containing code to manually run your class through its paces.

So, which way is preferred?

Serious testers often use both #1 and #2, depending on the specific
test at hand. #3 doesn't work well in practice.

Seems to me that unittest is the way to go, at least because it has
you separate your test code from the code being tested.

Equally true of #1, if you write doctest files separate from your
code. I suggest you look at Zope3, which is a major project using
that style extensively. In fact, it's standard practice there to
write "tutorial doctest" files for a new feature before writing code
to implement it. The text serves as a tutorial introduction to the
feature, and the tests scattered throughout clarify, illustrate, and
verify the text's claims. By writing the text in ReST format, these
doctest files also serve as the source for auto-generated
documentation in HTML format (available interactively from a Zope3
console). This combination of virtues has proved extremely
productive.

BTW, doctest makes it a little easier to write prose than to write
Python code, and your typical frazzled programmer is highly influenced
by small biases ;-) The reverse is true of raw unittest, which goes a
long way toward explaining why you see so many unittests in real life
that are incomprehensible: when it's easier to write code than
explanations, that's what people do.

If unittest is the standard way to write test code, why do we still
have doctest?

Because doctest is much better for some kinds of testing. See above.

(I notice there's no mention in PEP 3000 of deprecating
the doctest module).

Or of deprecating the unittest module ;-)

 

[Christoph Zwerschke]

Anyhow, I'm not attacking doctest, but rather, just trying
to understand why there seems to be two very similar ways
of testing your code built into the system (considering that
one motto around here is "There should be one -- and
preferably only one -- obvious way to do it.").

I don't think doctest and unittest are similar ways; they are pretty
different ways. Also, the "one obvious way" should not be overstressed
and taken as a dogma for libraries, modules and frameworks. Actually, on
that level we see the opposite in the Python world: A plethora of tools
for XML processing, GUI frameworks, Web frameworks etc. Some think this
is deplorable, but I see this as something exciting and positive.

-- Christoph

 

[Christoph Zwerschke]

By that, do you mean you can write your tests and your
docstrings in one shot with doctest?

Right. The tests serve also as usage examples.

 

[Terry Hancock]

Because both have their pros and cons and their right to
exist. Doctest is really easy to use and you can kill two
birds with one stone.

Rather than removing one of the two, I would like to see
yet another alternatise such as py.test in the standard
lib, because unittest is indeed standard, but clumsy and
un-pythonic.

e.g....
Doctest is very easy to use, so it's easy to ensure that
tests get written. OTOH, they're only really good for stuff
that can be easily tested in the interpreter (e.g. that can
be easily verified from their text output). When you get
into stickier stuff like graphics and web programming, the
formal structure of pyunit can be easier to adapt than
something which is intrinsically based on string processing.

Haven't seen py.test before, but I'm looking now -- thanks
for the link.

Cheers,
Terry

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

For writing testcode, it looks like there's three ways that it's
typically done:

Actually, there's at least one more

(1). using the doctest module,

(2). using the unittest module (i.e. "pyunit"), or else

(3). just putting an "if __name__ = '__main__':" at the bottom of your
module containing code to manually run your class through its paces.

(4) using pytest

So, which way is preferred? Seems to me that unittest is the way to go,
at least because it has you separate your test code from the code being
tested.

Which may or may not be such a good thing... I prefer to have my tests
with the code they're testing.

If unittest is the standard way to write test code, why do we still
have doctest?

Because doctest is simple, great, and definitively more pythonic than
unittest ?

(I notice there's no mention in PEP 3000 of deprecating
the doctest module).

Hopefully not.

 

[Fredrik Lundh]

Well, since this is a mailing list where this sort of thing is
discussed, and since I was was asking about which way
is preferred, could you tell us why you (presumably) think
doctest is vastly superior to unittest?

doctest comes with extensive documentation, which, among other
things, discuss the advantages of using doctest. have you read the
documentation ?

http://www.python.org/doc/lib/module-doctest.html

That wasn't very friendly.

so you think that a "why all this creativity when you could just
standardize on something ported from java, and throw away every-
thing else" post is friendly ? really ?

Anyhow, I'm not attacking doctest, but rather, just trying
to understand why there seems to be two very similar ways
of testing your code built into the system (considering that
one motto around here is "There should be one -- and
preferably only one -- obvious way to do it.").

if you think that motto means that there should be only one non-
deprecated solution to any higher-level problem, you've completely
and utterly missed the point.

</F>

 

[Fredrik Lundh]

Doctest is very easy to use, so it's easy to ensure that
tests get written. OTOH, they're only really good for stuff
that can be easily tested in the interpreter (e.g. that can
be easily verified from their text output). When you get
into stickier stuff like graphics and web programming, the
formal structure of pyunit can be easier to adapt than
something which is intrinsically based on string processing.

that's puzzling. I do lots of graphics, and some web programming (my
company's main product is a web application server for graphics), and
we use doctest for everything.

I find it a bit odd that someone thinks that there are tests for which

assert something == somevalue

is easier to write than either
somevalue

or
True

</F>

 

[Fredrik Lundh]

Equally true of #1, if you write doctest files separate from your
code.

newer versions of doctest have more extensive support for this, but the old
trick of using doctest to test your *test programs* work remarkably well.

In fact, it's standard practice there to write "tutorial doctest" files for a
new feature before writing code to implement it.

another good practice is to add regression doctests for each verified bug,
*before* attempting to fix it.

</F>

 

[Max M]

and you can kill two birds with one stone.

By that, do you mean you can write your tests and your
docstrings in one shot with doctest?

Exactly.

>>> '\n'.join(['Doctests are absolutely brilliant!'] * 100)

They combine two brilliant ideas that are hard to do in practice.
"Testing" and "Literate Programming"

In the process it even manages to make both a lot easier.


--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science

Phone: +45 66 11 84 94
Mobile: +45 29 93 42 96

 

[john_sips_tea]

so you think that a "why all this creativity when you could just

standardize on something ported from java, and throw away every-
thing else" post is friendly ? really ?

If unittest is the standard way to write test code, why do we still
have doctest? (I notice there's no mention in PEP 3000 of deprecating
the doctest module).

when that should've been

If one is the standard way to write test code, why do we still
have the other one? (I notice there's no mention in PEP 3000 of
deprecating either module).

Also, as an aside, I don't think there's anything inherently wrong
with porting something from Java (or Smalltalk, or any other
language). Such ported software may or may not be a good fit
for Python.

 

[Marc 'BlackJack' Rintsch]

For writing testcode, it looks like there's three ways that it's
typically done:

(1). using the doctest module,

(2). using the unittest module (i.e. "pyunit"), or else

(3). just putting an "if __name__ = '__main__':" at the bottom of your
module containing code to manually run your class through its paces.

So, which way is preferred? Seems to me that unittest is the way to go,
at least because it has you separate your test code from the code being
tested.

If unittest is the standard way to write test code, why do we still
have doctest? (I notice there's no mention in PEP 3000 of deprecating
the doctest module).

I see it this way: `unittest` is for testing code and `doctest` is for
testing examples in docstrings. If you put all your tests in the
docstrings there will be much code that's quite irrelevant for documenting
the function/method/class.

Testing real examples in doctstrings, or external documentation like
tutorials, is important because it's very frustrating for people reading
the docs if the examples don't work as advertised.

Ciao,
Marc 'BlackJack' Rintsch

 

[Fredrik Lundh]

I see it this way: `unittest` is for testing code and `doctest` is for
testing examples in docstrings. If you put all your tests in the
docstrings there will be much code that's quite irrelevant for documenting
the function/method/class.

so put the tests in your test program instead, and use doctest to make
sure that the test program works as advertised.

(or use doctest on separate test scenario documents)

if you prefer the doctest approach, there's no reason to use unittest for
anything.

</F>

 

[lollipopenator]

Hi there Christopher, I was wondering if you (or anyone reading this )
could quickly summarize the ways in which unittest is unpythonic, or
point me to somewhere which discusses this.

Is this 'consensus opinion' or mainly your own opinion?

Is there a summary somewhere (in addition to the Zen of Python thingy)
of what kinds of things are 'pythonic' and why they are considered so?
I see it referred to a lot, and am starting to get a feel for it in
some areas but not others.

I am fairly newbie-ish, having been actively working as a python
programmer for the last 6 months, although I did first learn the basics
several years ago.


I have noticed some distinctly funny and confused feelings I get when
using the unittest module, stuff that feels clunky and odd about how it
is set-up, however I thought that this was just due to *my personal*
lack of understanding of the deep magic and sophisticated design
patterns used in this module!

If it turns out to be 'unpythonic' then I can blame the funny feelings
on that and sigh in a smug and vaguely superior way whenever I get them


I have not yet used the doctest module but would like to begin soon




I am also very interested in py.lib, just looked at the website and
will be keeping an eye on it.

One of the things I am doing a lot of at the moment is documentation
and testing, hence the interest!

Wendy Langer

 

[Benji York]

When you get
into stickier stuff like graphics and web programming, the
formal structure of pyunit can be easier to adapt than
something which is intrinsically based on string processing.

As pointed out earlier in this thread, Zope 3 has embraced doctests in a
big way (and they've paid off in a big way too). Obviously we test lots
of web apps and had to come up with something to allow us to do decent
doctests of them. Unsurprisingly it's documented with a doctest:
http://svn.zope.org/*checkout*/Zope3/trunk/src/zope/testbrowser/README.txt

 

[Paul Rubin]

I have noticed some distinctly funny and confused feelings I get when
using the unittest module, stuff that feels clunky and odd about how it
is set-up, however I thought that this was just due to *my personal*
lack of understanding of the deep magic and sophisticated design
patterns used in this module!

If it turns out to be 'unpythonic'

The unpythonicness stems from it being basically a reimplementation of
JUnit, which was designed for use with Java.