Unit Testing Examples

[Tom Verbeure]

Hello All,

so I'm now convinced that unit testing is really the way to go and I
want to add it to an existing project. The first thing that I find
out, is that, well, it's hard work. Harder than writing the real
functional code.

My application manipulates graphs. I first write the skeleton in
Python, then convert it to C++ for high performance (some graphs can
have millions of nodes.)
Up to now, I simply evaluated the validity of a graph by eye-balling
the results of toy problems. Obviously, this gets harder as the graphs
grow, so unit testing sounds like a good way to make sure I catch
mistakes early.

I'm not sure, though, at what level I have to write unit tests. It's
easy to write a tests that validates the consistency of a single node
of a graph, but the real complexity is, of course, in the interaction
of multiple nodes.

What I'm looking for is some examples, existing open source projects
that make heavy use of unit testing. I'd like to see a real life
example about how those tests are structured and how the complexity of
these tests expands from object level to system level.

Thanks,
Tom

 

[A.M. Kuchling]

I'm not sure, though, at what level I have to write unit tests. It's
easy to write a tests that validates the consistency of a single node
of a graph, but the real complexity is, of course, in the interaction
of multiple nodes.

Not too long ago I wrote a set of unit tests for a graph data type;
unfortunately the code belongs to my employer and isn't open source, so you
can't see a copy. What I did was:

* Test basic things: create an empty graph and check that it has zero nodes
and edges. Add a vertex and edge; remove the vertex and edge.
In my case there are various accessors for .number_of_nodes()
and .number_of_vertices(), so I checked that they went up by one.
* Create a little graph, ~5 nodes, and test the BFS and DFS methods to see
that they output nodes in a sensible order.

Sancho, the testing framework I use, provides code coverage measurement, so
I used that to find which lines of code weren't exercised by the test suite
and added tests to cover more lines. It's usually difficult to exercise
100% of the code, because error cases can be difficult to provoke,
especially if they represent an internal invariant being broken or an
external resource such as a file or server not being available, and some
trivial accessors aren't worth testing, so I'm usually happy with 70%-80%
coverage as long as the complicated methods are completely or mostly
covered.

In my case I'm not bothering to run a very large graph as a test case,
because the little graph exercises all of the relevant branches; I'm
therefore betting that the code is OK for larger graphs, too. If a large
graph demonstrates a bug someday, I would try to boil it down to a small
test case and add this case to the test suite.

If performance is important, you might consider adding a performance test:
run an algorithm on a large graph and check that the running time is
suitably fast. This might save you from making some change that doesn't
affect correctness but results in a sizable performance regression.

--amk

 

[Nicholas Wieland]

snip
What I'm looking for is some examples, existing open source projects
that make heavy use of unit testing. I'd like to see a real life
example about how those tests are structured and how the complexity of
these tests expands from object level to system level.

IMHO Twisted Matrix is a good example.
www.twistedmatrix.com

 

[John J. Lee]

so I'm now convinced that unit testing is really the way to go and I
want to add it to an existing project. The first thing that I find
out, is that, well, it's hard work. Harder than writing the real
functional code.

Graphs aside, it's easier 1. when you write them before you write the
code 2. when you get some practice (of course).

[...]

I'm not sure, though, at what level I have to write unit tests. It's
easy to write a tests that validates the consistency of a single node
of a graph, but the real complexity is, of course, in the interaction
of multiple nodes.

[...]

Test at both levels. Also, even for a graph object containing nodes
(if that's how it's written), it can make sense to test the graph
object independently of the nodes. Sometimes you look at tests like
that and think "how could that possibly fail?", but it's surprising
how often they do if you do any refactoring (and even bug fixing,
sometimes). This is related to "mock objects", though I think mock
objects are worth avoiding if they don't turn out to be necessary.

I certainly don't like to put this forward as a beacon of good
practice, but I was quite unhappy with the tests in ClientForm (too
long, too apparently repetitive and trivial), *until* I wanted to port
it from an event-based HTML parser to a DOM. I was amazed at the
number of tests that failed in subtle ways. If I didn't have the
tests, I would never have finished it, I'm quite sure, and the tests I
thought were not pulling their weight turned out to be valuable.
Elegant graph algorithms and messy W3C standards (both HTML DOM and
HTML are quite nasty) are different things, and the latter has more to
benefit from unit tests, but I think it's worth experimenting with the
more paranoid end of unit testing. One thing I don't do and I'm quite
sure is valuable is code coverage measurement, so if I were starting
afresh I'd be attracted to amk's framework (though there are several
tools to do it with unittest -- they're just not integrated to make it
trivial).

As a BTW, one interesting thing about moving from an event parser to a
DOM (and kept the old implementation: the event-based code is still in
use) was that the unit-test nature of some of the tests was actually a
hindrance to getting the tests running, and a barrier to
maintainability. Necessary background: the module parses HTML to get
a list of HTMLForm objects; HTMLForms contain Control objects
(controls are things like text entry boxes, buttons, menus,
checkboxes). Now, my tests of the various Controls constructed
Controls directly rather than using the parser -- these were supposed
to be unit tests, after all, so I didn't want to be testing the
Controls *and* the parser at the same time. The problem, of course,
was (still is, I haven't fixed it yet) that when I started using the
DOM, the constructors for the Control objects changed completely, and
I had to fork the test code in a nasty way just to get the tests to
run. If I'd just been lazy and used the parser interface to build
Control objects, I could have run the tests with almost no changes.
Not entirely sure what the moral of this story is yet... maybe just.


John

 

[Thomas Guettler]

Am Mon, 17 Nov 2003 09:10:32 -0800 schrieb Tom Verbeure:

Hello All,

so I'm now convinced that unit testing is really the way to go and I
want to add it to an existing project. The first thing that I find
out, is that, well, it's hard work. Harder than writing the real
functional code.

If you are using objects which are containers for
other objects, you could do it like this:

Every object gets a method called "unittest". This performs
a unittest on its own data, and then calls unittest() for
all child objects.

This way you can check all objects by calling unittest for the root object,
or call the method for only a part.

Be sure to do no recursion if you have circular data.

thomas

 

[Tom Verbeure]

Thanks all for your suggestions!

Tom

 

[David Jaquay]

Thanks all for your suggestions!

Tom

Sorry to be late to get to this thread, but I have a Python O/S
project at sourceforge.net/projects/d-rose that strives to have
complete unit testing, and is a simulation that uses/is-based-on a
graph/network. Hope this is of some help to you as an example, and
feel free to ask if you have any questions. I certainly won't claim
that mine is the best example of such, as I'm just learning TDD, too,
but...

Dave