Testing Java

[Rhino]

I'm worried that my knowledge of industry standard testing procedures for
Java is getting a bit out of date.

Can anyone refer me to a website that focuses on testing for Java code and
that describes the best practices and tools for doing that testing?

I am a one-man development team at the moment and I want to be sure that my
code is going to be considered thoroughly tested when I eventually release
it.

Some guidance on what should be tested and how thoroughly it should be
tested - as well as what does NOT need to be tested - would be very helpful.

 

[markspace]

I'm worried that my knowledge of industry standard testing procedures for
Java is getting a bit out of date.

Can anyone refer me to a website that focuses on testing for Java code and
that describes the best practices and tools for doing that testing?
http://en.wikipedia.org/wiki/Software_testing



I am a one-man development team at the moment and I want to be sure that my
code is going to be considered thoroughly tested when I eventually release
it.

Some guidance on what should be tested and how thoroughly it should be
tested - as well as what does NOT need to be tested - would be very helpful.

Test everything.

 

[Mike Schilling]

Test everything.

And write automated tests, so that you can rerun them easily to catch
regressions. Don't worry about the time required to create and automate the
tests -- it will more more than pay for itself in the end.

 

[Martin Gregorie]

I am a one-man development team at the moment and I want to be sure that
my code is going to be considered thoroughly tested when I eventually
release it.

I think the biggest win comes from automated regression testing.

IME its well worth the effort of writing a scriptable regression test
harness for any chunk of code that's more complex than, say, a package
that writes and parses CSV files. Done right, the code is relatively
simple and can be used to implement a whole bunch of tests merely by
writing a well thought out set of scripts.

Think about capturing human-readable test output so that the test harness
can compare this with the expected results. Design the scripts to be self
documenting and easy to understand: allowing comments in them is an
enormous help. Design the harness so its output lists the script with
interspersed test results. This makes debugging much easier.

Ideally it will output a single line reporting a successful test and show
understandable diagnostics if the test fails. Just comparing test output
with expected output and displaying the differences can do this if you
design the test output to be readable.

A well written regression test setup will accept a single, simple command
will that runs the entire set of tests and produces a small, unambiguous
report that can indicate pass/fail at a glance. Do this and you'll tend
to run regression tests after any and all code changes. The effect on
code reliability can be dramatic.

Write the tests from the code specifications. The effort of writing a
clear specification is never wasted. Write it hierarchically so it has an
overview, class and method descriptions. This pays off in:
- better code
- more complete test coverage
- the specification can be turned into javadocs.

Some guidance on what should be tested and how thoroughly it should be
tested - as well as what does NOT need to be tested - would be very
helpful.

See above, but you need a decent specification as the basis for test
coverage. Develop a test set that covers all specified functions in terms
of correct functioning, corner cases and error handling. Its essential to
write tests from the specification, not the code or you'll just end up
testing what the code does, not what its meant to do. There is a good
case for writing the test harness and scripts before you start coding
because this way you may spot errors in the specification and will
certainly avoid writing tests for the code you actually wrote.

Never write code that can't be tested. Design it with this in mind.

 

[Joshua Cranmer]

IME its well worth the effort of writing a scriptable regression test
harness for any chunk of code that's more complex than, say, a package
that writes and parses CSV files.

After having found a bug in a simple getter, I've found it easier to
recommend to always write a test for everything, no matter how simple it is.

Also, any time you find and fix a bug, add a check for it in your test
suite to make sure you don't regress the bug fix.

 

[Arne Vajhøj]

I'm worried that my knowledge of industry standard testing procedures for
Java is getting a bit out of date.

Can anyone refer me to a website that focuses on testing for Java code and
that describes the best practices and tools for doing that testing?

I am a one-man development team at the moment and I want to be sure that my
code is going to be considered thoroughly tested when I eventually release
it.

Some guidance on what should be tested and how thoroughly it should be
tested - as well as what does NOT need to be tested - would be very helpful.

What type of testing.

If unit testing look for JUnit.

For other types of testing you may need other tools.

Arne

 

[Arne Vajhøj]

After having found a bug in a simple getter, I've found it easier to
recommend to always write a test for everything, no matter how simple it
is.

Very good point.

Before IDE's generated getters/setters it occasionally happened that
those were copy pasted, method name modified but field name was
not modified.

Also, any time you find and fix a bug, add a check for it in your test
suite to make sure you don't regress the bug fix.

Also very good point.

If one person at some point in the past did not think of a
potential problem, then there is a much higher risk than
usual for that during some heavy refactoring another person
will not think of it either.

Arne

 

[Martin Gregorie]

After having found a bug in a simple getter, I've found it easier to
recommend to always write a test for everything, no matter how simple it
is.

That goes without saying - I wasn't being entirely clear. What I meant is
that its not worth writing a test harness for a very simple class, e.g.
one with nothing but getters, setters and toString() methods. However, if
a class or group of co-operating classes is complex enough to deserve a
test harness, then you must test all its methods. If a method in such a
class 'doesn't need testing' then you should be asking whether the method
is really necessary and remove it if it isn't.

Also, any time you find and fix a bug, add a check for it in your test
suite to make sure you don't regress the bug fix.

Yes. The set of test scripts should be extended if it misses bugs and
*must* be extended if extra functionality or methods are added to the
class.

 

[Jean-Baptiste Nizet]

Rhino a écrit :

I'm worried that my knowledge of industry standard testing procedures for
Java is getting a bit out of date.

Can anyone refer me to a website that focuses on testing for Java code and
that describes the best practices and tools for doing that testing?

My advice would also be to test everything.
Now for the tools: JUnit and TestNG are certainly the most used test
frameworks.

I use Cobertura, which is free, to measure the code coverage of the
tests. You might also like Emma or Clover (but this last one isn't free).

To test web applications, I use HtmlUnit, but there are other tools
(Selenium is probably one of the most used tools).

Finally, these are not really testing tools, but they help avoiding
bugs, bad practices and ugly code: PMD, Checkstyle, and Findbugs are
great tools.

All of them can easily be included in your build. See
http://code.google.com/p/memwords/source/browse/trunk/ant/build.xml for
an example of an ant build which integrates all these tools.

I should also mention that if your application and/or your team becomes
larger, running the tests frequently can become tedious. You might then
add a continuous integration tool like Hudson to your toolbox.

JB.

 

[Tom Anderson]

Also very good point.

If one person at some point in the past did not think of a
potential problem, then there is a much higher risk than
usual for that during some heavy refactoring another person
will not think of it either.

Something i say that's just about applicable here is "If something can
break, test it. If something can't possibly break, definitely test it.".
When you make assumptions like that, you need to check them, and you'll
get bitten if you don't.

tom

 

[Rhino]

I'm worried that my knowledge of industry standard testing procedures for
Java is getting a bit out of date.

Can anyone refer me to a website that focuses on testing for Java code and
that describes the best practices and tools for doing that testing?

I am a one-man development team at the moment and I want to be sure that
my code is going to be considered thoroughly tested when I eventually
release it.

Some guidance on what should be tested and how thoroughly it should be
tested - as well as what does NOT need to be tested - would be very
helpful.

Thanks everyone for your replies to my question, especially Jean-Baptiste
Nizet who gave me several specific tools to examine.

I've been using JUnit for some time but the other tools he mentioned look
like they will be very useful as I further expand the testing I do.