C++ fluency

[Bart van Ingen Schenau]

It sounds to me like your test are too tightly coupled. You should be
able to run the tests for a module or a class in isolation.

Nah, I think it has more to do with a different understanding of
"multiple modifications".

Suppose I have a Point class that uses cartesian coordinates for its
internal representation. To satisfy a new testcase (for a new
requirement), I find that the internal representation should have been
in polar coordinates. Changing the internal representation in this way
requires me to modify multiple existing member functions. That is what I
mean with multiple modifications, and I see no reason to rerun my
testcases until all existing member functions have been changed to use
the new internal representation. I don't even expect the class to
compile before that.

True when you add test after the fact. But with TDD, test n tests the
function before the line is added, test n+1 tests it after.

Adding that line can have one of three effects:
- It adds a new path through the function
- It adds something that is 100% independent of all the existing code in
the function
- It modifies the function in such a way that at least one existing
testcases becomes invalidated.

In the first case, I would have a testcase as well.
In the second, I might have separate testcases, but I also might have
combined some of the tests in a single testcase.
In the third case, there is no way to have tescases M and N+1 pass at
the same time, because they have conflicting expectations about the
presence of that new line of code. One of those testcases would have to
be removed.

Note that it is not frequent testing or even TDD that I have a problem
with. It is the metric of lines.
How many lines do these two functions have? And how many testcases would
you have written to come to the functions? Do the number of testcases
match the number of lines?

long
divide (long lhs, long rhs)
{
long result;

if (rhs == 0)
{
throw std::domain_error("Division by zero");
}

result = lhs / rhs;
return result;
}

long divide (long lhs, long rhs) {
if (rhs == 0) throw std::domain_error("Division by zero");
return lhs / rhs;
}

Bart v Ingen Schenau

 

[James Kanze]

I have yet to see a successful company that's really doing Agile
development.

For what definition of "agile development"?

More often, "Agile" is an excuse used for the lack of any
formal process.

Most of the places I've seen which have adopted something they
called "agile development" had no process what so ever before.
I'm perfectly willing to believe that just about any of the
various "agile processes" will beat no process. But no process
is far from state of the art, when it comes to quality software
development. All of the places I've seen using something called
"agile development" are far behind state of the art in terms of
quality and productivity, and all of the concrete proposals I've
seen under the name "agile" would represent a serious step
backwards for some of the places where I've worked.

Agile looks good in print only because authors insist on
comparing it with horrible, extinct methodologies that no
longer reflect the way commercial software is developed;

I was in agreement up until you said "the way commercial
software is developed". I fear far too many firms still don't
have any process what so ever.

nobody here is advocating Waterfall, or BDUFs, or any of the
other "Worst Practices" that have been described to you.

I've never seen anyone recommend waterfall. The very term was
invented by people looking for a strawman to knock down.

Scripting language advocates had a similarly nastly habit for
years. They would proudly proclaim all the ways their
language was superior to C, without acknowledging that most
applications were no longer developed in C, anyway.

Realistically, this is usually the case. I've seen a lot of
arguments for some other language, instead of C++, based on how
people wrote C++ twenty years ago.

Normally, I would have hoped that in this forum, we could
discuss technical issues, rather than marketing hype.
Apparently this is not the case. (Note that I don't want to
criticize all of the proponents of TDD in this respect. Some of
them have been exceptionally good at providing concrete,
technical explinations, and I've appreciated that. Regretfully,
it hasn't been the case for all of them.)

 

[James Kanze]

James Kanze wrote:

[...]

Writing tests before writing the code meant to pass them.
This is what I meant to praise, and to distinguish from TDD.

But tests are code. And if the test is to pass, the code it
tests must have been written.

 

[James Kanze]

I haven't worked on anything that needed floating point
testing more than abs(f() - expected) < (tolerance *
expected). That doesn't mean that others haven't found ways
to unit test these things; I just haven't and don't know of
any.

The problem is the non-linearity, and the fact that the
requirements often specify less that 1/2 LSB precision. If your
tests prove that you have this for the input of 1.0, and of 2.0,
you still can't claim it for all of the intermediate values.

 

[James Kanze]

And yet you repeatedly use the same technique, and swap around
the elements.
Where?

You cannot credit that paper with inventing Waterfall. And, on
a small project like an experiment, Waterfall simply means
"Big Design Up Front", which is what you repeatedly espouse
here.

Where?

Once again, you're creating strawman to try and make your ideas
look good. Why don't you try arguing some technical facts for a
change? And sticking to the truth?

 

[James Kanze]

A scientific study of large projects would cost way too much to run.

The bigger the project, the more you should apply XP,
especially. Look up how the Space Shuttle's software is
written. They use pair programming and their version of TDD.

That's not what the people who actually work there say. For
starters, of course, most of the software on the Space Shuttle
was written before anyone had heard of TDD.

For those interested: see the article "NASA's Shuttle Software:
Lives Depend on Its Quality", in
http://lifelong.engr.utexas.edu/sqi/sqi_newsletter/winter97.pdf.
For example: "Testing obviously plays a central role in
producing a quality product, but since we know that exhaustive
testing of any nontrivial software system is impossible, we must
put into place `a defined, controlled, repeatable, predictable
process that will result in a product of known quality,'"

They integrate continuously, and run a humongous test
batch.

Studies of very large projects have shown that the more
carefully they collect all their requirements up front, the
more likely they are to fail.

I suspect that this is just another invention on your part, but
it really doesn't matter, since it's just a strawman anyway.

 

[James Kanze]

OK,I should have said scheduling policy. But I still think
you were being obtuse.

I'm trying to understand what you're saying.

You wrote "you have absolutely no control over when Windows or
Unix does a context switch" I provided you with an example of
how at least one of those does provide such control.

Except that they don't. They define policies, but they
certainly don't allow you to control exactly when the context
swaps occur.

 

[James Kanze]

[ ... ]

Well James isn't!

No, but he lives in France, and has for quite some time.

OTOH, I'm not sure it makes much difference anyway -- every
time I've been to France, I've been treated quite politely as
a rule. I've been treated much more rudely in London than any
part of France I've visited.

None of which makes any difference, of course. Unless you're
bigotted, and prefer prejudices to facts.

 

[Phlip]

Suppose I have a Point class that uses cartesian coordinates for its
internal representation. To satisfy a new testcase (for a new
requirement), I find that the internal representation should have been
in polar coordinates. Changing the internal representation in this way
requires me to modify multiple existing member functions. That is what I
mean with multiple modifications, and I see no reason to rerun my
testcases until all existing member functions have been changed to use
the new internal representation. I don't even expect the class to
compile before that.

Today is "prior art day"!

http://c2.com/cgi/wiki?DeprecationRefactor

"...by implementing two systems in parallel, getting the new one to work,
then nuking the old one"

"Use TestDrivenDevelopment to write a better system, leaving the old one in
place in parallel"

And stop harping on "lines" - nobody cares about them except you.

 

[Jerry Coffin]

(e-mail address removed)>, (e-mail address removed)
says...

[ ... ]

Except that they don't. They define policies, but they
certainly don't allow you to control exactly when the context
swaps occur.

It's certainly possible to do so in Windows, and I'm pretty sure it is
in (at least most) UNIX as well. It's decidedly non-trivial though. To
do it in Windows, you turn your test harness into a real debugger --
i.e. you set breakpoints in the client code and use those to give you
control over the relative order of code execution.

On the face of it, that sounds fairly complex, but in reality it's
really much worse. The problem is that the test harness needs knowledge
of the client code to ensure that it doesn't force a context switch at a
time that it couldn't happen under normal circumstances. For example, it
has to track all usage of critical sections, to be able to force context
switches between threads that don't depend on the same critical section,
but ensure against context switches among threads that do depend on the
same critical section.

In short, before you have a working test harness, you've duplicated a
large part of the functionality of a complete multitasking operating
system. Worse, there's almost no real scaling factor -- which is to say
that the test harness is tremendously huge and complex even if the code
under test is tiny.

 

[Jerry Coffin]

I invented "waterfall" as a strawman argument?? I wish!!

Philip please read what he wrote. The "invention" of which you're
accused isn't the waterfall model, but the studies that showed that
collecting requirements up front led to failure.

http://www.craiglarman.com/wiki/dow...ch2-applying-agile-iterative-evolutionary.pdf

"...waterfall values promoted big up-front speculative requirements and design
steps before programming. Consistently, success/failure studies show that the
waterfall is strongly associated with the highest failure rates for software
projects"

You didn't invent it (in fact, it was probably invented before you were
born), but the waterfall model has been know to have serious problems
for decades. On the other hand, I'm not sure anybody every _really_
expected it to work either. The earliest reference (of which I'm aware)
to the waterfall model (though it didn't use that terminology) is in a
1970 paper by Winston Royce:

http://www.cs.umd.edu/class/spring2003/cmsc838p/Process/waterfall.pdf

The entire paper is about the fact that this model doesn't work, and
what modifications have to be made to get something that does work.

The evidence suggests that nobody has ever really esposed or attempted
to follow the waterfall model. Rather the contrary, it appears to have
been set up as a straw man from day one, and every mention of it (ever)
has been to point out that it is deficient, impractical, and will lead
almost inevitably to disaster.

Though the economics were somewhat different in 1970, the paper cited
above also advocates many (perhaps even most) of the procedures that now
go under the names like XP, Agile Development, and so on. Way back in
1970, he essentially advocated pair programming: "Every bit of an
analysis and every bit of code should be subjected to a simple visual
scan by a second party who did not do the original analysis or code but
who could spot things like dropped minus signs, missing factors of two,
jumps to the wrong address, etc., which are in the nature of
proofreading the analysis and code." He then demonstrates the change in
economics in the next sentence: "Do not use the computer to detect this
kind of thing -- it is too expensive."

Another step he considered important was: "Involve the customer". He
said: "For some reason what a software design is going to do is subject
to wide interpretation even after previous agreement. It is important to
involve the customer in a formal way so that he has committed himself at
earlier points before final delivery."

Unless somebody can show evidence to the contrary, I'm prepared to
believe that James was entirely correct -- the waterfall model has been
a straw man from the very beginning, and nobody has ever believed or
claimed that it does or even could really work.

 

[Phlip]

You didn't invent it (in fact, it was probably invented before you were
born)...
1970 paper by Winston Royce:

You're too kind.

 

[Ian Collins]

Except that they don't. They define policies, but they
certainly don't allow you to control exactly when the context
swaps occur.

If you put a high priority real-time thread on a single core Solaris
system in a while(1) loop, you will brick the box. The last context
switch the system will make is the one that starts your thread.

I know, I did it once!

 

[Jerry Coffin]

If you put a high priority real-time thread on a single core Solaris
system in a while(1) loop, you will brick the box. The last context
switch the system will make is the one that starts your thread.

I know, I did it once!

You can do the same on Windows -- but neither one really has much (if
anything) to do with the original discussion.

Real time priority will _prevent_ context switches from taking place. To
test thread synchronization issues, you need to do exactly the opposite:
force thread switches to take place at every point in the program that
it could possibly happen.

 

[Phlip]

Real time priority will _prevent_ context switches from taking place. To
test thread synchronization issues, you need to do exactly the opposite:
force thread switches to take place at every point in the program that
it could possibly happen.

The only reason someone brought up concurrency is they hallucinated someone
saying "tests can catch any bug, including concurrency bugs". That never
happened, but this side thread did indeed draw attention away from TDD's power
as a design technique, including for threading...

 

[Jerry Coffin]

The only reason someone brought up concurrency is they hallucinated someone
saying "tests can catch any bug, including concurrency bugs". That never
happened, but this side thread did indeed draw attention away from TDD's power
as a design technique, including for threading...

I, for one, think there's a more substantive discussion than that. The
question, at least as I see it, is whether, and if so how, one can
automate testing of multithreaded code to produce meaningful results.

Now, doing that does take a large, complex piece of infrastructure -- so
large and complex that it's decidedly impractical to write it as a set
of tests for an individual program. OTOH, most of it should be reusable
and relatively easy to apply to an arbitrary piece of multithreaded
code. As such, it's almost crying out for some serious discussion of
what it should do and how it should work. Once at least some degree of
agreement can be reached, perhaps somebody can actually start work on
it, and make an attempt at doing something about applying testing to an
area where it's currently impractical.

 

[Ian Collins]

You can do the same on Windows -- but neither one really has much (if
anything) to do with the original discussion.

Real time priority will _prevent_ context switches from taking place. To
test thread synchronization issues, you need to do exactly the opposite:
force thread switches to take place at every point in the program that
it could possibly happen.

I've spent a lot of time over the years looking for an ideal solution to
this problem. The best I've come up with is the combination of code
inspection, static analysis tools like lock-lint and soak testing in
simulation.

The soak tests (for my embedded applications) involve running the
application using real-time threads on one core while generating
external events (waking high priority interrupt threads) from another.

 

[Phlip]

I, for one, think there's a more substantive discussion than that. The
question, at least as I see it, is whether, and if so how, one can
automate testing of multithreaded code to produce meaningful results.

Absolutely. Note that CPUs are themselves unit-tested, with little out-of-band
contact pads. That's so chip vendors can push up throughput by getting at least
a few of the dies on each wafer to perform correctly. When the surviving dies
get packaged, the extra contact pads are covered up forever.

And this is orthogonal to TDD. Designing the software is not the same as
soak-testing it. However, I did hear once that Ward Cunningham (one of the

>cough< inventors of TDD) once tested asynchronicity by writing a mock CPU in

Java, with its own scheduler and real-time hooks. I tried, but that's the kind
of thing Google can't find... (-;

 

[Phlip]

I've spent a lot of time over the years looking for an ideal solution to
this problem. The best I've come up with is the combination of code
inspection, static analysis tools like lock-lint and soak testing in
simulation.

I seem to like the idea of using TDD to build a synchronous, event-driven,
time-sliced architecture, and prevent concurrency for as long as possible. If
you prove you need it, the time-slicing gives you the structure you need to pull
it off.

Insert what I know about real-time concurrency issues here --> [ ].

Oh, except I used to hack AmigaDOS, back before Jerry Coffin was born...

 

[Jerry Coffin]

[ ... ]

Oh, except I used to hack AmigaDOS, back before Jerry Coffin was born...

My how I wish -- maybe if that was true, my knee (for one thing)
wouldn't have been bothering me so much today.

Unfortunately, for people like me who did our first real programming in
FORTRAN on mainframes that weighed as much as an average house, age has
caught up to the point that a few extra aches and pains are an everyday
sort of thing...