automated coding standard validation?

[Ian Collins]

Little comfort to the customer who manages to break it, but a good
reason for making the customer responsible for the specification.

Yes, we normally do this by making the customer (or a proxy if there
isn't a single customer) responsible for the acceptance tests, which
validate the specification against the customer's requirements.

 

[Keith Thompson]

That's where we differ, when you use TDD, your units test /are/ your
live runnable specification. So if they pass, the code meets its
specification.

I'm skeptical. I don't believe that tests of program behavior can
prove correctness.

For example, suppose the code works properly when plain char is
signed, but breaks when it's unsigned. No amount of testing can
detect such a bug *unless* (a) you test on a platform where plain char
is unsigned, or (b) you make the signedness of plain char one of the
acceptance criteria (which is possible only if you happen to think of
it).

There are plenty of potential system dependencies that are far more
subtle than the signedness of plain char. I don't see how you can
think of all of them, let alone test for all of them.

Remember that undefined behavior can always result in your program
behaving exactly as it should -- until it fails at the most
embarrassing possible moment.

 

[Ian Collins]

I'm skeptical. I don't believe that tests of program behavior can
prove correctness.

I agree that tests can only prove correctness on the platform where the
tests are run. That's where the Java/PHP/.NET crowd have a distinct
advantage when doing TDD.

For example, suppose the code works properly when plain char is
signed, but breaks when it's unsigned. No amount of testing can
detect such a bug *unless* (a) you test on a platform where plain char
is unsigned, or (b) you make the signedness of plain char one of the
acceptance criteria (which is possible only if you happen to think of
it).

In my shop, acceptance tests were always run on the target platform(s).

There are plenty of potential system dependencies that are far more
subtle than the signedness of plain char. I don't see how you can
think of all of them, let alone test for all of them.

No one can, that's why we use at least two levels of testing (developer,
customer acceptance and in my case, end user). Problems found in
acceptance testing require a unit test to be added in order to fix the
problem, so the unit tests become more comprehensive over time. the
signedness of char would be one of these conditions.

Remember that undefined behavior can always result in your program
behaving exactly as it should -- until it fails at the most
embarrassing possible moment.

I have too many bite marks to forget that one!

 

[Richard Bos]

I'm skeptical. I don't believe that tests of program behavior can
prove correctness.

Whereas I don't believe that correctness proofs can assert that the
program works correctly in practice.

Richard

 

[Al Balmer]

Whereas I don't believe that correctness proofs can assert that the
program works correctly in practice.

In the same sense - it meets the specification. Unfortunately,
specifications are often incorrect, and almost always incomplete.

 

[Ian Collins]

In the same sense - it meets the specification. Unfortunately,
specifications are often incorrect, and almost always incomplete.

At least when you are using your tests as a runnable specification, it
will remain current and adapt to meet changes to requirements.

 

[ena8t8si]

Why use a tool at all? Just adopt some agile practices such as
collective code ownership and pair programming. Way more effective than
a tool.

The usual hit-and-run tactics of agile/extreme programming
zealots. Great claims, no evidence.

 

[ena8t8si]

...
Sorry if techniques for developing reliable C applications are
considered off topic here.

Apology accepted.

I was simply trying to point out there are often (but not always)
process techniques that can replace tools.

When you find a more appropriate newsgroup for the
topic, please be sure to offer some substantation
rather than just argument by REPEATED ASSERTION.

 

[Ian Collins]

The usual hit-and-run tactics of agile/extreme programming
zealots. Great claims, no evidence.

I'm happy with the evidence provided by the continued success of my
teams and their products. But as you said else thread, techniques for
developing reliable C applications are considered off topic here.

 

[ena8t8si]

I'm happy with the evidence provided by the continued success of my
teams and their products.

If you care only about whether _you're_ happy with
your conclusions, then kindly keep your opinions
to yourself. Other people are more discriminating.

But as you said else thread, techniques for
developing reliable C applications are considered off topic here.

Yes, that's why I'd added OT: in the Subject line. HELLO!

 

[Ian Collins]

If you care only about whether _you're_ happy with
your conclusions, then kindly keep your opinions
to yourself. Other people are more discriminating.

I don't, I care that my clients and customers are happy.