BDD and TDD - What are they for?

[Eleanor McHugh]

The whole "requirements in front" idea is perfectly simple and
logical; also, typically impossible or wrong.
"If I'd asked people what they wanted, they would have asked for a
faster horse." -- attributed to Henry Ford.

Very true.

As my experience is primarily green-field development I've spent my
career arguing with clients who not only didn't understand their
problem domain but were entirely blinded to that fact by their
preconceptions. This is not to say that the requirements they bring to
the table are completely worthless, but they have to be viewed in
terms of need much more than of wants to develop a cost-effective and
lasting solution.

Taking the horse analogy, a man who wants to buy six horses may be
doing so because he needs to plough three fields in a day and be used
to thinking in terms of three teams all running in parallel, when in
actual fact a tractor may be able to do those same three fields in a
single day by serialising the task and thus do away with the need for
two additional teams. That's essentially the point of a faster horse.

To convince the man of this you need a working tractor to demonstrate
and the only test that matters is the ploughing, which is a good
argument for prototyping. All other tests that you may frame for the
tractor - or for the horses for that matter - fall under the general
category of robustness/quality and will generally be ignored by the
man at this stage unless the tractor breaks down during the
demonstration.

From what I've seen of BDD and TDD in practice, they focus a lot of
effort into making the teams of horses as close a match to those the
man wants as possible so as to achieve incredibly high satisfaction
ratings: they're thus an excellent marketing technique. However what
the man fundamentally needs is a more effective ploughing solution
that costs him less money, and figuring that out requires a broader
perspective than that provided by just-in-time requirements analysis.

He probably has to put more effort into getting a tractor - as
reflected by a higher capital investment and higher fuel costs - but
the productivity of this solution more than pays for the extra
investment. Would he be satisfied watching the whole tractor
construction project from initial back-of-the-envelope design to real-
world deployment? I very much doubt it. What he's interested in is
ploughing, not engineering. But what he ends up with is still the
better solution to his actual problem than the wonderfully supportive
development of a six horse ploughing solution even though that's what
he wanted all along.

I can see that BDD + inspiration could develop the tractor as a
solution to the problem, but could BDD on its own? How flexible is it
in practice and can a project that's 100k+ SLOC of actual application
(plus whatever overhead in test code) take revolutionary departures?
Does it suit the traditional hacker habit of forking early and forking
often? And what are the implications for all that test code?


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

 

[Robert Dober]

You should run a TDD test suite after the fewest possible edits: 10 at the
most, and hopefully just 1. A coding session should go type-type Test,
type-type Test, type-type Test, in tiny cycles.

Some believe that no editing at all shall be done other than making a
failing test succeed. I quite adhere to this philosophy, I am however
taking the liberty to omit the failing test run before editing as my
experience has shown me that enough failing test runs will follow
anyway
However I do not code unless I want a new test case to succeed, that
is only when I am doing TDD of course.
Cheers
Robert

 

[Phlip]

The whole "requirements in front" idea is perfectly simple and
logical; also, typically impossible or wrong.
"If I'd asked people what they wanted, they would have asked for a
faster horse." -- attributed to Henry Ford.

Tx I am so passing that one to the XP mailing list!

 

[Avdi Grimm]

From what I've seen of BDD and TDD in practice, they focus a lot of effort
into making the teams of horses as close a match to those the man wants as
possible so as to achieve incredibly high satisfaction ratings: they're thus
an excellent marketing technique. However what the man fundamentally needs
is a more effective ploughing solution that costs him less money, and
figuring that out requires a broader perspective than that provided by
just-in-time requirements analysis.

I think you may be conflating two different scenarios. There's the
scenario where a customer knows approximately what they want, and an
agile team can help them triangulate in on exactly what they need with
a minimum of wasted effort. Then there's the scenario where someone
who has worked in a particular domain for awhile realizes that things
could be done in a much better way, and starts a project to implement
it. Often, this is someone who has worked on several teams such as
previously described. In this case, the visionary, not the projected
end-user, is the "customer", at least in initial development. If all
goes well, and they hire a good marketing team, the visionary and her
team then convince their end-users that what they REALLY want is this
revolutionary product. This goes a lot easier if the team in question
is already well known and respected for their *evolutionary* work in
the same field.

I would submit that when a customer hires you to build them a better
plow, you should build them a better plow. However, once you have
done that a few times you are in an excellent position to go off for
six months and innovate a new hydroponic gardening system and sell it
to your existing customers.

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[Robert Dober]

Tx I am so passing that one to the XP mailing list!

Why be offended, our planet might be better off with faster horses, I
actually would like horses running at Warp 10! (No that is not 42
times 10 square times 12 square, 10 is fast enough).
Holding on to them might prove difficult though.
Well did we get off topic here?
R.

 

[Clinton D. Judy]

Thanks for the great replies. I'm attempting to learn BDD through rSpec
now.

Actually, are there any suggestions for learning rSpec for someone who's
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven't touched it yet.

-----Original Message-----
From: Robert Dober [mailto:[email protected]]=20
Sent: Monday, August 18, 2008 12:45 PM
To: ruby-talk ML
Subject: Re: BDD and TDD - What are they for?
Importance: High

Tx I am so passing that one to the XP mailing list!

Why be offended, our planet might be better off with faster horses, I
actually would like horses running at Warp 10! (No that is not 42
times 10 square times 12 square, 10 is fast enough).
Holding on to them might prove difficult though.
Well did we get off topic here?
R.

 

[Avdi Grimm]

Actually, are there any suggestions for learning rSpec for someone who's
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven't touched it yet.

Peepcode's pretty awesome, I'd say go with that

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[Phlip]

Thanks for the great replies. I'm attempting to learn BDD through rSpec
now.

Actually, are there any suggestions for learning rSpec for someone who's
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven't touched it yet.

Yes - learn TDD, because that's what most of the tutorials will cover. And
Test::Unit comes bundled with Ruby. At the entry level, if you try to learn BDD
first, it's just TDD with inverted syntax.

Then, also look at non-Ruby TDD tutorials, such as /Test Driven Development/ by
either Kent Beck or Dave Astels...

 

[paron]

I think you may be conflating two different scenarios.  There's the
scenario where a customer knows approximately what they want, and an
agile team can help them triangulate in on exactly what they need with
a minimum of wasted effort.  Then there's the scenario where someone
who has worked in a particular domain for awhile realizes that things
could be done in a much better way, and starts a project to implement
it.  Often, this is someone who has worked on several teams such as
previously described.  In this case, the visionary, not the projected
end-user, is the "customer", at least in initial development.  If all
goes well, and they hire a good marketing team, the visionary and her
team then convince their end-users that what they REALLY want is this
revolutionary product.  This goes a lot easier if the team in question
is already well known and respected for their *evolutionary* work in
the same field.

I would submit that when a customer hires you to build them a better
plow, you should build them a better plow.  However, once you have
done that a few times you are in an excellent position to go off for
six months and innovate a new hydroponic gardening system and sell it
to your existing customers.

--
Avdi

Home:http://avdi.org
Developer Blog:http://avdi.org/devblog/
Twitter:http://twitter.com/avdi
Journal:http://avdi.livejournal.com

Yes, there's a fundamental difference between making incremental
changes and reimagining the problem domain. My background, like
Ellie's, is in greenfield situations, which colors my thinking. As she
and Avdi point out, BDD is fine for evolutionary changes, but not for
revolutions in thinking. BDD is a caterpillar tractor -- controlled
and unstoppable. Real innovation is more like dynamite.

Even a non-working prototype will help your customer make the leap to
see his problem in a different light. It also helps if they are
desperate. If they aren't desperate, they will stick with various
combinations of horse until they can no longer do so. Then they might
be willing to make the leap -- but you still have to show them a
prototype.

Once you have buy-in to a fundamental change or a novel idea, THEN BDD
comes into its own.

 

[Avdi Grimm]

Yes, there's a fundamental difference between making incremental
changes and reimagining the problem domain. My background, like
Ellie's, is in greenfield situations, which colors my thinking. As she
and Avdi point out, BDD is fine for evolutionary changes, but not for
revolutions in thinking. BDD is a caterpillar tractor -- controlled
and unstoppable. Real innovation is more like dynamite.

I think you missed my point. BDD is equally applicable to both
status-quo development and to radical departures and prototyping.
What changes is not the development practices, but the *customer* who
helps determine what the proper specs should be. When you are
building a typical evolutionary system, the customer is the client;
the end user. When you are developing an innovative new product, the
customer is *you* - or whoever has the vision for the new product.
BDD/TDD is still equally important, but the person defining the specs
differs.

There is a line of thinking that says that TDD/BDD is inapplicable to
prototypes, because the prototype will just get thrown out anyway. In
my experience, though, this rarely happens - in the real world you
don't have *time* to throw out the prototype, and it winds up getting
pressed into service as the basis for the final product. Which then
necessitates a lot of painful retrofitting of test coverage.

Agile methodologies do generally accept creating "spikes" without
testing up front. A spike is basically an exploratory
proof-of-concept for one particular high-risk part of the application
- e.g. "let's do a spike to prove that we can process 1000 SOAP
messages per second". Spikes are not prototypes, however. Their
scope is much smaller, usually taking only a few days to a week or two
to create. As a result they are a lot easier to throw away than a
working prototype.

Once you have buy-in to a fundamental change or a novel idea, THEN BDD
comes into its own.

In my experience BDD/TDD comes into it's own as soon as your code
exceeds somewhere around 100 lines of code. Or less.

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[David Masover]

"If I'd asked people what they wanted, they would have asked for a
faster horse." -- attributed to Henry Ford.

That's actually a really good analogy -- but you see, Ford's leap wasn't
abandoning what his clients wanted altogether. People obviously do want cars.

But why do they want cars? What's the benefit of cars?

It would not be very hard to write, up front, a requirement that it be able to
transport people and some limited amount of cargo, and that it should go
faster than a horse.

More importantly, I don't think BDD (or TDD) is about specifying this
requirement up-front, nor do they prevent emergent phenomena.

It's more about specifying it before you code it. Which could be down to
several minutes before you code it.

Unless you really are blindly banging around on your keyboard, to see what
happens. Which I'll admit to doing -- but I tend to do that in irb, not in
source code.

 

[David Masover]

Yes, there's a fundamental difference between making incremental
changes and reimagining the problem domain.

Yes, there is. But it's not a difference in testing methodology.

I would use Ruby for any program I could -- I find it equally useful for
experimental greenfield apps and aging production apps.

Similarly, I would use BDD as soon as I could, in an experimental greenfield
app. And I do.

That's my opinion, of course. I'll never understand why, but some people
prefer to use Ruby or Python to prototype an app, then throw away the
prototype and start from scratch in Java or C++.

Even a non-working prototype will help your customer make the leap to
see his problem in a different light.

That is true. However, properly-implemented BDD could (sometimes, at least)
lead to that prototype faster.

All else equal, a working prototype is better than a non-working prototype.

I'd also argue that a non-working prototype carries several other, hidden
risks -- mainly, the risk of promising too much.

No matter how many times you tell them "This is just a prototype; it'll have
to be rewritten from scratch; we can't actually do this for another six
months..." They're still going to come out of that presentation
thinking "Man, that's an awesome prototype! I can't wait!"

No, you show the awesome-but-nonworking prototype to your boss. Not even
that -- if you can, you show him a whiteboard sketch of it.

But even with bosses, and especially with clients, remember the Scotty
philosophy: Underpromise, so you can overdeliver. That's how you earn a
reputation as a miracle-worker.

It also helps if they are
desperate. If they aren't desperate, they will stick with various
combinations of horse until they can no longer do so. Then they might
be willing to make the leap -- but you still have to show them a
prototype.

If you've got all the time it takes them to figure out that various
combinations of horse won't work, you have the time to do it right.

The first car that Ford showed the world didn't explode. The Wright Brothers
built a prototype plane, and it flew. They weren't complete, but for some set
of requirements (aka specs/stories), they were working.

 

[Phlip]

That's actually a really good analogy -- but you see, Ford's leap wasn't
abandoning what his clients wanted altogether. People obviously do want cars.

The car industry also did it incrementally. The original "horseless carriages"
were exactly that - a carriage frame with an engine strapped into the luggage
rack. Model T cars look square and funny because that carriage was still there,
underneath.

After people stopped thinking they needed faster horses, and after they
participated in reviewing the designs of cars, then the horseless carriage
morphed, feature by feature, into modern cars.

More importantly, I don't think BDD (or TDD) is about specifying this
requirement up-front, nor do they prevent emergent phenomena.

TDD allows you to safely change your design and features while keeping out the
bugs. That makes just-in-time requirements safer.

 

[d c]

the counter point:

TDD seems badly suited to *really* rapid prototyping.

I often use TDD for production code that I need to work long after i
have moved on,
or for production apps that many people need to work on. but its kind
of like red-tape.
if you dont trust yourself or others, then get a contract. if you do,
then word of mouth is fine.

also its an issue of scale - if you are working on sth for > 1 week,
you need tests to keep it all together.
but many proofs are done over a weekend or a hectic all-nighter.

if you are looking for a proof of concept - if something works or not
(today, and you don't care about tomorrow) then writing iron-clad
tests for everything seems redundant. if you have a level of
confidence, you can fly blind, and i feel get into a much better state
of flow without tests. a week later things may not work as planned
tho.

additionally tests prove things on a very abstract level. if the
wright brothers had spent a bunch of time with RSpec they would have
theoretically proved their model of a cylinder would have enough
horsepower to lift it itself. and the tests would ensure that the next
change to the air system, would be guaranteed to not break the
cylinders. good, solid, mainstream engineering.

but they would never have captured people's imaginations as much as a
working flying machine.
if it blew up in a future flight, at least it flew first...

/dc

 

[Avdi Grimm]

additionally tests prove things on a very abstract level. if the
wright brothers had spent a bunch of time with RSpec they would have
theoretically proved their model of a cylinder would have enough
horsepower to lift it itself. and the tests would ensure that the next
change to the air system, would be guaranteed to not break the
cylinders. good, solid, mainstream engineering.

You mean "Unit Tests", right? The term "tests" encompasses manual
tests, unit tests, functional tests, integration tests, usability
tests, user acceptance tests, stress tests, fuzz tests, smoke tests,
final acceptance testing, etc.

I don't know where people get the idea that because you are doing Unit
Tests or BDD, you must not be doing any other kind of testing - for
instance, firing the app up and using it. TDD/BDD is a great way to
drive your coding, but it's not the only kind of testing in a healthy
project.

See also http://railspikes.com/2008/7/11/testing-is-overrated

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[Gregory Brown]

if you are looking for a proof of concept - if something works or not
(today, and you don't care about tomorrow) then writing iron-clad
tests for everything seems redundant. if you have a level of
confidence, you can fly blind, and i feel get into a much better state
of flow without tests. a week later things may not work as planned
tho.

I understand the sentiment, but if you feel the need to write
iron-clad tests up front, you're probably not going to see most of the
benefits of TDD.
For me, the flow isn't broken if I limit each test to a single, basic
change that I'm about to make. I don't even begin to think about edge
cases until I write tests that fail for the bare minimum basics (a
single use of a single function). Then I write that code. Then I
move on to the next thing.

Often times, figuring out what tests I need to write next involves
toying around with a prototype, or writing parts of my function in
IRB. This helps me figure out how my code should work without having
to think it through mentally. Once a get a sense of how that stuff
should work, I might refactor a bit under the green light to make room
for the new behaviour. Then I write the new failing test, then I
write the code.

Most of the time, this doesn't cause any noticeable decrease in my
velocity, it even increases it sometimes. Purists might say you
shouldn't write any code at all before your tests, but I think what's
important is that you shouldn't *integrate* any code before you write
the tests. Hacking around with an experiment, so long as it's tiny
and can easily be followed by a formal test case, is always helpful.

-greg

 

[d c]

additionally tests prove things on a very abstract level. if the

You mean "Unit Tests", right? The term "tests" encompasses manual
tests, unit tests, functional tests, integration tests, usability

well, i guess you could say looking at things in a browser is part of
"testing" but thats not the point of this discussion?

i came to ruby cos i like to realize ideas quickly.
premature test cases is like premature optimization.
making a change means fixing a bunch of failing tests...
in the first 24 hours of a project, where you *know* things are likely
to change a lot, then testing + coding is close to 2X the work.

if you're working on a billing system for a bank,
or a demo for a flying pumpkin that blows up at midnight,
the needs are completely different.

but i think the prevailing opine that tests somehow give you more
freedom is relative to your event horizon.

/dc

 

[Avdi Grimm]

i came to ruby cos i like to realize ideas quickly.
premature test cases is like premature optimization.
making a change means fixing a bunch of failing tests...
in the first 24 hours of a project, where you *know* things are likely
to change a lot, then testing + coding is close to 2X the work.

I don't have a problem with doing 24 hours worth of exploratory coding
sans tests. But if I decide at the end of that I decide I want to
continue, my best bet is to THROW IT AWAY and rewrite it test-first.

If you and/or your organization have the discipline to a) stop
prototyping early; and b) throw the prototype away, then test-less
prototyping is workable.

--
Avdi

Home: http://avdi.org
Developer Blog: http://avdi.org/devblog/
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

 

[David Masover]

the counter point:

TDD seems badly suited to *really* rapid prototyping.

I thought so, too. And I got quite a lot done, very fast, with this hobby
project I'm working on...

Then it reached a threshold -- not a very high one -- where I could no longer
reasonably develop the app, because of how long it was taking me to test it,
manually. At this point, I at least needed a testing framework.

And that was short of having the working prototype that I want. It's obvious
at this point that from here on (till the working prototype), not having
tests will slow me down, not speed me up.

Now, I suppose it depends on the complexity of the prototype. If you want to
prototype absurdly fast (a few hours or days), and it's a reasonably simple
concept (a couple hundred lines or so), then yes, it's probably faster to
just code it.

But the longer you put off writing those tests, the worse it's going to be
when you finally start.

 

[Phlip]

Confer "Architectural Spike". That means a prototype where your colleagues are
the demonstration's audience. You should write enough of a new module to "close
the loop". The new code will spike thru a view, middleware, back-end, a little
productization, and a little testage. Y'all will then rewrite the spike via TDD.

However, if the prototype's audience is client-side, they might point here and
there, and say, "okay, keep this here, but change that around. In one hour."

If they say that, you are already around the bend. The tests must already be in
place to sustain the rate-of-change your client now expects of you. This is why,
in some circles, the ideal is "there are no Prototypes!"

I thought so, too. And I got quite a lot done, very fast, with this hobby
project I'm working on...

Then it reached a threshold -- not a very high one -- where I could no longer
reasonably develop the app, because of how long it was taking me to test it,
manually. At this point, I at least needed a testing framework.

Right. The bigger the project, the longer it runs, the more you need tests. They
help you add new features faster than you added old ones. For example, at work,
last year we added a bad database field that got stale very soon. Back then, the
code was too fragile to just yank it out. But since then, we have relentlessly
tested, and cleaned all around that variable, and today we took it out with no
issues.

Under Brand X, the sooner you take that variable out, the less you suffer. The
code will ossify around it.