All About Agile | Agile Development Made Easy!
BDD, TDD, Acceptance Testing and Executable Specification
Here's a question that was posted on our internal Agile Forum ->
In terms of TDD, BDD, Acceptance Testing and Executable Specification, I have heard:
1. Acceptance Testing is the same as Executable Specification
2. BDD is the same as Executable Specification
3. BDD replaces TDD
But if all three were true, than one could say Acceptance Testing replaces TDD, which certainly not what anyone is saying.
So I'm missing something....
I wondered if anyone had a good reply to this ?
Views: 252
Replies to This Discussion
-
Permalink Reply by Les Oliver on -
Sina
I put your question to a colleague of mine who has much experience in this area, this are his thoughts on the matter:
"If you were to express your requirements as a set of acceptance tests (and I advise you do) and you were to automate those tests, every time you ran them, you could be said to be executing the requirements. If they all passed, you could be reasonably certain that your application worked as per the requirements. Sounds simple and in concept it is, the difficulty is discovering all the tests.
BDD or Behaviour Driven Development is a way of expressing tests/requirements in a way that is easily understandable by both humans and computers. It usually takes the form of
Given the system is in a particular state
When I perform some action
Then I expect this to be true
For example to express one aspect of the behaviour of a calculator in BDD we might say
Given the calculator is turned on and the current total is zero
When I press the key sequence "2+2="
Then I expect the display to show the value 4
BDD doesn't replace TDD but is one way of practising the acceptance testing part of it. One of the issues we have with TDD is that most of literature skimps on the acceptance tasting part of it. There is even an Acceptance Test Driven Development (ATDD) group, which to me is all wrong because it separates two things that should be one and the same. Any development should always start with a 'pull' from the customer. We don't do anything without a customer/acceptance test. Unfortunately, most of the literature on TDD skimps on the acceptance testing part and focusses purely on the technical aspects of unit testing. Hence unit testing and TDD have become synonymous in many quarters with acceptance testing being thought of as "somebody else's problem." If we end up in a scenario where acceptance testing truly is somebody else's problem then we're back to the same probems that TDD was meant to help cure in the first place. Acceptance testing doesn't replace TDD but is an integral part of it."
Does that help?
-
Permalink Reply by Sina Jazayeri on -
Thanks - Here's a bit more detail as to where the confusion stems from :
"
My gut feel is that BDD doesn't replace TDD, but the BDD advocates seem to imply it does. Their argument is "TDD doesn't address the problem BDD addresses", which seems like they're saying "Don't do TDD, do BDD". BDD tools, RSpec, NSpec, certainly sound like they're talking about automating Specs. And BDD terminology talks about the Given/When/Then method of writing tests, which some advocate as the way to write Stories, which has to do with Acceptance Tests. Cucumber is a popular Executable Specification platform, but it's very similar to Green Pepper and FitNesse which are Acceptance Testing frameworks.
Furthermore, I myself can easily argue for, and provide real examples of when you still need TDD like unit tests, even when you have full Acceptance Tests. For instance, there may not be a business case for a memory manager, but my design may require one, and I would certainly want to have a good unit test set to verify the memory manager. Same thing can be set for any data structure that you have to implement. TDD / unit tests seem to verify structural aspects of the software, validating the design, whereas BDD / acceptance tests verify the specification, validating the requirements
"
-
Permalink Reply by Ilja Preuß on -
OK, here is what it looks like from my perspective (disclaimer: I'm not an expert on BDD).
In an Agile project (and non-Agile, too, probably), there should be at least two layers of testing.
Acceptance Tests are specified by the customer and describe how features of the system are expected to work, from the perspective of a user.
Unit Tests are specified by the developers and describe how a small part of the system is supposed to work, from the perspective of the developer. These often test implementation details that the customer isn't interested in, or won't even be able to understand.
Both layers are necessary: if all unit tests pass, that doesn't necessarily mean that the customer's requirement really is fully implemented. Also, the unit tests often don't communicate *why* that part of the system is supposed to work that way - that is, which implemented feature requires it to be implemented that way. On the other hand, because of combinatorical explosion and because Acceptance Tests are black box tests, they often can't take into account all the border cases that occur deep inside the system.
Test Driven Development is a process for writing *unit* tests, where the developer switches between writing a new test, making the test pass by writing production code, and refactoring the design of the code. This cycle typically has a length of just a few minutes.
Acceptance Test Driven Development is a process for writing *acceptance* tests, where the customer, together with the team, defines the tests for new features before they get implemented, as part of the iteration planning. It has in common with TDD that the tests are defined before implementation - therefore the similar name. It is very distinct from TDD in that it doesn't have the tight feedback loop of writing a test, making it pass, and writing the next test based on what you've learned so far.
BDD mostly seems to be a *style* of expressing requirements in tests. It involves using a specific structure (given-when-then) and expressing statements using a very deliberate language (for example, using "should" in place of "assert" or similar). Originally used to *complement* TDD, it can as well be used for Acceptance Testing.
I hope this helps.
-
Permalink Reply by Sina Jazayeri on
-
Thanks - I found this set of slides reference on the subject that very clearly lays out the concepts.
-
Permalink Reply by Phlip on -
The Test Food Pyramid:
Goal Donors don't need to hear about TDD directly. They view a project by looking down at that pyramid from the tip, where they see features, metrics, and literate BDD tests.
Developers look up from the bottom level. Most tests are TDD tests. And all tests share the same infrastructure, fixtures, etc.
-
Permalink Reply by Ilja Preuß on
-
Interesting pyramid. My only concern is that I don't think the customer should care a lot about metrics (other than perhaps bug count, age of oldest bug or similar). How would the typical customer even make sense of code coverage?
© 2012 Created by Kelly Waters.
