There seems to be some consensus in the TDD and BDD community that the tests (or behaviors) for a given software system constitute an (executable) specification of the system.
Wrong.
A specification is an unambiguous and complete description of a system or subsystem. Or at least it should be a heroic attempt.
A test, whether written as a traditional unit test, or as an executable scenario in BDD is neither unambiguous nor complete. Worse, it’s not even trying.
A test is almost always written as one specific example of how a subsystem should behave. An implementation that can fail on other examples will pass the test, and so still fulfill the non-specification. Furthermore, there are many implementations with wildly differing behavior that can pass the test.Doesn't sound like a specification to me, or at least not a good one.
In fact, if TDD/BDD advocates would really follow their advice to program no more than what the tests test, then given the following test:
Assert.Equal (Reverse [1,2,3] , [3,2,1])
they should implement the pseudo function:
Reverse list = [3,2,1]
Compare with what FsCheck's properties look like (again, in pseudo-code):
Reverse [x] = [x]
Reverse (x::xs) = Reverse xs @ [x]
That's what I call a specification: unambiguous, and complete. So TDD talks the talk, but does not walk the walk.
Before you start commenting, foam at the mouth, I’m not saying testing is bad – on the contrary. It’s just that you should stop kidding yourself by calling tests specifications. It’s like saying that “8” is an algorithm for adding two numbers, because it happens to work for 3 and 5.
Using frameworks and tools such as Code Contracts, Pex and FsCheck (I’m being completely unbiased here, of course), you will take a big step towards writing real specifications, not tests. That’s where their real value is.
| Share this post : |  Technet! |  del.icio.us it! |  del.iri.ous! |  digg it! |  dotnetkicks it! |  reddit! |  technorati! |
I say, let's humor them.
ReplyDeleteUnambiguous, mecanically verifiable couples of inputs and outputs are baby steps in the right direction, and an improvement over purely informal descriptions. Once (if) everyone starts doing it, the problems with the approach will
become clear soon enough, and the solution too :)
By the way, what you call "specifications" some call "complete specifications", leaving the unqualified "specifications" for the incomplete specifications that are often the best one can do with human capabilities, even heroically trying.