While I've written unit tests for most of the code I've done, I only recently got my hands on a copy of TDD by example by Kent Beck. I have always regretted certain design decisions I made since they prevented the application from being 'testable'. I read through the book and while some of it looks alien, I felt that I could manage it and decided to try it out on my current project which is basically a client/server system where the two pieces communicate via. USB. One on the gadget and the other on the host. The application is in Python.
I started off and very soon got entangled in a mess of rewrites and tiny tests which I later figured didn't really test anything. I threw away most of them and and now have a working application for which the tests have all coagulated into just 2.
Based on my experiences, I have a few questions which I'd like to ask. I gained some information from New to TDD: Are there sample applications with tests to show how to do TDD? but have some specific questions which I'd like answers to/discussion on.
- Kent Beck uses a list which he adds to and strikes out from to guide the development process. How do you make such a list? I initially had a few items like "server should start up", "server should abort if channel is not available" etc. but they got mixed and finally now, it's just something like "client should be able to connect to server" (which subsumed server startup etc.).
- How do you handle rewrites? I initially selected a half duplex system based on named pipes so that I could develop the application logic on my own machine and then later add the USB communication part. It them moved to become a socket based thing and then moved from using raw sockets to using the Python SocketServer module. Each time things changed, I found that I had to rewrite considerable parts of the tests which was annoying. I'd figured that the tests would be a somewhat invariable guide during my development. They just felt like more code to handle.
- I needed a client and a server to communicate through the channel to test either side. I could mock one of the sides to test the other but then the whole channel wouldn't be tested and I worry that I'd miss that. This detracted from the whole red/green/refactor rhythm. Is this just lack of experience or am I doing something wrong?
- The "Fake it till you make it" left me with a lot of messy code that I later spent a lot of time to refactor and clean up. Is this the way things work?
- At the end of the session, I now have my client and server running with around 3 or 4 unit tests. It took me around a week to do it. I think I could have done it in a day if I were using the unit tests after code way. I fail to see the gain.
I'm looking for comments and advice from people who have implemented large non trivial projects completely (or almost completely) using this methodology. It makes sense to me to follow the way after I have something already running and want to add a new feature but doing it from scratch seems to tiresome and not worth the effort.
P.S. : Please let me know if this should be community wiki and I'll mark it like that.
Update 0 : All the answers were equally helpful. I picked the one I did because it resonated with my experiences the most.
Update 1: Practice Practice Practice!
As a preliminary comment, TDD takes practice. When I look back at the tests I wrote when I began TDD, I see lots of issues, just like when I look at code I wrote a few year ago. Keep doing it, and just like you begin to recognize good code from bad, the same things will happen with your tests - with patience.
"The list" can be rather informal (that's the case in Beck's book) but when you move into making the items into tests, try to write the statements in a "[When something happens to this] then [this condition should be true on that]" format. This will force you to think more about what it is you are verifying, how you would verify it and translates directly into tests - or if it doesn't it should give you a clue about which piece of functionality is missing. Think use case / scenario. For instance "server should start up" is unclear, because nobody is initiating an action.
First, yes, tests are more code, and requires maintenance - and writing maintainable tests takes practice. I agree with S. Lott, if you need to change your tests a lot, you are probably testing "too deep". Ideally you want to test at the level of the public interface, which is not likely to change, and not at the level of the implementation detail, which could evolve. But part of the exercise is about coming up with a design, so you should expect to get some of it wrong and have to move/refactor your tests as well.
Not totally sure about that one. From the sound of it, using a mock was the right idea: take one side, mock the other one, and check that each side works, assuming the other one is implemented properly. Testing the whole system together is integration testing, which you also want to do, but is typically not part of the TDD process.
You should spend a lot of time refactoring while doing TDD. On the other hand, when you fake it, it's temporary, and your immediate next step should be to un-fake it. Typically you shouldn't have multiple tests passing because you faked it - you should be focusing on one piece at a time, and work on refactoring it ASAP.
Again, it takes practice, and you should get faster over time. Also, sometimes TDD is more fruitful than others, I find that in some situations, when I know exactly the code I want to write, it's just faster to write a good part of the code, and then write tests.
Besides Beck, one book I enjoyed is The Art of Unit Testing, by Roy Osherove. It's not a TDD book, and it is .Net-oriented, but you might want to give it a look anyways: a good part is about how to write maintainable tests, tests quality and related questions. I found that the book resonated with my experience after having written tests and sometimes struggled to do it right...
So my advice is, don't throw the towel too fast, and give it some time. You might also want to give it a shot on something easier - testing server communication related things doesn't sound like the easiest project to start with!