At the Melbourne eXtreme Programming Enthusiasts Group the other night we had a big rantorama about TDD. One of the points in my presentation was that Kent had said that unit tests had to be fast, and he cited one project in which there were 4,000 tests that ran in about 20 minutes. A few guys from my team were there and guffawed out loud. Our current project (4 months old) has 1,300 unit tests that run in about 10 seconds, so they weren't impressed with Kent's numbers at all. Even allowing for Moore's law over a few years, we'd run about 4,000 tests in 30 seconds, which two speed-doublings ago would take 2 minutes. That's still an order of magnitude faster...so what's the deal?
This led to lots of enthusiastic debate about what the hell a unit test actually is. For what it's worth, here's my simplistic test categorization model:
1. A unit test is a test that requires only the class under test and its immediate dependencies (or mocks thereof). This means no deployment, no packaging, nothing. That's why they're fast.
2. A functional test is a test you can't run until the app is packaged and deployed in some way. So you "in-container unit test" fans aren't doing unit tests by this model. I'm hard on this stuff because anything requiring deployment puts you in a different order of magnitude speed wise; this doesn't mean these tests aren't valuable, just that I don't count them as _unit_ tests, even if they only test a single unit of functionality.
3. An integration test is a test of your external dependencies, not necessarily requiring your app to be deployed at all, but requiring your external dependencies to be available. This is where databases come in, or other services provided by EJB's or anything else. These tests are also usually way too slow for me to want too many of them, and I want to isolate them from my app as much as possible. This means a separate test harness for the external stuff and leave my app just unit testing against the assumptions proven by the integration test suite.
The point is that I want to push unit tests as far as I can because they just provide so much more performance than the other types of tests. That's the only reason. I want as few of the others as possible because I spend enough of my life waiting for computers to get done with stuff as it is. Yes, you _can_ unit test custom tags to within an inch of 100% coverage!
So what's everyone else getting speed-wise???
