It’s not very uncommon for software projects to rely only on integration testing made relatively late in software development cycle. I propose move bug-catcher earlier and closer to developer in order to minimise testing overhead and give better stability of code.
The promise of automated integration testing
A typical project manager may ask himself a question: “Isn’t the best way to check the implementation running it all at once?”. During such big run some script recorded by a tester could examine functionality of all components and check if those components are properly integrated. Sounds very promising: verify components and interfaces by using one method.
There are many tools on market targeted at recording and replaying sequences of operations done thru user interface. Salesman will promise you that soma magical testing tool will be easy to and guarantee you high reliability of your software. It wont.
Why automated integration testing sucks?
Many times in the past I saw typical picture: scripts that should control (and validate) elements of an user interface were left unmaintained and nobody wanted to take care of them. One good question arise then: what’s wrong with integration-level testing automation?
The main problem with integration testing is complicated state setup. We have to deal with whole system, so in order to make test repeatable we have to setup state of all components. That’s why it’s very hard to do reliably (especially when system spans multiple servers/networks).
Next big problem with automated integration testing if fragility. In incremental development methods (Agile methods are gaining more focus today) system is supposed to be build incrementally. Incremental build assume that architecture will evolve during development. If you connect automated integration testing to development process early (to early catch as many bugs as possible, of course) small change in source code could easily break many testing scripts. Integration test require non-trivial setup, so they won’t be performed by ordinary developer. Result: maintaining failing test cases after few weeks are nightmare.
Unit testing problems
First of all: unit testing must be done by a programmer. “So” – project manager thinks – “they are more costly than integration testing done by tester” (there are differences in per-hour prices). Second: system must be designed to be testable. If you have high coupling in project it’s very hard to properly setup simplest test case, so nobody will do that.
Unit testing benefits
In my opinion unit testing influence on system architecture is the best side effect (and Test Drive Design from Extreme Programming). We are getting loosely-coupled system that is easy to maintain and extend.
