In most human endeavors that take a stretch of time to complete, the idea is latched onto and people become excited about being involved. It doesn’t matter if it is building a skyscraper, inventorying a massive warehouse or even going to war. People are intrigued by the possibilities and want to get started doing the thing. After a time, though, this enthusiasm wanes, and most often is replaced by a weary determination to get the thing done. The nature of the endeavor doesn’t have to change; the reality of its implementation just becomes the dominant factor.
IT projects in general—and continuous integration (CI) projects in particular—are no different. Whether a new project or implementing comprehensive CI on an existing project, the beginning is the most exciting part. Soon the excitement wanes and achieving the desired level of test coverage and a consistent build success rate becomes the goal that is trudged toward. It is a rare team that reaches for 100 percent code coverage with testing on initial implementation. The key becomes “testing what’s important.” And that varies by team and project.
But testing at all levels—unit, integration, smoke, system, regression—could be better on almost every project. That improvement can be done incrementally while running through sprints. While some of the testing is managed by QA, some is managed by the developers—and in a CI/CD environment, arguably more is managed by developers. They are more involved in the testing process in CI/CD, so are more likely to be more involved in test development/management.
The point is, good enough coverage is just that. And there is a hidden cost in getting too carried away with testing every line of code for every possible input, but improving tests is more than simply cranking out more permutations. Improving existing tests, or writing tests for functionality currently poorly tested, is increasing testing.
So, as you strive to make your code get better, strive to make your testing get better, too. Testing after the baseline is established is one of the first things to get minimum attention, even in automated test environments. But improving it reduces the risk of releases, and that’s a benefit to the team and to the company. So it’s worth staying up on.
Unless you have my friend, who we will only call Pete, on your staff. Then, no matter the quality of your testing, no matter the number of tools and layers of testing, he’ll find a way to break your product within 5 minutes. That made him a good tester, made him a better developer and, now, a better architect. But most of us don’t have Pete, so work on those tests.
