One common question during a development cycle is, “are we done?” Answering that question is often met with hesitation or uncertainty. How do we address it? Metrics! As we define our test scope, we have the ability to determine how many test cases are needed to verify the system.
For example, if we have 100 test cases defined, we have an objective and quantitative criteria for completion. Even with that definition in place, an argument can be made that having executed 50 test cases does not translate to 50 percent done because not all test cases will be executed during the same frame of time. While this argument is technically true, having completed 50 test cases will still be an important metrics, considering that you have executed 50 percent of the required test cases to verify that the system adheres to customer requirements. You can still say you are halfway done.
In addition to test case completion, defect count is also a key metric that can be used to signify completion. The goal is to have zero “critical” or “very high severity” defects (how each company categorizes defects will vary, but each company should establish exactly what types of defects will prevent the system from going live in production). Ultimately, as you complete test cases and defect counts are brought down to zero, then you can safely say, “yes, we’re here!”