Coverage analysis of software testing reflects the level of testing that has been accomplished. Ultimately, as you move up the coverage scale the release quality of the software is dramatically improved.
There are 2 types of test coverage analysis currently available, path and code. Code coverage tools show the lines of code you have executed, and missed. The summary is normally given in a % coverage… i.e. nn% code coverage. This is the first step to improving quality. Path coverage is more complex and results in higher quality. Path coverage looks at the logic paths in the code and takes into account all positive and negative outcomes against each logic statement (1 true and 1 false combination for each logical statement). Path coverage tools graphically depict the logical map of the source code. This map shows what is executed and what is missed. Path coverage is summarized with % coverage i.e. mm% path coverage. There are also a set of test cases specified by some toolsets, that if executed will result in 100% path coverage.
Path coverage has a distinct advantage. There is a fair amount of logic that has no software for false conditions on a logic statement, for instance and IF – END IF scenario without an ELSE statement or a SWITCH statement without a DEFAULT CASE. In these cases, if you only run the positive test, you will get a 100% result in code coverage. However, there is only a 50% result in path coverage for the IF statement and a little more depending on the number of cases in a SWITCH statement .
Code coverage runs up faster than path coverage for the reasons given above. Some examples of testing coverage results follow (from experience with McCabe toolset):
|
Path Coverage |
Code Coverage |
Quality |
|
20% |
30-40% |
Nominal coverage when not tracking path coverage. |
|
40% |
50-60% |
A good first target when starting path coverage. |
|
60% |
75-85% |
World class software. |
|
80% |
95%-98% |
Software that shall not fail – lives are at stake (medical, defense, NASA…) |
Path coverage is more difficult to hit. Many projects are at the 20-30% path coverage as we first run the coverage analysis tools against the preexisting test suite. It takes a directed effort, with the proper toolset to track progress. The automation of data driven tests is crucial to path coverage. It required engineering analysis into development of the data required to hit each path through the source code. As you move up the path coverage metric the result is a higher quality set of source code, with ultimately a set of software that will not fail.
