Have you ever wondered how well your software will perform? What makes for a solid application and what results in defects for customers? Why does a simple fix take months? Why does a simple new feature drag on in development? How maintainable will the software be once released? What level of customer service and sustaining will be required once released?
These issues are collected under the notion of verification (did we build it right) and can be managed and understood by utilizing complexity metrics. This class of metrics relies on logic or conditional statements found in the body of code for each unit of code. The name of a unit is dependent on the software language being used. Some examples are a method, function, or subroutine.
To measure complexity, each unit in the software code base needs to be measured quantitatively and qualitatively. Understanding the architecture of each unit of code that constitutes the code base will facilitate uncovering these issues and answering these questions. As complexity rises, so does the possibility to add defects. A rise in complexity also results in software that is less stable and harder to maintain or modify. The complexity metric is not linear, but rather it is logarithmic in nature.
The McCabe Cyclomatic Complexity Metric provides answers to many of these issues. The complexity metric can be calculated for unit or on an entire project. There are known thresholds for the metric. As code complexity rises, the ability to understand diminishes, and the level of effort for completing full path coverage tests increases.
The ability of the engineering team to understand the code, the level of effort to fully exercise the code, and the quality symptoms are directly related to complexity metric ranges. In my experience, the McCabe Cyclomatic complexity (www.McCabe.com ) levels are as follows:
|
Metric |
Understandability |
Test Effort |
Symptoms |
|
1-10 |
Fully Understandable |
.5 to 1 day |
Easy to add new features. No defects. |
|
11-20 |
Understood with a fair amount of effort |
2-5 days |
Need to reengineer to keep out of the 21-25 metric when modifications will add complexity. |
|
21-25 |
On the edge of understandability |
7-15 days |
Must reengineer prior to any modifications. |
|
26-30 |
Not fully understood without extensive effort. As we near 30, it may not be fully understood. |
1 month |
Need to be re-architected into understandable, logical function breakout. |
|
31-99 |
Highly complex, not fully understood. |
Cannot completely test. |
This is a dangerous unit and will accumulate many defects. |
|
100+ |
Obscure at best. Code will work in ways not intended. |
2-3 months to get 40-50% coverage. |
Most defects will be around these modules. |
When the metric arises above 60, there is no way to understand all the complexities that come with the software. As the metric breaks 100, it results in the most obscure piece of code in the system. This is where a simple fix or a simple new feature does not have a simple solution. As code is added to these highly complex modules (60+), defects are uncovered. I have experienced many cycles of code corrections and defects added during this process. The process to reduce this to a stable module takes between 6-8 weeks and sometimes longer. Once in the field, these highly complex modules are the most fragile, and are subject to failure.
Software maintainability (effort for adding new features/modifications) and field defects (customer found defects) are directly correlated with the McCabe Cyclomatic Complexity Metric.
|
Metric |
Maintainability |
Field Defects |
|
1-10 |
Easy to add new logic and remain below 15. |
Rare occurrence. |
|
11-20 |
Can add new logic and remain below 25 |
A few field defects may occur. |
|
21-25 |
Cannot add further logic. Must find a way to break into 2-3 units when adding new logic. |
First potential for defects in the field |
|
26-99 |
Extensive rework required when adding new logic. This should break into 8+ units as we near a metric of 70. |
Very susceptible to field failure. |
|
100+ |
Rework ASAP. A minimum of 12 units is required. Functional break-out required. If these have made it to the field, it is highly recommended that you talk with customers to understand how they use the system in this related functionality. |
Guaranteed field failure |
There are exceptions to these rules. This occurs when a set of simple logic statements occur within a software unit. One example is a switch statement with many cases and no other logic. It is similar to a post office box. There are many places to send things, but a very simple match against the address results in the proper response. A post office with 50 addresses would have a metric of 50. These units of code can run to up 100 on the complexity metric, but are very easy to test and have the metric of 1-10 symptoms.
This document has been covering verification against the code. Verification is only one aspect of ensuring the product is ready for customers. This results in ensuring that we built the software correctly (verification). Validation (we built the correct features) is complementary and looks at meeting the requirements. Validation is tracked through requirement traceability and not covered here.