A number of perspectives about vulnerability risk management have suggested that the following metrics are what’s most important for an enterprise vulnerability management program:
- Accurate asset inventory: “Do we know about all the resources for which vulnerabilities need to be managed?”
- Scanner coverage: “Are we finding all the vulnerabilities we should, as frequently as we should?”
- Issues by status: “What is the status of the vulnerabilities we know about, over time?”
- Remediation time: “How quickly are known vulnerabilities being addressed?”
- Inadequate fixes: “How effectively are known vulnerabilities being addressed?”
On the surface, these seem quite reasonable. In fact, they align fairly well with the metrics published by the Center for Internet Security, which were developed from a consensus reached by a group of 150 experts from the government, private industry, and academia. Their metrics for vulnerability risk management include:
- Number of known vulnerability instances: “How many vulnerabilities do we know about?”
- Vulnerability scan coverage: “Are we finding all the vulnerabilities we should?”
- Mean-time to mitigate vulnerabilities: “How quickly are our known vulnerabilities being addressed?”
- Percentage of systems without known severe vulnerabilities: “Have the most severe vulnerabilities been addressed?”
However, none of this is really dealing with risk, at least not as risk is properly defined. As security professionals, we have a tendency to talk on and on about threats, vulnerabilities, and exploits, and we think we’re talking about risk. But when we speak about security risks, we should be speaking about the probability of successful exploits of vulnerabilities and the magnitude of the corresponding business impact. If we aren’t talking in these terms, then we aren’t actually talking about risk.
Frankly, the one metric that the business really cares about is this: “How much have our vulnerability management activities actually reduced our risk?”
The nine metrics enumerated above are, in a way, circling around the topic of risk—if we know what resources we have, if we know what vulnerabilities those resources have, and if we do a timely and effective job at addressing those vulnerabilities, then we are reducing the likelihood that a threat agent will be able to exploit those vulnerabilities and cause harm to our organization or its customers.
That’s pretty indirect, though. It says nothing about the magnitude of the corresponding business impact should any potential exploits be successful. And it says nothing about the actual risk reduction from all of this activity compared to the cost of performing all of this activity. That’s what the business decision-makers really care about.
One way to solve this problem is to be honest with our language. Just refer to it as “managing vulnerabilities,” for example, and there would be no issue. But let’s not pretend that it’s about managing risk, when it’s really not.
I’ve tried to start analyzing and writing this way in my own work—and I can tell you that it isn’t easy, at least not at first. Here is one recent example, which happens to be in the area of endpoint protection. I hope to get even better at it with time.
By taking a different look at how we approach vulnerability risk management, we are opening the doors for further discussion and exploration into how we can improve our risk analysis and reduction strategies.