Save time with unbiased, independent feedback on vendor solutions.
Watch weekly bite-sized webinars hosted by IANS Faculty.
Creating metrics to support budgeting and decision-making requires a focus on threats, losses and return on controls (ROC). This piece explains how to use a cyber-risk quantification (CRQ) process to create cybersecurity metrics that resonate with the board and senior leadership.
The ideal approach for creating metrics that inform decision-making with an emphasis on budgeting is called cyber-risk quantification. CRQ methodologies help build strategies that rank order security investments based on return on controls. ROC considers
the value the business puts at risk (value at risk), losses transferred to insurance and the value controls bring in reducing threats to the business. Using a canonical effectiveness metrics framework provides a comprehensive approach to CRQ and are
used by the Department of Defense CISO program out of Carnegie Mellon and dozens of major universities and enterprises around the globe. Below are recommended framework steps:
A quick way to get started with a CRQ methodology is to start building a metrics-driven budget by identifying macro asset classes. Assets are a qualitative proxy for the quantitative value at risk. Value at risk and related losses are defined by things
like data loss, business disruption, extortion and wire fraud. For now, you want to identify critical asset groups that, if compromised, could lead to outsized financial impact. Start by identifying non-mutually exclusive assets, such as:
This can be done per critical macro asset at a very high level. Imagine having a general catalog that lists out threats and their loss types. Figure 1 is an example. You want to map the threat and loss combinations that apply to a particular macro asset.
Imagine we have done that for PCI-hosted data center systems. Not all loss types will necessarily apply to every threat in every circumstance. And not all threats apply all the time. The circled threat and loss combinations are the ones assessed as
most pertinent here.
Note how ransomware has three loss types. This means we’ve decided all loss types apply in this instance. However, that is not always the case for all threats or for all macro assets.
READ: Operational Metrics CISOs Should Present to the Board
For controls, consider using the Center for Internet Security (CIS) Critical Security Controls. It maps to several
frameworks and is accepted (and, in many cases, is expected) across industries, even internationally. An additional bonus is that the CIS controls has a metrics list we can adopt. Of course, you are not limited to CIS. It’s just expedient for this discussion.
When starting out with CIS, consider using the Control Implementation Group 1 (IG1). This is the most basic level of cyber hygiene recommended for any business of any size. As your program matures, you can upgrade to IG2 and then IG3.
Your next job is mapping relevant controls and basic metrics onto the list of threats and loss types. This applies per asset group, e.g., PCI-hosted data center systems. The ideal would be to use a multi-tabbed spreadsheet. Each tab covers a macro asset
and its controls, threats, losses and metrics. Figure 2 shows an example for one control (10) with two sub-controls (1 and 2).
Figure 2: Map Macro Assets to Controls, Threats, Losses and Metrics
PCI-hosted data center
Source: IANS, 2023
Note that for a control to be “on,” it must be turned on, configured correctly and up to date. We are taking our lead from the previously mentioned CIS metrics that consider correct configuration and deployment (aka Sigma) level.
Your next job is adding the capabilities (i.e., investments in people, process and technologies) that apply to each instance. This is not a small undertaking. You could have anywhere from five to as many as 20 or more tabs of macro enterprise asset groups,
depending on your level of decomposition. Figure 3 shows what a couple of rows could look like that this point.
You can put specific vendor products in the solutions column. This is particularly useful if you already have pricing.
Starting out use a qualitative approach to aid in prioritization, but eventually, a more thorough CRQ-based approach should be considered. For now:
Review the asset groups with the most in-scope risk classes, the most loss scenarios and 0% controls deployed: These are the control gaps that should likely be rank-ordered first on your budget. Why? Three reasons:
Keep in mind this is a qualitative look at what controls should go into a budget. You could, in theory, have one asset with one risk class and with one loss scenario. Indeed, it could be 70% of the way deployed in terms of required controls. That fact
might tempt you to overlook that asset to focus on less mature and seemingly more risky use cases. You want to avoid that temptation. CRQ approaches catch these potential areas.
Look for the single solution, or group of solutions, that cuts across the most control items: That is, find the controls that seem to have the most risk-reducing impact across asset groups and cost the least. This is a qualitative proxy for finding the
economically best ROC. In this case, you are looking for the best economies of scale from a qualitative perspective to buy down the most risk.
Use this qualitative ROC method to establish controls across your portfolio over time. Work through the critical combinations, increasing the deployment rate across your critical asset classes. Ideally, you would set key performance indicators (KPIs)
to increase the Sigma levels (per CIS metrics) to more than 90% for all critical combinations.
READ: Use a Framework to Create Meaningful Metrics and KPIs
The highest levels of leadership want you to talk about macro KPIs as they relate to upleveling capabilities. Fortunately, CIS, as well as the NIST Cybersecurity Framework (CSF) and others, readily
support this line of reasoning.
As mentioned earlier, CIS uses a concept called an IG, which is somewhat analogous to NIST CSF Tiers. CIS has IG Levels 1–3. You can translate these into qualitative capability-based terms if you like as follows:
Alternatively, you can use whatever makes the most sense for your organization.
Below is an example board narrative based on the approach outlined above:
From that introduction you can share more detailed coverage metrics from an asset and or threat perspective. For example, you could discuss the coverage level for all in-scope assets as it relates to ransomware, phishing, business email compromise or
various classes of insider threat. Or you could focus on cloud assets and/or a combination of asset and threat class.
Although reasonable efforts will be made to ensure the completeness and accuracy of the information contained in our blog posts, no liability can be accepted by IANS or our Faculty members for the results of any actions taken by individuals or firms in
connection with such information, opinions, or advice.
February 29, 2024
By IANS Research
Access key data sets from the 2023 -2024 IANS and Artico Search’s Cybersecurity Staff Compensation Benchmark Report. Gain valuable insights on cybersecurity staff roles to hire and retain top security talent.
Access key data from IANS and Artico Search’s Compensation, Budget and Satisfaction for CISOs in Financial Services, 2023-2024 report. Find valuable insights around the Financial Services CISO role to help better understand your situation, improve job satisfaction and drive organizational change.
February 21, 2024
Learn why cloud IR is critical to security and not just another box to check. Find guidance to get started building a strong cloud IR program.