Cyber Risk Assessment: Moving Past The "Heat Map Trap": by Vince Dasta, Protiviti Associate Director

Cyber Risk Assessment: Moving Past the
“Heat Map Trap”

By Vince Dasta, Protiviti Associate Director
Given the limits on time, attention and resources with which every cyber team must contend, risk
assessment plays a critical role in helping set priorities and decide between options. Having a rigorous and
accurate risk assessment process goes a long way in determining an organization’s cybersecurity
performance.
Unfortunately, our observation has been most cybersecurity professionals significantly overestimate the
quality of their risk assessment programs. The common weakness? A reliance on what can be called
“pseudo-quantitative” methods, in which risks, benefits and other factors are given labels or colors (such as
red, orange, yellow and green) or ratings on an ordinal scale that run, say, from 1 to 5. These approaches
have the veneer of objectivity but are actually highly subjective. The illusion of objectivity is all the more
deceptive because the frequent use of scientific-looking heat maps. The problems become apparent as soon
as one starts asking questions: How clear is the differentiation between red and orange risks? Does a
consequence rated a “4” have twice the impact of one rated a “2”? Just because you’ve assigned numbers to
risks doesn’t mean you can do math with those numbers.
Figure 1: A typical heat map. How well-defined is the difference between a “high financial loss” and a “large financial loss”?
Yet despite the limits to the pseudo-quantitative approach, it is often the basis for setting strategy and
budgets with a high level of specificity. This was brought home to us at Protiviti when we recently co-
sponsored a survey of 1,300 executives regarding the cybersecurity practices of their organizations. Of the
nearly 850 respondents whose firms weren’t using quantitative risk assessment, nearly half said their
companies had a less than 10 percent chance of suffering more than $1 million in cyber losses in 2019 —
despite the fact that these companies, without quantitative risk assessment, were unlikely to be able to base
such a claim on anything more than gut instinct. Call it the “Heat Map Trap.”
The good news is that there is a clear path to implementing a risk assessment program that is authentically
quantitative and in which confidence is justified. The first step is to clearly delineate the issue at hand and
Internal Audit, Risk, Business & Technology Consulting

the possible outcomes. Take, for example, the essential task of evaluating the relative impact of various
cybersecurity risks. One such risk might be that of a non-malicious insider clicking on a phishing email that
gives an intruder access to financial accounts. That threat scenario is actually a string of several separate
events, each of which has a range of probabilities of occurrence and losses attached to it (a distribution).
This deconstruction of larger business questions into their components is the most challenging step in the
process — it’s common in first attempts at quantitative modeling to struggle with data and estimates
because the scope or scenario hasn’t been properly defined. But these speed bumps are easily overcome
with guidance and experience.
Once the models have been developed, they are then populated with the distributions of frequencies and
losses associated with each event. These distributions let us account for the level of uncertainty in our
estimates, the less sure of the actual value we are the wider the distributions become. We have found that
meaningful insights can still be obtained even when we have seemingly large degrees of uncertainty and
wider ranges of possible values.
Figure 2: Proper quantitative risk assessment reduces risk to combinations of probabilities and losses.
This is the critical point where the subjectivity of the heat map is replaced by the objectivity of tangible
data. It’s also the point at which many people shy away from quantitative risk assessment because they
don’t have all (or any of!) the data they need. But it’s a fallacy to think that you have to have perfect data —
even imperfect data can provide meaningful estimates that lessen the uncertainty of the analysis. (There
are also cybersecurity data sets that have been validated over time that can be used until the enterprise
begins to collect its own data.) And remember that we use statistical methods precisely because we don’t
have complete information. If we had 100% complete data we wouldn’t need to do an analysis at all.
Once frequencies and loss distributions have been assigned, Monte Carlo simulations generate a
probability distribution curve plotting the likelihood of a loss exceeding a certain amount. This gives cyber
decision makers a much clearer view of the threats that they are facing and the relative benefits of
mitigating those threats — or, to take the question we asked on the survey, the probability of suffering a
cybersecurity loss of more than $1 million over the course of a year.
Translating the problems or questions that arise-in decision making into probabilities and losses is
necessary to make meaningful comparisons of different options. In the hypothetical example below, the
orange data shows how addressing certain issues in a system reduces expected losses, allowing for a much
more concrete cost-benefit analysis of that plan of action.

Figure 3: Decision making is easier when you can make direct comparisons between courses of action.
According to our cybersecurity survey, half of the companies that are not currently using quantitative risk
assessment expect to do so within the next two years. Most of those firms will find that starting with a
clearly defined pilot program is the most convincing way to establish the proof of concept necessary for
wider adoption.
Cybersecurity professionals labor in an environment in which little is certain. But even if it is full of
shadows, there are definite contours to the landscape. Quantitative risk assessment offers a way of
determining the shape of those contours and enabling more-confident decision making at a time when
every organization finds its cybersecurity function under scrutiny.
This article was originally published on The Protiviti View.
Article from Protiviti KnowledgeLeader – www.knowledgeleader.com
KnowledgeLeader is a subscription-based website that provides audit programs, checklists, tools, resources
and best practices to help internal auditors and risk management professionals save time, manage risk and
add value. Free 30-day trials are available.

Cyber Risk Assessment: Moving Past The "Heat Map Trap": by Vince Dasta, Protiviti Associate Director

Uploaded by

Copyright:

Available Formats

Cyber Risk Assessment: Moving Past The "Heat Map Trap": by Vince Dasta, Protiviti Associate Director

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cyber Risk Assessment: Moving Past The "Heat Map Trap": by Vince Dasta, Protiviti Associate Director

Uploaded by

Copyright:

Available Formats

Cyber Risk Assessment: Moving Past the

“Heat Map Trap”

Internal Audit, Risk, Business & Technology Consulting

Internal Audit, Risk, Business & Technology Consulting

This article was originally published on The Protiviti View.

Article from Protiviti KnowledgeLeader – www.knowledgeleader.com

Internal Audit, Risk, Business & Technology Consulting

You might also like