Security Experts:

What's the Real Value of "Cost of Breach" Studies?

The European Union Agency for Network and Information Security (ENISA) published The cost of incidents affecting CIIs – a review ‘of studies concerning the economic impact of cyber-security incidents on critical information infrastructures’. Published this month, it is an analysis of ‘cost of breach’ reports; and it draws some worrying conclusions.

For example:

…each one of them examines the topic from a different perspective, focusing on certain industries, using different metrics, counting only certain types of incidents etc. The lack of a common approach and criteria for performing such an analysis has allowed the development of rarely comparable standalone studies, often relevant only in a certain context.

ENISA is not alone in this view. The current Verizon DBIR states,

…we are still looking for the meaning of life and a better predictor of bottom line impact to organizations that suffer a security incident… With limited tangible, hard data available on the cost of breaches, that exercise was not going to be a dragon we attempted to slay.

Cost of Data Breach: Value to Defenders

But other large research organizations do not shy away from the dragon. Ponemon’s latest report puts the average cost of a breach at $4 million, or at $158 per stolen record. In a study conducted for the UK government, PwC put the overall cost of a breach for major companies at between £1.46 million and £3.14 million (smaller companies £75,000 to £311,000). In 2015 Kaspersky Lab put the average direct cost at $551,000 for large companies and $38,000 for SMBs (with indirect costs adding an extra $69,000 and $8,000 respectfully).

This huge discrepancy between different costings places doubt on which ‘cost of breach’ organizations should use in their own risk assessment, and questions the accuracy of them all. If a cost of breach study is relevant to only some companies in only some contexts, what is their real value?

ENISA clearly considers it important. “Knowing the real impact can help define proper, coherent and cost effective (beneficial) mitigation policies.” Since this is not realistic from today’s cost of breach studies, it is worth considering how organizations currently tackle the problem.

Aligning Cost of Breach with Mitigation Policies

The general consensus is that a third-party cost of a breach is useful, but not necessarily something that can be used to form the bedrock of a risk mitigation policy. Chris Kellogg, director of service reliability engineering at a healthcare technology company explains: “The absolute cost of a breach is still one of the foundation stones for risk-based security as it provides a basis in financial impact on which to measure investment. It serves as a good reminder headline for why this conversation is important when we enter the boardroom.”

But that absolute cost is still only part of the equation. “It's just as important to consider the soft cost and long-term impact of such an event –  brand management, customer relationships, increased legal and regulatory scrutiny, etc –  as these have greater potential energy than the hard cost in a breach event, particularly when insurance programs are in place.”

Steven Lentz, the chief security officer at Samsung Research America, has a similar take. Yes, it is useful, but, he suggests, “Putting a cost to a breach is pretty much subjective, and it’s hard to gauge the cost of subjective data breaches… you need to look at the consequences of a breach. What will it cost besides money, such as reputation; and how do you put a cost on that? Or is it a good marketing tool? Look at Apple: every time the iPhone leaked, big headlines. But the end result is record sales. Was the breach intentional or just good luck? Well it happened twice...”

In reality, Lentz ignores the cost of breach studies for two reasons. “My goal is to mitigate [the effect of a breach]. I cannot stop a determined user – they will always find a way.” His purpose is to keep the company’s employees productive and to minimize the effect of any breach, rather than use excessive controls, make life difficult, and still get breached.

The second reason is that he simply isn’t sure that you can get an absolute cost of a breach. As a research company, his own ‘crown jewels’ are intellectual property. “Our risk lies in the future with patents,” he explained. “It’s hard to put a cost on data and ideas that are many years out and depend on whether the patent takes off or not. I factor in how to mitigate a breach, not its cost. I think reputation and headlines are the biggest risk.”

Charles White, the CEO and founder of risk management firm IRM, isn’t sure that it is meaningful to seek an overall cost of breach. “The cost of a breach to each organisation can vary enormously depending on what assets are targeted, how important they are to this particular company, and what recovery capabilities they currently have,” he explains. “The theft of the exact same set of data could incur wildly different costs on two organisations based on the way they utilise the data and how quickly they can get back on track.”

He believes that companies need to focus on their own situation rather than rely on third-party averaged figures. He is concerned, however, whether many companies are even aware of their own assets. “Our Risky Business Report found that more than a third of CISOs have no clear idea of what assets their businesses have or where they are located on the network. Further, only 28 per cent regularly conduct exercises to categorise and value the data within their IT estate.”

To White, the reality is that third party breach cost reports are of limited value; but that many companies are not capable of generating accurate cost projections for themselves. But it gets worse. “CISOs not only need to be aware of the value of their assets,” he continued, “but they also need to effectively communicate the associated risks and costs to the board if they are to effectively protect them.”

Cost of Breach and Other Metrics

Despite ENISA’s belief that the cost of a breach can help an organization to “define proper, coherent and cost effective (beneficial) mitigation policies,” this is not universally accepted. One issue appears to be a fundamental difficulty in aligning the cost of a breach and the risk of a breach. “The cost of a breach and its relevance to a risk based approach to security has the ability to be aligned, but depending on how the business mixes and matches the model this effort can also become very skewed,” comments Todd Borandi, a lead information security architect.

“Take the basic concept of a qualitative vs quantitative approach to metrics,” he continued. “What I have seen is multiple organizations attempt to mix the two measurements, and then throw in their own formulas to try to account for ‘acceptable risk’. So you end up with a risk ‘value’ of 2 and a dollar estimated loss of $1.5 million... What does this even mean once an organization is compromised?”

Torsten George, VP global marketing and products at RiskSense, isn’t sure that aligning cost and risk is really relevant. “While a risk-based approach to security is the only way to have a chance in tackling today’s dynamic threat landscape, risk in itself is defined differently from company to company. Thus, reports on the ‘cost of data breaches’ are not something that are easily comparable – as ENISA points out. However, they’re good instruments for practitioners to raise awareness and kick off an internal discussion to move from a compliance, check-box mentality to a more pro-active, risk- and business-driven approach to security.”

Risk, he suggests, “is made up of multiple factors (e.g., compliance posture via compensating controls, security posture with both vulnerability and threats, as well as business criticality), and organizations have to understand that focusing on just one of these factors will not derive the necessary results needed to stand up to cyber-attackers.” Aligning all of these factors in risk-based security to a global cost of breach figure may not be all that meaningful.

The way forward

ENISA is confident that cost of breach is a valuable metric. It suggests that it doesn’t currently exist; but suggests that it could be done. “The development of such studies in the future should be done through a unified analysis, based on a well-structured methodology, and considering all critical variables,” it states. “ENISA could be an actor capable of doing such work, but only with suitable resources and a clear mandate in this area.”

Practitioners remain unconvinced. “How can this be?” asks Lentz. “With security being so dynamic it’s hard to put a standard in place. By the time they do it will be outdated.”

Kellogg commented, “Even with a definition of common variables and calculations, every company has different values for those variables and different factors for those equations. The net cost of a breach will vary widely based on the company's industry and infrastructure (people, process, tools), making comparisons within an industry inconsistent and across industries difficult; while leaving aside the impact of breach mechanism and breach type.”

Borandi adds, “This is a guessing game and should be regarded as an estimate regardless of the metrics or methods used. A business will not fully realize the monetary loss until it understands the depth and breadth of the exploit. Sure, you can estimate the cost for credit protection for every record lost, but even then factors like media coverage and customer response are unknown factors which play a huge role in determining brand damage and overall costs.

“There is no exact science to determine these things, just as there is no exact science to predict what security gaps are going to cause an exploit.

“Risk is exactly that, a risk – and the metrics used can be full of complicated outdated formulas or approached very simply; because honestly once an organization is compromised, the estimated risk scores are the very last thing anyone ever looks at after the fact.”

view counter
Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines.