Security Experts:

Mitre ATT&CK Matrix Used to Evaluate Endpoint Detection and Response Product

The growing acceptance that it is impossible to detect and block all malware at the perimeter requires some form of response to malware post-breach. Endpoint Detection and Response (EDR), using machine learning behavioral rules to detect an intrusion, is the security industry's reply.

Anti-malware testing, however, is still largely predicated on malware detection, leaving the efficiency of the response side of EDR less well understood. EDR firm Endgame has sought to address this by using the Mitre ATT&CK Matrix to emulate the post-breach tactics used by the China-based APT3 group.

Mitre's Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) Matrix is a curated framework that describes the techniques used by adversaries once they are inside a network. It lists, under 10 tactic categories, all known tactics, techniques, and procedures (TTPs) used by adversaries to make decisions, expand their foothold and execute their intentions. These 10 categories derive from the later stages (control, maintain, and execute) of Lockheed Martin's seven-stage cyber kill chain.

The ATT&CK categories comprise persistence, privilege escalation, defense evasion, credential access, discovery, lateral movement, and execution.

“To best understand what an adversary can do post-exploit,” explains Mitre's Frank Duff, “we released the ATT&CK framework. The next logical step is to show how ATT&CK can be actionable, and we have done so with ATT&CK-based adversary emulations. These emulations provide a method to prove the effectiveness of security solutions against known threats.”

It was the APT3 attack emulation against which Endgame was tested. Endgame, the firm announced today, successfully stopped APT3 in the emulation exercise before any data theft or damage would have occurred. “At Endgame,” commented CTO Jamie Butler “we're committed to holding ourselves to the highest standard of protection, which means going beyond malware-based testing regimens to include post-exploitation techniques. I encourage other security vendors to expand their measurement criteria to include the Mitre's ATT&CK Matrix to clearly demonstrate the true value of protections for customers.”

Endgame was founded in 2008 by Chris Rouland and other executives who previously worked with the CIA and Internet Security Systems. It originally concentrated on government customers, and is still strong in this area. The HB Gary leak of 2011 indicated that the early Endgame sold zero-day vulnerabilities. By 2014, the firm had ceased this involvement and used a $23 million Series B funding round followed by a $30 million Series C round to finance the growth of its commercial offering in both the federal and commercial market.

The firm has a tradition of innovation. It uses adversarial machine learning to probe (and improve) its own machine learning defense product. It was an early 'next-gen' adopter of 'traditional' third-party anti-malware testing, and became a member of the Anti-Malware Testing Standards Organization (AMTSO) in January 2017. “As advanced attacks become more pervasive,” said Endgame's director of threat research and adversary prevention Mark Dufresne at the time “it's critical that we establish objective testing methodologies that enable enterprises to make transparent buying decisions based on efficacy of solutions. We look forward to working closely with AMTSO.”

In December 2016 it was certified in an independent evaluation by SE Labs, scoring 100% efficacy. In June 2017 it achieved a 99.5% protection rate in a test conducted by AV Comparatives. This new expansion into using Mitre's ATT&CK Matrix is designed to highlight the ability of the 'response' side of EDR if (and when) detection fails.

SE Labs director Simon Edwards told SecurityWeek in an emailed statement, “We use a similar matrix when building targeted attacks and judging systems designed to detect/ protect against such attacks. We approached the project by analyzing real-world breaches and coming up with different criteria for each part of the attack chain. We did this independently of Mitre but the results are not very different.”

“We are engaging with the security industry to encourage this thinking,” added Mitre's Duff, “so that they can effectively articulate their capabilities to our government partners, as well as the public. We look forward to continuing to work with commercial vendors to articulate their capabilities in the future.”

Related: Firms Unite to Hunt Threats From Network to Endpoint

Related: Inside The Competitive Testing Battlefield of Endpoint Security

view counter
Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines.