Embattled cybersecurity vendor CrowdStrike on Tuesday released a root cause analysis detailing the technical mishap behind a software update crash that crippled Windows systems globally and blamed the incident on a confluence of security vulnerabilities and process gaps.
The new CrowdStrike root cause analysis documents a combination of factors that caused the Falcon EDR sensor crash — a mismatch between inputs validated by a Content Validator and those provided to a Content Interpreter, an out-of-bounds read issue in the Content Interpreter, and the absence of a specific test — and a vow to work with Microsoft on secure and reliable access to the Windows kernel.
“Sensors that received the new version of Channel File 291 carrying the problematic content were exposed to a latent out-of-bounds read issue in the Content Interpreter. At the next IPC notification from the operating system, the new IPC Template Instances were evaluated, specifying a comparison against the 21st input value. The Content Interpreter expected only 20 values,” CrowdStrike explained.
“Therefore, the attempt to access the 21st value produced an out-of-bounds memory read beyond the end of the input data array and resulted in a system crash,” the company said.
“While this scenario with Channel File 291 is now incapable of recurring, it also informs process improvements and mitigation steps that CrowdStrike is deploying to ensure further enhanced resilience,” the EDR vendor said.
The company said its kernel driver, which is loaded early in the system boot process, allows the Falcon sensor to observe and defend against malware that launches before user-mode processes start and pledged to update its agent to leverage new support for security functions in user space, reducing reliance on the kernel driver.
“As new versions of Windows introduce support for performing more of these security functions in user space, CrowdStrike updates its agent to utilize this support. Significant work remains for the Windows ecosystem to support a robust security product that doesn’t rely on a kernel driver for at least some of its functionality. We are committed to working directly with Microsoft on an ongoing basis as Windows continues to add more support for security product needs in userspace,” the company said (PDF).
CrowdStrike also announced it has engaged two independent third-party software security vendors to conduct an extensive review of the Falcon sensor code for security and quality assurance. In addition, the companies said an independent review of the end-to-end quality process from development through deployment is underway, with a particular focus on the impacted code from July 19.
The release of the root cause analysis comes as CrowdStrike and Delta Airline publicly battle over who is to blame for damage that the airline suffered after a global technology outage. Delta’s CEO has threatened to sue CrowdStrike for what he said was $500 million in lost revenue and extra costs related to thousands of canceled flights.
Related: CrowdStrike Says Logic Error Caused Windows BSOD Chaos
Related: CrowdStrike Faces Lawsuits From Customers, Investors
Related: Insurer Estimates Billions in Losses in CrowdStrike Outage Losses
Related: CrowdStrike Explains Why Bad Update Was Not Properly Tested
