Xen Hypervisor Vulnerability Exposed Virtualized Servers

A serious security vulnerability (CVE-2014-7188) affecting the open-source hypervisor Xen has forced Amazon, Rackspace and other cloud providers to reboot their systems in order to apply a patch.

The vulnerability, discovered by Jan Beulich of SUSE, affects Xen 4.1 and onward. However, only x86 systems are impacted (ARM systems are not), the Xen Project noted in a security advisory published on Wednesday.

The flaw can be leveraged by a malicious actor who owns a virtualized server to read data from other systems on the host server. An attacker could also exploit the vulnerability to cause the host to crash.

“The MSR range specified for APIC use in the x2APIC access model spans 256 MSRs. Hypervisor code emulating read and write accesses to these MSRs erroneously covered 1024 MSRs. While the write emulation path is written such that accesses to the extra MSRs would not have any bad effect (they end up being no-ops), the read path would (attempt to) access memory beyond the single page set up for APIC emulation,” reads the advisory.

Hosting firm Rackspace rebooted its cloud systems over the weekend to apply the patch made available by the Xen Project. Rackspace notified its customers of the reboot, but didn’t mention anything about the Xen vulnerability to “avoid alerting cybercriminals.” There is no evidence that any data has been compromised due to the vulnerability, the company said.

Rackspace apologized to its customers for not warning them sooner about the reboot.

“This maintenance affected nearly a quarter of our 200,000-plus customers, and in the course of it, we dropped a few balls. Some of our reboots, for example, took much longer than they should. And some of our notifications were not as clear as they should have been. We are making changes to address those mistakes,” Taylor Rhodes, CEO and president of  Rackspace, said in a blog post.

Amazon, which had to reboot roughly 10 percent of its EC2 fleet, told its customers that the maintenance update was related to a Xen security announcement. However, it didn’t provide any details until after the patch was applied.

“Because our customers’ security is our top priority and because the issue was potentially harmful to our customers, we needed to take fast action to protect them. For the reasons mentioned above, we couldn’t be as expansive as we’d have liked on why we had to take such fast action,” Amazon Web Services Chief Evangelist Jeff Barr explained.

Hypervisor vulnerabilities are relatively rare, but as Bromium researcher Rafal Wojtczuk demonstrated at the recent Black Hat security conference, there are several weak spots that can be leveraged in attacks against hypervisors.