Red Hat Researchers: Spectre Chip Vulnerability Likely Worse For VMs Than Containers
The tapestry of security patches currently being applied to computer systems around the world will likely degrade the performance of applications running on virtualized systems slightly more than those running on bare metal or in Docker containers, according to open source software vendor Red Hat.
Software updates preventing hackers from taking advantage of the Spectre exploit—specifically its second variant—will be the culprits in impairing virtual machine performance even beyond environments that don’t sit on top of hypervisors, though the extent of that disparity has not yet been established, Jon Masters, Red Hat's chief ARM architect, told CRN.
Masters has been on the front lines of combating Meltdown and Spectre since well before those chip security vulnerabilities became known to the public last week. Red Hat's Performance Engineering team is still assessing combinations of microcode and kernel patches to quantify performance degradation across environments, as well as workloads and devices.
"The virtualization hit could be minor," Masters told CRN, "but we're still doing a lot of benchmarking to know that. Everyone is still investigating that."
A Red Hat report concluded: "We expect the impact on applications deployed in virtual guests to be higher than bare metal due to the increased frequency of user-to-kernel transitions."
The report said: "due to containerized applications being implemented as generic Linux processes, applications deployed in containers incur the same performance impact as those deployed on bare metal."
Preventing Meltdown, the vulnerability that's unique to Intel processors and easier to exploit, won't disproportionately affect virtualized environments. Nor will patches for variant 1 of Spectre, which concerns a broader array of chip architectures, including AMD and ARM.
However, the second variant of Spectre, known as branch target injection, will, to some uncertain degree, take a higher toll on users running their applications on top of a hypervisor.
Preventing Spectre variant 2, Masters said, "is currently the most-expensive from a performance standpoint."
The fix turns off the chip's indirect branch predictor—a circuit that effectively tries to guess which way coded instructions will execute—every time the operating system kernel or the hypervisor is accessed. Doing so prevents a rogue application from "training" the kernel or hypervisor to allow it to access different applications being executed on the same chip, Masters explained.
Red Hat has released a patch for its virtualization product, based on the KVM hypervisor. VMware and Citrix have also said the Spectre variant 2 exploit is the only one of the side-channel attacks revealed last week that affects their virtualization products, and have made software fixes available.
The complete protocol to guard against that vulnerability involves a microcode patch to the firmware, patching interfaces used by the hypervisors, and the kernels of both the host and guest operating systems.
"That currently has a cost and that cost is pretty high," Masters said. Red Hat and other vendors, especially Google, are trying to develop novel approaches to reduce the burden of implementing those safeguards, he said, and they could become available in a few months.
While accessing the hypervisor isn't more burdensome than the OS kernel, implementing branch predictor fixes on both "can magnify the cost a little bit," Masters said. At the same time, the impact might be mitigated by the tendency of most well-optimized software to rarely call on the hypervisor.
There's also an operation executed when switching between VMs called "flushing the predictor state" that carries a heavy performance cost with the hypervisor patches. But systems aren't supposed to do that operation very often, so it might prove relatively insignificant.
With the complexity of the systems, and the current lack of hard data, no one can definitively state if containers will ultimately benefit from a significant performance advantage over VMs that previously didn't exist.
"I don’t expect the numbers to be that far off in terms of a hit," Masters said about the two environments.
Several partners focused on implementing containers told CRN they are not expecting the technology to take the shock off any performance degradation caused by patching for the security vulnerabilities.
However, containers can help mitigate Meltdown and Spectre problems for customers in other ways, Chris Ciborowski, CEO of Nebulaworks, a container-focused solution provider, told CRN.
"Any process will have a performance hit regardless of virtualization tech, machine or container," he said.
"Containers won’t directly solve the problem. What they will do is allow you to pick up and migrate or scale your services onto machines with AMD CPUs or new Intel hardware easily. That’s the beauty of their portability," Ciborowski said.
Masters warned that for all the problems Meltdown and Spectre have caused across the industry, and the tremendous financial costs those vulnerabilities have and will continue to impose, they might just be the tip of the iceberg.
"Now people are seriously paying attention to something they should have been paying attention to a long time ago—the implementation of security on processors," the Red Hat engineer told CRN.
It is likely new vulnerabilities in chips will continue to surface, Master told CRN. "I think there will be more issues found. This is not over," he said.