It’s been almost five years since we launched Bromium at Structure 2011 – showing the world that we could use CPU virtualization to hardware-isolate individual OS tasks to enforce least-privilege separation. Our approach allows an un-patched / vulnerable endpoint (client, server, cloud) to protect itself, even on an unprotected network, by reducing the attack surface of the system by many orders of magnitude.
Today, as I unthinkingly click on links, docs and exes I will probably create and destroy 200 micro-VMs – each of which takes only a few tens of milliseconds to launch. Collectively Bromium users create and destroy tens of millions of micro-VMs per day – without knowing it. This is a rather arbitrary stat, but cool nonetheless. It’s probably close to the number of VMs booted and stopped in AWS in a day, and almost surely greater than the number in Azure or any other cloud. Quite an achievement, but we are a long way from done…
Our journey has taken a little longer than we thought it would, and has required our dev team to solve the fiercest of technical problems. But although micro-virtualization seemed like a crazy idea when we started, it is exciting to see the concept taking hold more broadly – with other vendors developing infrastructure abstractions that are philosophically aligned with our approach to using virtualization to enforce least-privilege. As we head into RSAC week, I want to use this blog to recap our approach and to showcase the work of other vendors & communities that are adopting light-weight “micro-virtualization” to solve problems in infrastructure security. Note that here I use the term “micro-virtualization” broadly – to mean lightweight, task-centric, hardware-isolated execution – and not to narrowly refer to Bromium’s use case or technology.
A working model for micro-virtualization
Micro-virtualization is a second-generation CPU virtualization technology that extends the isolation, control and security principles of hypervisor-based virtualization into an OS and its applications using CPU features for hardware virtualization to isolate individual application tasks or OS services. It provides a powerful hardware backstop for granular enforcement of generally accepted principles of separation by least-privilege, and reduces the attack surface of the system by taking advantage of an additional ring of hardware protection and the small code-base of a light-weight hypervisor that we call a Microvisor. Micro-virtualization can be used within the OS to protect high value services/data, and it can be added to an existing legacy OS+apps to isolate untrustworthy execution, such as tabs in a browser or documents. The Microvisor enforces least-privilege access control using hardware isolation. For example it can ensure that high-value files, networks, sites, shares and devices are not available to an untrusted, isolated task. This prevents an attacker from accessing valuable data, networks or sites, or accessing devices.
Micro-virtualization is simply an extension of well established principles of OS virtualization, and a traditional hypervisor can be augmented to support micro-virtualization. At Bromium we have augmented the Xen hypervisor, which we call uXen, to run both traditional VMs and hardware isolated tasks in micro-VMs.
A micro-VM is a hardware-isolated, least-privilege enforced execution construct that executes a component of an application (eg: a browser tab or a Docker container) or a component of an operating system (eg: Credential Guard in Windows 10). Unlike a user-mode “virtual container” or “sandbox”, a micro-VM supports execution of both user- and kernel-mode code – all of which is isolated from the rest of the system.
Micro-VMs execute in SLAT isolated memory and are subject to least-privilege control for access to all system resources – including device access, networks, sites, files, shares and access to critical OS services. Least privilege is enforced by the Microvisor when a micro-VM attempts to access a resource, resulting in a hypercall. During this enforcement, micro-VM execution is paused by the CPU. Fortunately, micro-virtualization generally does not require changes to be made to the isolated applications or tasks, through there are some use cases (eg: Credential Guard in Windows 10) where a system is modified to take advantage of micro-virtualization.
On an end-user device, the user is unaware of the presence of the Microvisor or micro-VMs. In the cloud, micro-virtualization can be used to hardware-isolate and enforce mutual separation between application components or container-based applications. Other use cases follow.
For use cases where micro-virtualization is used to ensure safe execution of untrusted code, the execution within a micro-VM is copy-on-write. Any changes to user- or kernel-mode memory or to system state (eg: file system, registry) are made to efficiently managed local copies that are isolated in the micro-VM and discarded when the task terminates. This eliminates persistence for malware or unwanted side effects of execution.
For use cases where micro-virtualization is applied to isolate tasks or services of high value, eg: Credential Guard in Windows 10, the micro-VM need only provide simple API-level access to its protected service. Such micro-VMs typically have no need for device access, network services or access to the user desktop. They must be protected from DMA attacks using an IOMMU, limiting the applicability of this approach to modern systems that offer such protection. Their implementation is simple because they have no impact on end user workflows, and can be hidden within an operating system, but they are necessarily a designed-in feature of the system that requires source code access.
Finally, because each micro-VM executes only a single task the job of identifying malicious execution in a micro-VM is also dramatically simplified. Using Microvisor-based introspection it is easy to detect the side effects of malicious execution when an attack has actually executed, as opposed to trying to detect malicious software or activity before it executes its attacks. The net consequence is that micro-virtualization enables a detection system to eliminate false-negatives (since the micro-VM is always discarded) and to reduce the rate of false-positives by waiting for clear evidence of an attack. Finally, the CoW execution “diffs” capture all changes to memory, the file system, registry and even packets sent/received, facilitating a rapid forensic analysis of the execution context to identify relevant threat information.
Lessons We’ve Learned
- Hardware isolation is the industry’s most robust tool for protection: Using the endpoint CPU to enforce the principle of least privilege is the best way to protect against attacks on the OS or its applications. Software abstractions – “virtual containers” and other software based endpoint protection can typically be easily bypassed. At the end of the day the real problem is the reliance on only two rings of CPU protected execution by all widely used OSes. Virtualization in effect allows us to dynamically grab an additional hardware protection ring on the CPU, use it to granularly enforce least privilege separation on legacy, vulnerable OS and app stacks, and reduce the attack surface of the system to the Microvisor, which is many orders of magnitude smaller than the OS itself.
- Micro-virtualization can be deployed and managed at scale: Adding micro-virtualization to existing,endpoints in the largest organizations has been challenging, but I’m proud to say we’re well on our way. We know the benefits that accrue: The endpoint protects itself “by design” on an unprotected network, running un-patched software, in the hands of a user who clicks on malware. The endpoint self-remediates and delivers precise, real-time forensics for each attack. Bromium receives reports daily of malware that bypassed millions of dollars of sophisticated network devices to face defeat in a simple micro-VM on the endpoint.
- Using micro-virtualization as an infrastructure building block to help secure endpoints by design – both clients and clouds – is both viable and within reach. It has the potential to revolutionize infrastructure security. We are still at the beginning of that journey but the benefits are incredibly powerful. Broader industry adoption of this technology/approach is required, and fortunately it is coming. Micro-virtualization is going mainstream: It’s been inspiring to see other vendors and communities build on the idea of ephemeral, hardware-isolated tasks/apps, to address a range of needs:
- Bromium has partnered with Microsoft to help harden Windows 10 using micro-virtualization “in the box”, but Microsoft is focusing also on other use cases – including the use of micro-virtualization for light-weight, hardware-isolated Hyper-V Containers on Windows Server 2016 and Azure.
- Windows 10 Credential Guard uses micro-virtualization to help prevent pass-the-hash attacks.
- CoreOS has followed Intel’s leadership in Clear Containers, embracing hardware-isolation for containerized Docker
- VMware projects Lightwave/Photon promise a light-weight virtualization abstraction to hardware-isolate containers.
- Docker’s recent acquisition of Unikernel Systems raises the tantalizing prospect for tiny, ephemeral, secured containers as building blocks for secure infrastructure.
- Micro-virtualization revolutionizes detection: In the hardware-isolated confines of a micro-VM there is no need to detect malware before it executes – it is OK to just wait for it to execute and to “do bad”. It is easy to spot malicious execution and its results, and all side-effects of execution can be recorded – every packet, memory change, file or registry change. When malware executes the Microvisor can deliver precise detailed forensic information in real time, before simply erasing the micro-VM, eliminating persistence.
Virtualization is a fundamental infrastructure building block – for secure cloud and mobile endpoints – that offers enormous benefits. Hardware enforced execution isolation improves security far more than any other technology. There is a fundamental IT lesson here: You should move forward fast. Adopt virtualization everywhere – clouds and clients, segment your networks, distrust your endpoints, assume your network is indefensible and that your users will click on bad stuff.
If you’d like more information or a demo, please reach out to me directly