Physical Memory, Virtual Memory, and Address Translation
A modern operating system running on a physical computer uses a combination of physical memory and virtual memory to support the concurrent execution of a large number of processes. Physical memory provides an address space that is limited by the size and number of memory modules actually installed in the computer. In contrast, virtual memory presents a larger address space that results in the ability to run more processes concurrently than actually fits in physical memory. Conceptually, an operating system generally supports:
An address translation scheme (or mapping) between the virtual memory address space and the physical memory address space
A page table that stores the virtual memory to physical memory address space mapping information
A paging method that swaps data between physical memory and secondary storage devices, typically hard drives, using memory pages
On the processor hardware side, a Translation Lookaside Buffer (TLB) supports virtual memory to physical memory address translation. A TLB is a processor cache that contains recently used mappings from the page table. When a virtual to physical address translation is required, the TLB is checked to determine whether or not it contains the mapping information. If a match is found, the physical memory address is provided and data access proceeds. If no match is found, a page fault occurs, and the operating system checks the page table for the mapping information. If the mapping information is found, it is written to the TLB, the address translation takes place, and then the data access proceeds. Figure 1 illustrates the address translation steps when the mapping must be loaded into the TLB from information in the page table.
Figure 1: Virtual to physical address mapping with page table lookup
Address Translation in Windows Server 2008 Hyper-V
Virtualization complicates the mapping between virtual and physical memory address spaces. In Windows Server 2008 Hyper-V, virtual machines load and run concurrently in actual physical memory (i.e., no oversubscription of memory or page sharing between virtual machines). The amount of memory configured and used by a virtual machine is defined when it is created and stored in the virtual machine settings. The Windows hypervisor performs the allocation from physical memory when the virtual machine is powered on, and the amount of virtual machine system memory cannot be modified while the virtual machine is running. In addition, the Windows hypervisor manages the following memory address spaces:
System physical address (SPA)
Guest physical address (GPA)
Guest virtual address (GVA)
The SPA space is the physical memory address space of the computer on which the Windows hypervisor is executing. The GPA space is the physical memory address space of a child partition (virtual machine). Each child partition has a single GPA space just like every physical computer has a single SPA space. For each child partition, there is also a virtual address space referred to as the GVA space. Similar to a physical environment, the GVA space provides an address space that is much larger than the GPA space, and pages of memory that are not mapped to a GPA location are paged to disk.
It is important to note that the parent partition is treated as a special child partition that owns the SPA space. Although the parent partition can page virtual memory pages to disk, child partitions do not have this privilege.
As you can see in Figure 2, the Windows hypervisor manages not one, but two levels of address space translation for a guest operating system running in a virtual machine. The first level of translation maps the GVA space into the GPA space. The second level of translation maps the GPA space into the SPA space. In order to do this, the Windows hypervisor maintains a shadow page table that combines the two levels of address space translation into a single page table. Every time the guest operating system modifies a page mapping, the hypervisor adjusts the shadow page table to reflect the modification. From an overhead perspective, continuous updates of the shadow page table require many processor cycles. For memory intensive applications, this process can make up the largest part of the performance overhead for virtualization.
Figure 2: Example of mappings between the SPA, GPA, and GVA in Windows Server 2008 Hyper-V
Second Level Address Translation in Windows Server 2008 R2 Hyper-V
With respect to memory management, Windows Server 2008 R2 Hyper-V supports a new feature named Second Level Address Translation (SLAT). SLAT leverages AMD-V Rapid Virtualization Indexing (RVI) and Intel VT Extended Page Tables (NPT) technology to reduce the overhead incurred during virtual to physical address mapping performed for virtual machines. Through RVI or EPT respectively, AMD-V and Intel VT processors maintain address mappings and perform (in hardware) the two levels of address space translations required for each virtual machine, reducing the complexity of the Windows hypervisor and the context switches needed to manage virtual machine page faults. With SLAT, the Windows hypervisor does not need to shadow the guest operating system page mappings. The reduction in processor and memory overhead associated with SLAT improves scalability with respect to the number of virtual machines that can be concurrently executed on a single Hyper-V server. As an example, the Microsoft Remote Desktop Services (RDS) team recently blogged about performance tests conducted using an internal simulation tool on a Windows Server 2008 Terminal Services configuration running as a virtual machine on Windows Server 2008 R2 Hyper-V. The results showed that a SLAT-enabled processor platform increased the number of supported sessions by a factor of 1.6 to 2.5 when compared with a non-SLAT processor platform. Overall, Microsoft reports that with SLAT-enabled processors, the Windows hypervisor processor overhead drops from about 10% to about 2%, and reduces memory usage by about 1 MB for each virtual machine.
Although RVI is not required to support workloads running on Windows Server 2008 R2 Hyper-V, if you intend to run memory-intensive workloads like Remote Desktop Services, SQL Server, or web services, you should perform testing on SLAT-enabled platforms to determine whether or not you can gain significant performance improvements.
AMD-V RVI Processor Support
If you are evaluating an AMD-V platform for your Windows Server 2008 R2 Hyper-V deployment, AMD first introduced RVI in 2007 with their Quad-Core AMD Opteron processor code-named “Barcelona”. RVI is currently supported in all Quad-Core and Six-Core AMD Opteron processor models.
In addition to RVI, another AMD-V feature that supports SLAT is the Tagged TLB which extends TLB functionality. Originally, the TLB was designed to contain virtual to physical address mappings for a single operating system. The AMD-V architecture extends this functionality by adding an Address Space Identifier, or ASID. An ASID is a unique identifier assignment in a TLB to distinguish between coexisting Hyper-V server and virtual machine memory mapping entries, and it helps to improve the performance of a context switch by removing the need to flush the TLB each time processing switches between virtual machines. It is important to note that for multi-core processors, each processor core maintains an individual TLB.
Intel VT EPT Processor Support
Intel supports EPT in Dual-Core and Quad-Core Xeon Processor 5500 processors (code name Nehalem-EP), and in the next Xeon processor release (code name Nehalem-EX) which will feature up to eight cores. Nehalem-EX processors are targeted for release in early 2010.
Similar to the AMD-V Tagged TLB feature, Nehalem processor TLB address mapping entries include a Virtual Processor Identifier (VPID), which is a unique identifier that assigns the address translation entry to a specific virtual machine.
In Windows Server 2008 R2 Hyper-V, the support for SLAT-enabled processors provides a means to lower the Windows hypervisor overhead associated with virtual machine address translations. And since virtual machines can be easily moved between SLAT-enabled and non SLAT-enabled processor platforms, it is an easy feature to leverage. As you plan the deployment of Windows Server 2008 R2 Hyper-V, you should strongly consider using a SLAT-enabled AMD-V or Intel VT platform to take advantage of the performance improvements provided for your virtualized workloads.