One of the techniques that virtualization administrators have long used to bring down the cost of running a virtualized workload is to do everything possible to maximize a host’s virtual machine density (or potential density). This means creating and configuring virtual machines in a way that allows the host to simultaneously run as many virtual machines as possible. One of the problems with trying to achieve the highest possible virtual machine density, however, is that you can never allocate all of the server’s hardware resources to running virtualized workloads. The hypervisor is always going to have to claim some hardware resources for itself. Without these resources, the hypervisor cannot run, and therefore cannot host virtual machines. This is true of Hyper-V, ESXi, and every other Type 1 hypervisor. Being that some administrators will inevitably try to squeeze every last byte out of virtualization hosts, Microsoft long ago implemented a root memory reserve for Hyper-V. The root memory reserve sets aside a small amount of the server’s physical memory for use by Hyper-V. This ensures that the server’s memory will never be depleted to the point that Hyper-V itself ceases to function.
Root memory reserve: Things not always as they seem
Although the basic concept of a root memory reserve seems undeniably important, things are not always what they seem. There are situations in which you might be doing yourself a disservice by using the root memory reserve. Specifically, the root memory reserve might reduce a host’s potential VM density.
Hyper-V is no longer a one-size-fits all solution. Back in the day, Hyper-V came in two flavors. There was a standalone version of Hyper-V Server, and Hyper-V could also be deployed as a server role within Windows Server. In either case, Hyper-V was run and managed as a server service. Today, however, we also have client Hyper-V. Client Hyper-V runs on a desktop operating system such as Windows 10 and is almost never used to host production workloads. The point is that client Hyper-V is used differently from Hyper-V Server, so it does not necessarily make sense to manage the two platforms’ memory in the same way. Thankfully, Microsoft has modified Windows 10 so that client Hyper-V does not use a root memory reserve by default.
So why wouldn’t you want to use a root memory reserve on Windows 10? I mean sure, Windows 10 is different from Windows Server, but doesn’t the root memory reserve still have its place on Windows 10? Sometimes it does, and you can manually set a root memory reserve by creating a registry key. However, the root memory reserve is known to sometimes cause problems related to starting virtual machines.
The problem is that sometimes a Hyper-V host may not have enough memory available to start a VM, even though it would seem that there is plenty of memory available. In fact, I encountered this problem myself a few months ago when attempting to run a large number of VMs on a Hyper-V host. Some of the VMs refused to start due to insufficient memory, even though the Task Manager told me that there was more than enough memory available.
Suppose for a moment that a Hyper-V host has 16GB of RAM. If we assume that the parent operating system needs 2GB of RAM to function, then that theoretically leaves 14GB for hosting Hyper-V virtual machines. In actuality, however, the system reserve could be consuming up to about another 2.5GB of RAM, meaning that the amount of memory available for hosting VMs could be as little as 11.5GB.
The reason why this is a problem is because the host reserve adheres to a progressive scale, in which the size of the host reserve increases as the amount of physical RAM increases. Although Microsoft provides guidelines as to how big the host reserve could be, it is difficult to predict how much memory will actually be used by the host reserve. Furthermore, until Microsoft made some recent updates to Windows 10, the Task Manager totally ignored the host reserve.
Check your memory
If you want to know how much memory is actually available to Hyper-V hosts, open the Performance Monitor and load the System Balancer instance of the Hyper-V Dynamic Memory Balancer \ Available Memory counter, as shown below.
If you want to see why this counter is so important, check out the figure below, taken from a Windows Server 2012 R2 Hyper-V host. In this figure, you can see that the Performance Monitor reports that roughly about 16GB of memory is available to Hyper-V virtual machines. However, the Task Manager shows 18.3GB of memory available. There is about a 2GB difference in this case between the amount of memory that Task Manager says is available, and the amount of memory that Hyper-V can actually use.
Earlier, of course, I explained that in some cases, the root memory reserve is no longer required. As a general rule, Hyper-V servers still need a root memory reserve, but client Hyper-V does not. Hyper-V servers may need to be configured to provide high availability to virtual machines or might need to adhere to an SLA, and so a root memory reserve can be beneficial. Because Windows 10’s client Hyper-V does not generally run production workloads, Microsoft has made it so that Windows 10 no longer requires a root memory reserve.
It is worth noting that if you have manually configured a root memory reserve, Windows 10 will continue to use it. However, a root memory reserve is not required. To see the difference, take a look at the screen capture below. This screen capture was taken from a Windows 10 machine running the 2017 Fall Creators Update. This time, the Task Manager reports 9.4GB of memory available. Performance Monitor shows roughly about 9GB available for hosting Hyper-V virtual machines. Although there is still a bit of a discrepancy, the two tools show a much closer estimate of the available memory than what was shown in the previous screen capture.
If you are still unclear as to why Microsoft did away with a default root memory reserve in Hyper-V, Sarah Cooley of Microsoft explains the changes like this:
In Windows 10, you’re probably running several applications (web browsers, text editors, chat clients, etc) and most of them will reserve more memory than they’re actively using. With these changes, Hyper-V starts allocating memory in small chunks (to give the operating system a chance to trim memory from other applications) and will use all available memory (no root reserve). Which isn’t to say you’ll never run out of memory but now the amount of memory shown in task manager accurately reflects the amount available for starting virtual machines.
Photo credit: Wikimedia