Maximizing Your Virtual Machine Density in Hyper-V (Part 4)

If you would like to read the other parts in this article series please go to:

One of the themes that I have tried to stress throughout this article series, is the idea that achieving the highest possible virtual machine density means making efficient use of the available physical hardware resources. The idea is to provision virtual machines with sufficient resources to allow them to run properly, but to avoid provisioning virtual machines with unnecessary resources.

It is equally important to keep track of the resources that have been provisioned. Every Hyper-V host server has a finite amount of hardware resources available. These resources must be shared between the parent partition (Windows Server and Hyper-V) and any virtual machines that are running on the host.

Smaller organizations that are running standalone Hyper-V servers, or unmanaged Hyper-V clusters will typically have to track resource allocations manually. The Hyper-V Manager console doesn’t really include a mechanism for keeping track of assigned resources and available resources. If you need these capabilities then the tool of choice is systems Center 2012 Virtual Machine Manager (VMM 2012). VMM 2012 allows you to reserve resources specifically for the host operating system. It also allows you to define pools of resources that can be allocated to virtual machines.

Using System Center 2012 Virtual Machine Manager for resource provisioning is beyond the scope of this article series. Even so, I wanted to mention its abilities because you can use VMM 2012 to make what I am going to show you in the remainder of this article series a little bit easier.

Virtual Processor Assignments

One of the resource types that you will need to control if you want to achieve the highest possible virtual machine density is virtual processors. Before I talk about virtual processor allocation, there are a couple of things that I need to point out.

First, the CPU is rarely the limiting factor for Hyper-V virtual machines. Typically a host server will run low on memory resources or on disk I/O long before it reaches the maximum capabilities of the CPU. Even so, it is still important to use CPU resources efficiently if you truly want to squeeze the highest possible virtual machine density out of your hardware.

The other thing that I need to point out before we get started is that Hyper-V does not allow you to assign CPU cores directly to virtual machines. Instead, virtual machines are provisioned with virtual processors. A virtual processor is not the same thing as a physical CPU or as a physical CPU core.

So if a virtual processor is different from a physical CPU core, then what is a virtual processor? A virtual processor is really nothing more than a virtualized representation of a CPU core. Hyper-V provides CPU time to each virtual CPU, and does so on a round robin basis. In fact, you can almost think of virtual processors as CPUs queues.

Because there is no direct correlation between virtual processors and physical CPU cores, you could theoretically create virtual processors to your heart’s content. However, just because you can do something does not necessarily mean that you should.

In the case of Windows Server 2012 Hyper-V (Hyper-V 3.0), Microsoft does not impose any limit to the ratio of virtual processors to logical processors (Microsoft defines logical processors as CPU cores). However, Hyper-V supports a maximum of 320 logical processors (CPU cores) per host server. Hyper-V also limits you to creating a maximum of 2048 virtual processors on any one host server.

The maximum number of virtual processors that you can assign to any single virtual machine is 64. However, most guest operating systems will not even support what they perceive to be a 64 core server. As such, you have to take the guest operating system’s limitations into account when determining the number of virtual processors to assign to each VM.

Some might be quick to point out that based on today’s hardware prices, only the largest organizations will have to worry about hitting any of these limits (aside from the guest operating system limits). Even so, it is still important to be careful about creating an excessive number of virtual processors.

As previously mentioned, Microsoft does not impose a hard limit on the ratio of logical processors to virtual processors. You are free to create as many virtual processors for each logical processor as you like, so long as you don’t get any of the limits mentioned earlier. The problem with doing so however, is that virtual server performance can be diminished as the ratio of virtual processors to logical processors increases.

The best of my knowledge, Microsoft does not have a current recommendation for the maximum virtual processor to logical processor ratio. However, there is an MSDN article that clearly states that you should use a one-to-one ratio in order to achieve the best performance. A one-to-one ratio would mean that you only create a virtual processor if you have a corresponding CPU core available.

Obviously using this technique would significantly limit the number of virtual processors that you can create, which would also limit the maximum number of virtual machines that you can create since every virtual machine has to have at least one virtual processor. The maximum number of virtual machines that a server can accommodate when using this plan is further diminished when you consider that Microsoft recommends reserving two physical CPU cores for the host operating system.

Although a one-to-one ratio of virtual processors to logical processors is good for performance, it doesn’t work so well for achieving the highest possible virtual machine density. To give you a more concrete example, the Hyper-V servers in my lab all have eight logical processors. If I were to insist on a one-to-one ratio and I were to reserve the recommended two physical CPU cores for the host operating system, it would mean that the very highest number of virtual machines that I could hope to host on the server’s would be six. Needless to say, it is possible to achieve a much higher virtual machine density on a server with a logical CPUs.

As I said earlier, I haven’t found any official Microsoft best practices for the maximum ratio of virtual processors to logical processors. There are some really old documents related to the first release of Hyper-V that advised maximum ratios of 1.5 to 1 or 2 to 1, but those articles are very outdated.

My experience has been that in most cases you can achieve a ratio of about 6 to 1 (and in some cases even 8 to 1) before performance begins to suffer.

One last bit of advice that I want to give you is that I recommend never assigning a virtual machine a greater number of virtual processors than you have logical processors. Suppose for instance that you had a server with eight logical processors. It’s perfectly okay in that type of situation to create multiple virtual machines with eight virtual processors each, so long as you do not deplete the system of CPU resources (I’m also ignoring host reservations for the sake of simplicity). However, on such a system you would not want to allocate more than eight logical processors to any one virtual machine because doing so would actually diminish the virtual machine’s performance due to the way that the Hyper-V virtual processor scheduling process works.

Conclusion

As you can see, there is a lot to consider when it comes to virtual processor allocation. In the next article in this series, I want to turn my attention to memory allocation for virtual machines.

If you would like to read the other parts in this article series please go to: