If you would like to read the other parts in this article series please go to:
- Taking a Fresh Look at Hyper-V Clusters (Part 1)
- Taking a Fresh Look at Hyper-V Clusters (Part 2)
- Taking a Fresh Look at Hyper-V Clusters (Part 3)
- Taking a Fresh Look at Hyper-V Clusters (Part 4)
- Taking a Fresh Look at Hyper-V Clusters (Part 5)
- Taking a Fresh Look at Hyper-V Clusters (Part 7)
- Taking a Fresh Look at Hyper-V Clusters (Part 8)
Introduction
So far in this article series, we have built a failover cluster and we have made some virtual machines fault tolerant. In this article, I want to continue the discussion by talking about some of the more practical aspects of managing highly available virtual machines.
Virtual Machine Prioritization
As you no doubt know, the entire reason why we went through all of the work involved in building a cluster was to protect our virtual machines against a host server level failure. In a non-clustered environment, the failure of a host server would mean the failure of all of the virtual machines running on that host. Needless to say, the failure of a single non-clustered host could translate into a major outage (depending on which virtual machines were running on the host).
Clustering is intended to prevent this situation from ever happening. The idea is that if a clustered Hyper-V server were to fail then all of the virtual machines that were running on that cluster node would fail over to another node that is still functional. That way, the virtual machines can keep running in spite of the failure.
While this idea sounds good in theory, you must also consider the logistics of a failover. Think about the reason why server virtualization became so popular in the first place. Server hardware manufacturers were creating hardware that was increasingly powerful. In fact, most workloads didn’t come anywhere close to utilizing a physical server’s full potential. As such, much of the available CPU capacity was wasted.
That being the case, server virtualization spawned from the relatively simple idea that physical hardware capacity could be shared among multiple workloads (virtual machines). The idea was initially pitched as a way of reducing hardware costs by more fully using the organization’s existing hardware resources. It was a good idea (obviously), but like so many other good ideas there were those who took things a little bit too far.
Almost overnight it seemed that the emphasis had gone from more fully utilizing server hardware to cramming as many virtual machines as possible onto a virtualization host by taking advantage of hardware over commitment and a number of other techniques. Although such methods of squeezing every last available resource from the physical hardware do work, they must be carefully controlled. Otherwise, the miserly use of hardware resources can lead to a number of different problems centering around performance and stability.
So what does all of this have to do with failover clustering or Hyper-V? Well, on the surface the last few paragraphs probably seem like little more than a history lesson or perhaps a cautionary tale. As always however, there is a method to my madness.
The point that all of this is that server virtualization absolutely demands that administrators maintain careful control of physical hardware resource consumption. However, failover clustering raises the stakes.
Those organizations that are using standalone Hyper-V servers must address physical resource control on a per host server basis. In other words, there are a specific number of virtual machines that are running on each host server and the administrator must make sure that each virtual machine receives the resources that it needs without depleting the server’s overall pool of resources. Otherwise, there might not be enough resources left to service certain virtual machines or the host operating system (in this case Windows Server and Hyper-V).
Although this same basic concept also applies to clustered Hyper-V deployments, the administrator must also take into account cluster level resource consumption, not just host level resource consumption. Let me show you what I mean.
Imagine for a moment that a particular organization has three identical Hyper-V servers that are each functioning as nodes within a failover cluster. Let’s also pretend that all of the virtual machines that are hosted on the cluster nodes have been made highly available and that each of the virtual machines is running 24 / 7. Now, just to make things interesting let’s pretend that the virtual machines running on each Hyper-V host server collectively consume the maximum amount of physical hardware resources that can be comfortably allocated without causing any performance, stability, or reliability problems in the process. In other words, our imaginary clustered Hyper-V cluster is running flawlessly, but there isn’t enough room to add any more virtual machines.
Now that I have set up an imaginary premise, I want to go back to something that I talked about earlier. The entire purpose of a Hyper-V cluster is to keep virtual machines running even when a cluster node fails. So with that in mind, imagine what would happen if a node in our imaginary cluster were to drop offline.
In this situation, an outage would occur. None of the cluster nodes have enough resources available to accommodate any more virtual machines. As such, the virtual machines that were running on the failed cluster node cannot fail over to another node, because none of the remaining nodes have sufficient resources to accommodate them.
So what is the solution to this problem? Add another node to the cluster and leave it empty? Hardly. Although that solution would work, it is a tremendous waste of resources.
A better solution would be to add an extra node to the cluster and then move some of the virtual machines to that node. That way, none of the nodes are completely empty, but there aren’t any nodes that are completely full either. But what can you do if another node isn’t in the budget?
In that type of situation, your best bet may be to implement virtual machine prioritization. The idea behind virtual machine prioritization is that in any organization, some virtual machines are more important than others. Virtual machine prioritization can help you to make sure that mission critical virtual machines continue to run during a failover situation because you have told Hyper-V that those virtual machines take priority over less important virtual machines such as redundant domain controllers.
Setting up virtual machine prioritization is really easy. To prioritize a virtual machine, open the Failover Cluster Manager and right click on the virtual machine that you want to prioritize. When the shortcut menu appears, select the Change Startup Priority command from the shortcut menu and then choose the virtual machine’s priority, as shown in Figure A.
Figure A: Open the Failover Cluster Manager, right click on a virtual machine, and choose the Change Startup Priority command from the shortcut menu.
Keep in mind that changing a virtual machine’s priority will not guarantee that the virtual machine will run. It simply controls startup order, which is useful in failover situations. High priority virtual machines start before low priority virtual machines.
Conclusion
In the next article in this series, I want to continue the discussion of virtual machine prioritization. In doing so, I want to talk just a little bit more about how prioritization works and I want to show you some useful PowerShell commands. My plan is to then wrap up the series by talking about manually moving virtual machines and related resources within the cluster.
If you would like to read the other parts in this article series please go to:
- Taking a Fresh Look at Hyper-V Clusters (Part 1)
- Taking a Fresh Look at Hyper-V Clusters (Part 2)
- Taking a Fresh Look at Hyper-V Clusters (Part 3)
- Taking a Fresh Look at Hyper-V Clusters (Part 4)
- Taking a Fresh Look at Hyper-V Clusters (Part 5)
- Taking a Fresh Look at Hyper-V Clusters (Part 7)
- Taking a Fresh Look at Hyper-V Clusters (Part 8)
