Get your geek on: Building a VMware home lab

Woodworkers have their workshops and potters have their kilns. What do IT professionals have in their homes? Labs set up so they can learn new hardware/software and test various scenarios. Yes, I confess I’m one of those geeks who over the years has collected a vast assortment of hardware for learning and testing purposes. And I’m sure that many of you who are reading this article are equally guilty of being hardware scavengers. After all, ours is a profession where we need to constantly learn new things and be exposed to new technologies, and since the best way to learn is by doing things hands-on, setting up a lab at home for trying various things out is clearly going to put your feet on the road to success in your profession.

IT pros who pursue technical certifications also need to practice, practice, practice, and having a lab set up at home for doing this is well worth the investment if the certification you are pursuing is likely to advance your career in some fashion. VMware, of course, is still a very popular technology at the enterprise level, so if you’re going after a VMware certification you need a good lab for this too. But since I’m not a VMware expert myself, I thought I’d ask an actual expert in VMware technologies if he could describe how he set up his own home lab for simulating real-world VMware environments. David Barker is a Senior Systems Engineer with over eight years of experience with Virtualization. He is a current VCP6-DCV and has been awarded the title of vExpert by VMware for the last three years. He is an active blogger and is on Twitter @davidbarker223. Let’s now put on our thinking caps and listen carefully so we can learn from David how to set up a fully-functioning VMware home lab using components you can buy easily from Amazon.

VMware home lab: 3 node vSAN Cluster running on NUC

I’ve been wanting to build a VMware home lab for a while at home now to test out new builds and versions of software not only from VMware, but Microsoft and others as well. Plus, being a vExpert and having access to production versions of software licenses from VMware was a huge bonus. The major component I wanted to play with and get more familiar with was vSAN. I started my lab with a single NUC and was able to get vSAN installed, but it was not happy with the single node. I thought about doing the nested ESXi route, but with the limited resources already on the NUC I just decided to do a full 3-node cluster.

Let’s start off with a list of my bill of materials (multiply everything by three and I’ll provide Amazon links for everything):

Here’s a photo of everything needed for building one node:

VMware home lab

I chose the NUC6I3SYH as I didn’t have a need for a faster processor and the dual-core hyperthreading gave me four logical processors, which is what I was looking for anyway. The SYH allowed me to install a full SSD for the Capacity Tier as well as a NVMe SSD for the Caching Tier. I installed the max 32GB of RAM and chose the SanDisk Cruzer Fit for its small form factor since I installed ESXi to it and needed it to stay in the NUC.

Installing ESXi

Using a program like Rufus or uNetBootin, you can create a bootable USB with the .iso file for ESXi. I downloaded the latest build of 6.5 from VMware to use for my lab. You will also need to download a version of vCenter — I suggest 6.5 as well. Obviously, you’ll want your vCenter and ESXi versions to match. One option, especially for a lab environment is to download the 6.0 builds and use your lab to test the upgrade to 6.5 if you’re not comfortable with that path. Another great use for a lab!

Installing vCenter

The new feature with vCenter 6.5 is the ability to bootstrap the vSAN configuration during the install. In previous versions of vCenter, you had to do a bunch of manual work, which William describes here in his blog if you choose to not go with 6.5. But with vCenter 6.5, you can choose to configure vSAN (since you won’t have any datastores set up yet) in the image below:

VMware home lab

Once vCenter is installed, you’re all set. Make sure to create your disk groups and add your disks to vSAN:

VMware home lab

Since the NUC is not listed on the VMware HCL, you will see some warning about hardware compatibility, which is normal. But hey, it’s a lab!

VMware home lab

And that’s it! Congratulations, you now have a 3-node vSAN lab up and running. Start deploying machines and testing things out. And if you break it, you just get more practice deploying ESXi and vCenter.

I’ve got my VMware home lab, now what?

There are some pretty great things you can do now, with your own set of resources at your disposal. A lab can be used for whatever you need it to be in that moment. Maybe you want to study for the VCP6-DCV exam or you want to study for a Cisco/networking exam. Well, you can go ahead and build VMs for whatever you need or want. I personally have built a few 2016 servers to check out the new OS as well as the new Server core builds. This allows me to learn the new commands and practice PowerShell when at home.

I have also used my lab to test scenarios for my work. And maybe you want to learn a new OS? Say you’re a Windows admin and want to learn some Linux or vice versa. This is a great way to be able to look at things on your spare time and at your own pace. Plus, if you break something in your lab, you don’t take down your production at work. You can just delete and re-create.

Finally, just have fun with your VMware home lab. It can be frustrating at times, but that just means you’re learning. And if we’re not learning something new in this industry, we are already behind.

Photo credit: Wikimedia

About The Author

17 thoughts on “Get your geek on: Building a VMware home lab”

  1. Hi Fred, I get my licensing through the VMware vExpert Program. However, most VMware products offer a trial license you can use, which is fine for a typical lab deployment where the box will likely just be built and destroyed shortly after anyway.

  2. I am a little confused about the setup. Did you use one NUC to install 3-node EXSI on it or used 3 NUC for 3 EXSI? I would appreciate further clarification. Thanks.

  3. Ahhhh, that answers my question as well. The shopping list should clarify that it’s three of everything shown. That triples the budget!!!!!

    The article also suggests one device that will run the entire lab. What are the other two NUCs doing? Is there set up different?

    Thanks,
    msbob

  4. do you think I would be able to have a functional lab with just 2 hosts? The main reason I ask would be to bring down the cost. What would the cons be? Also, do you know of VMWare has a link with a step-by-step tutorial on how to set up a home lab? Thank you.

    1. You can setup a 2 host VSAN, here is a link from vmware on how to do so:

      VMware vSAN™ TwoNode Architecture VMware Cloud Provider™ Use Cases
      https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/vcat/vmware-virtual-san-two-node-architecture-service-provider-use-cases.pdf

      The document has more than you are asking, but, it provides the information you need on how you’re going to get the 3rd host, without a physical machine. You will be, “employing a dedicated virtual appliance to provide Witness services” (the third host).

      Cons would be; 1) virtual appliance will take up some resources from one of your physical hosts (limiting other vm’s you can run on that host, 2) possibly some performance hits for the VSAN. I am sure there are others, the VMWARE document should provide additional information.

      I have never seen a VMWare document for setting up a home lab, although one may exist. Your best bet is a simple Google search for vmware home lab, that will provide you with abundant resources for this. Maybe limit the search for the last year, or custom search for last 2 years, you get the idea. Also try searching for, vmware home lab David Barker, to see if David has detailed his home lab.

      Good luck!

  5. Hi,

    Did you see any issues with how much memory ESXi reserved for the VMKernel in your 6.5 configuration?

    I have a NUC8i5BEH cluster running ESXi 6.7. Each host ends up having ~50%(16GB of memory) reserved for the VMKernel which only leaves 16GB of VM’s.

    I have seen numerous posts on this topic where the host System Resource Reservation has changed from 5.x, 6.0, 6.5 and 6.7. It used to be tunable via Vcenter in past code versions but now apparently is not.

    Anyway, I went down the NUC route for a home lab because I thought it would be a semi cheap route, but if 50% of the memory is lost to VMKernel on small memory hosts then NUC might not be the best route.

    I might want a host that allows a large memory capability if 16GB gets reserved by VMKernel right away.

    Hopefully, I can find an answer from VMWare on this and can tune it somehow as VMKernel using the 50% of the 32GB memory capacity on a NUC makes the device a poor choice IMO.

    Thanks…

  6. vSan has some pretty hefty memory requirements that you should be aware of before purchasing a server for a home lab environment. I found it out the hard way.

    See: https://kb.vmware.com/s/article/2113954

    vSan also has some other host requirements:

    See: https://kb.vmware.com/s/article/2106708

    The other interesting thing about the configuration in this article is that it doesn’t really address how to handle various ESXi logs that really shouldn’t be left on the USB flash drive that the configuration boots from. Those logs living on your USB flash drive used for the ESXi boot drive can lead it to fail sooner

    So the NUC is a nice device for an ESXi host in a home lab, but if you plan to use vSan it will be limited by it’s maximum memory capacity of 32GB.

    I have about 12 VM’s running across the 3 host cluster and am at 90-100% memory consumption as ~52GB of my 96GB of memory is tied up by the VMKernel.

    vSan requires the cache drive to be 10% of the host capacity. Ie. a 1TB capacity drive on a host should have a 128GB cache drive. A 2TB capacity drive on a host should have a 250GB cache drive.

    The problem with that is if you look at the vSan memory article you can quickly see how vSan chews up your host memory on a small server. ~7GB allocated out of the gate by VMkernel for vSan. Then a portion is also reserved based on the size of the “cache” drive. So you would want to use the smallest cache drive possible to achieve your 10% vSan requirement if you want to keep your vSan memory footprint down on the host that is limited to 32GB of memory.

    There are other things in the NUC configuration for vSan related to networking as well as vSan really requires/supports the host to have 10Gbit network connectivity which the NUC doesn’t have in this configuration. In my brief lab testing so far vSan runs on the 1Gbit network okay if your not overly concerned with doing performance tests and such.

    So the NUC with 32GB memory capacity is a cheap way to get a ESXi environment stood up, but IMO not a great device for a HCI using vSan.

  7. Confused as to the sequence. When is the ESXI installed for example, do I do host network so I can test various server configuration and how can I “attach” a separate server into the VMWare Workstation setup if it is not on the normal network but will be needed to be on the Workstation network.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top