Creating a Single-Node VSAN

Creating a Single-Node VSAN

Many of us homelab enthusiasts tend to build “whitebox” systems from spare PC parts and a few internal hard drives for local storage that we’ve either ordered or had laying around in order install ESXi and run a single-node lab environment. VMware Virtual SAN (VSAN) enables the ability to build a local SAN environment utilizing the local hard drives in the host. The only downside/caveat is that you need a minimum of (3) ESXi hosts in a cluster to enable and configure VSAN. Bummer!

Well, thanks to some very smart people in the community, there is a way to create a VSAN on your single-node host!

Keep in mind that this is not supported by VMware and is recommended for testing purposes only and should not be done in a production environment so use at your own risk.

I will mention that this topic has been covered by other bloggers (one of my favorites, William Lam, and a few others…) in the community but for my own knowledge and sharing, I decided to write this post detailing how I configured this in my environment.

Below, I will show you how to configure a hybrid and an all-flash VSAN on your ESXi host. Again, I remind you that running a single-node VSAN is not supported by VMware and you run the risk losing all your data in the event of a disaster scenario (system crash, etc.) Please be sure to understand the risks when deciding to use this in your own home lab.

Prerequisites:

(1) vCenter Server (Windows-based or VCSA)
(1)ESXi host in a cluster
(1) vmk for VSAN Traffic
(1) SSD for Caching Tier
(1) SSD for Capacity Tier (All-Flash configuration)…or…
(1) HDD for Capacity Tier (Hybrid Configuration)
SSH access to Host

For simplicity’s sake, I will be using a VCSA and ESXi VM deployed in VMware Workstation 12.1.1 Pro on Windows 10 for this demonstration.

Be sure that you have connected your ESXi host to your vCenter Server and have added it to a cluster, you do not need to enable VSAN on the cluster yet. Add an additional vmk to your vSwitch for VSAN traffic. Also ensure you have started the SSH service on the ESXi host.

Open an SSH session to your ESXi Host. If you’ve added a vmk for VSAN but have not enabled it for VSAN traffic yet, enter the following command.

esxcli vsan network ipv4 add -i vmkN

(Where “N” is the number of your vmk port – ie: vmk1)

In my environment, I already created a VSAN vmk and enabled it for VSAN traffic so I was able to skip the command above.

Using the vSphere Web Client or C# client, verify the hard drive that you want to use for your VSAN datastore. I will be using these drives, the 30GB will be my cache disk and the 120GB will be the capacity disk.

Back in your SSH session, enter the following command to determine and confirm the eligibility of the disks intended for use to create your VSAN.

vdq -q

Next, enter the following command to get the current default VSAN policy.

esxcli vsan policy getdefault

We will need to change the current policy by running the following commands.

esxcli vsan policy setdefault -c cluster -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vdisk -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vmnamespace -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"
esxcli vsan policy setdefault -c vmswap -p "((\"hostFailuresToTolerate\" i0) (\"forceProvisioning\" i1) (\"stripeWidth\" i1))"

Run this command again to confirm that the policy has been changed.

esxcli vsan policy getdefault

Run the following command to create a new VSAN cluster

esxcli vsan cluster new

Now, since my disks are all SSD, I am creating an All-Flash VSAN configuration. I need to run the following command to tag the capacity SSD as the data disk. The “-d” represents the “capacity disk” and you need to specify the identifier of the disk to tag. You can simply copy the identifier number directly from the ESXi hosts storage devices section in Web Client/C# Client, or from the SSH session where we ran the “vdq -q” command.

Note – If you are deploying a Hybrid VSAN, this command is not needed so you can skip to the next command to add the “cache & capacity” disks to your VSAN.

esxcli vsan storage tag add -d <disk identifier> -t capacityFlash

If you’d like to confirm that the disk has been tagged for “capacityFlash” simply run the “vdq -q” command again and check the disk.

Next, run the following command to add both your disks (cache & capacity) to your VSAN storage volume. The “-s” represents the SSD “cache disk”, and the “-d” represents the “capacity disk”. Be sure to enter the correct identifier number for the respective disks.

esxcli vsan storage add -s <disk identifier> -d <disk identifier>

Run the following command to show the VSAN cluster info.

esxcli vsan cluster get

Run the following command to list the VSAN storage

esxcli vsan storage list

Congratulations, if everything has been followed correctly, you should now have created a single-node VSAN datastore!

But we are not quite finished just yet. Even though I can see the VSAN datastore, I still want to officially enable VSAN on the cluster in vCenter. Do the following…

In my environment, I have an extra disk in my host, but I do not want to claim this as part of my VSAN. So, from the drop-down menu, I selected “Do not claim” and hit Next then Finish.

Now I can see that VSAN is “Turned On” and can see the disks that are associated with the Disk Group.

But, there’s still a bit more to be done for me to be able to provision VMs on this datastore. I need to edit the VSAN VM Storage Policy. Personally, I prefer to leave the default policies intact and instead create a new policy for my single-node datastore. I will show both editings of the default policy, for those who do not want to bother with creating a new policy, as well as creating a new policy. First, let’s check the VASA storage provider and ensure it has been synchronized so that we can edit/create our VSAN Storage policy.

Editing Virtual SAN Default Storage Policy

Here we need to simply change:

Number of failures to tolerate = 0 (Default is 1)
Force provisioning = Yes (Default is No)

Creating a new Virtual SAN Storage Policy

Give it a Name and a Description then hit Next.

Select VSAN from the “Rules based on data services” drop-down, then add all the rules from the drop-down and configure the same settings mentioned above, then hit Next and Finish.

Number of failures to tolerate = 0 (Default is 1)
Force provisioning = Yes (Default is No)

And, there you have it! A fully functional Single-Node VSAN to provision VMs on. You still have to add a VSAN license, but that will not be covered here as you should already be familiar with the licensing process.

The Finishing touches

The following optimizations commands are optional but highly recommended for better performance and stability in your VSAN environment.

Since this is a homelab, the disks I used may not be on the official VMware HCL and can potentially impact the performance of the lab environment. Corman Hogan wrote a great blog and included a tip on how to disable VSAN device monitoring. Open an SSH session to your host again and run the following command.

esxcli system settings advanced set -o /LSOM/VSANDeviceMonitoring -i 0

To confirm that the command was successful, run the following command. It should return a value of “0” as the default value is “1”.

esxcfg-advcfg -g /LSOM/VSANDeviceMonitoring

Cormac Hogan also wrote another great post about the new “Sparse VM Swap Object”. ESXi 6.0 Update 2 (aka 6.2) brings a new setting in VSAN 6.2 which allows VSAN to provision a VM swap object as thin instead of thick, where thick has historically been the default. So if you’d like to disable thick provisioning and use thin, run the following command.

esxcli system settings advanced set -o /VSAN/SwapThickProvisionDisabled -i 1

To confirm, run the following command. It should return a value of “1” as the default value is “0”.

esxcfg-advcfg -g /VSAN/SwapThickProvisionDisabled

And last, but definitely not least, if you intend on running any Nested ESXi VMs on your newly created VSAN, be sure to run the following command to prevent any errors when trying to create SCSI disks for your ESXi VM. This will enable an advanced ESXi setting that will “fake” SCSI reservations. William Lam has a nice post about this here.

esxcli system settings advanced set -o /VSAN/FakeSCSIReservations -i 1

And to confirm it took, run the following command. It should return a value of “1” as the default value is “0”.

esxcfg-advcfg -g /VSAN/FakeSCSIReservations

That’s all folks! I hope that you’ve found this post to be of use to you and I hope you come back for more content. Feel free to comment, share, and subscribe!

Giving credit where it is due, shout out to William Lam and Cormac Hogan!

-virtualex-