vSAN Stretched Clusters & VM Swap Files
- October 13, 2017
My buddy Pete Koehler put together a good post a few days ago about SPBM Policies with vSAN Stretched Clusters: vSAN Operations: Use separate SPBM policies for VMs in stretched clusters
In that post, he covers definition changes from Number of Failures to Tolerate (FTT) which was available in pre vSAN 6.6 builds, to Primary Failures to Tolerate (PFTT) in vSAN 6.6 and higher. This change was added to address the addition of local protection within a Stretched Cluster. This is accomplished using the secondary rule, Secondary Failures to Tolerate (SFTT), to determine the local protection within a Stretched Cluster Fault Domain (Preferred or Secondary). Pete covers this in the above post.
A question came up last week specific to the behavior of a VM’s swapfile in vSAN when using vSAN Stretched Clusters. There has been some confusion with my documentation on StorageHub, which I’m currently updating, but I wanted to post something specific to how a VM’s swapfile behaves.
I’ve setup a Stretched Cluster with a single VM on it for a little clarity. Once site is “Denver” and one is “Colorado Springs” in my demo environment. Since I’ve moved to CO, I figured I’d use some naming that has local flair.
Pre vSAN 6.6 Stretched Cluster Behavior (vSAN 6.1, 6.2, & 6.5)
In vSAN Stretched Clusters before 6.6, each object had 1 component in each Fault Domain (Preferred/Secondary) and a Witness component on the vSAN Witness Host. *If an object is greater than 255GB in size, it will be broken into multiple components (on each site) for every increment of 255GB.
Here’s a standard policy for Pre-vSAN 6.6 Stretched Clusters:
PFTT=1 (Protecting across sites)
Failure Tolerance Method = Mirroring (one replica component in each site)
Force Provisioning = No (requires both sites to be up) or Yes (can still provision with one site offline)
Looking at the vmdk, the same can be seen (the vmdk is <255GB)
Looking at the Swap file, it is also the same. One replica in each site, and a Witness component on the Witness host.
Notice though that the VM SWAP object does not have a policy associated with it.Hard coded policy for VM SWAP object
I’ve covered this before specific to the fact that VM SWAP objects are configured with an Object Space Reservation of 100%. We introduced the Sparse Swap setting in 6.2 back in March of 2016.
I wrote a blog post with some PowerCLI code showing how to easily use this new feature here: https://blogs.vmware.com/virtualblocks/2016/02/24/vsan62-powercli-sparse-vswp/
Going a bit deeper than the Object Space Reservation rule, VM SWAP objects are always mirrored (FTM=Mirroring) and adhere to traditional Fault Domain placement rules.
Put simply: Regarding vSAN Stretched Clusters, VM SWAP objects will always have one replica in each site, and a Witness component residing on the vSAN Witness Host.
vSAN 6.6 Stretched Cluster Behavior Using Local Protection
When using a SPBM Policy that supports traditional protection (across sites) as well as Local Protection , the VM objects (VM Home/vmdks/etc) will be mirrored across sites, but will be either mirrored or erasure coded (depending on policy) within the Preferred and Secondary site. Whatever SFTT+FTM is set to, both sites must be capable of satisto be in compliance.
There is a Witness component on each site (part of the Secondary Failures To Tolerate rule) and one on the Witness Host.
vSAN 6.6 Stretched Cluster Behavior Using Site Affinity
When using a SPBM Policy that provides local protection only, the VM objects (VM Home/vmdks/etc) will be either mirrored or erasure coded (depending on policy) within the Preferred or Secondary site. This is an or situation, because the Affinity rule will determine whether the VM’s data resides in one site or the other.
But what about the VM SWAP Object when using vSAN 6.6 with either Local Protection or Site Affinity?
Good question. Remember that the VM SWAP Object doesn’t have a policy applied. It is hard coded. Looking at the VM with the Site Affinity policy applied, what is expected from the VM SWAP Object?
That’s right, the VM SWAP Object still lives on both the Preferred and Secondary sites.
Regardless of vSAN version (6.1-6.6), VM SWAP Objects will still be written across sites when using Stretched Clusters. This is because the VM SWAP Object has a hardcoded policy.
**Note, I’ve got a code snippet that will set SwapThickProvisionDisabled for vSAN Clusters located here: