vSAN API Cookbook for Python

First Published On: 01-31-2017

Last Updated On: 02-01-2019

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Table Of Contents

1. Introduction
1.1.Expectations
1.2.vSAN Management API
1.3.vSAN SDKs
2. vSAN Recipes
2.1.Deploying vSAN
2.2.Configuring vSAN Stretched Clusters or 2 Node
2.3.Performing a vSAN On-Disk Upgrade
3. References
3.1.Additional Documentation
3.2.VMware Contact Information
3.3.About the Author

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

1. Introduction
Typically, vSAN management is performed through the Web Client. Tasks can include those such as
initial configuration, ongoing maintenance, and reporting of capacity, performance, or heath of vSAN.

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

1.1 Expectations

While most element management is easily accomplished with the Web Client user interface,
performing many repeatable tasks are a manual process. Some aspects of vSAN management are
automated, such as disk claiming, periodic health checks, as well as error and other reporting. These
automated tasks are specific to each individual vSAN cluster, and often have to be repeated many
times when managing multiple independent vSAN clusters.

Consistency and repeatability, is a challenge when performing tasks manually. It is quite common to
leverage tools such as an Application Programming Interface (API) along with code to execute tasks in
a consistent and repeatable fashion.

Expectations
This document is intended to assist you with understanding types of things that can be managed
programmatically through the vSAN Management API and SDK.

It is neither comprehensive in showing all possible actions nor prescriptive in showing the only way to
accomplish these tasks. This document will focus on the use of Python, but be aware other languages
may be chosen, which will not be covered in this document.

Throughout the document we will alternate showing what types of tasks can be done through the
standard web client user interface, and then how to achieve the same result through Python. None of
the included code samples are supported by VMware, and are merely representative of possible ways
to accomplish tasks.

1.2 vSAN Management API

vSAN 6.2 introduced a new vSAN Management API that which extends upon the current vSphere API.
This API is exposed by both vCenter Server managing vSAN, as well as VMware ESXi hosts. Setup and
all configuration of aspects of vSAN, as well as runtime state, is available by leveraging the vSAN
Management API.

There are a variety of vSphere Managed Objects exposed by the vSAN Management API that provide
functionality specific to vCenter Server, ESXi, or both. These Managed Objects are:

Managed Object Function Available

VSAN Cluster configuration and

VsanVcDiskManagementSystem vCenter
query APIs for disks

VSAN Stretched Cluster related

VsanVcStretchedClusterSystem vCenter
configuration and query APIs

VSAN Cluster configuration

VsanVcClusterConfigSystem vCenter
setting and query APIs

VSAN Cluster health related

VsanVcClusterHealthSystem vCenter
configuration and query APIs

VSAN Cluster space usage

VsanSpaceReportSystem vCenter
related query APIs

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Managed Object Function Available

VSAN Cluster performance

vCenter
VsanPerformanceManager related configuration and query
& ESXi
APIs

VSAN Cluster setting APIs for

vCenter
VsanObjectSystem object status query and storage
& ESXi
policy

Host level VSAN related

HostVsanSystem ESXi
configuration and query APIs

Host level VSAN Health related

HostVsanHealthSystem ESXi
configuration and query APIs

VsanUpgradeSystem Used to perform and monitor vCenter

VSAN on-disk format upgrades.
VsanUpgradeSystemEx
VsanUpgradeSystemEx deprecates VsanUpgradeSystem vCenter

Table 1 – Managed Objects presented by the vSAN API

1.3 vSAN SDKs

Software Development Kits (SDK) have also been provided with the release of vSAN 6.2 for several
popular programming languages. By providing SDKs in multiple languages, VMware has made it easier
to use the vSAN API. Some of the languages that have SDKs for the vSAN API include Python, Ruby,
Java, C#, and Perl. They each include respective language bindings to the vSphere Management SDK,
as well as documentation that details the usage of each API.

These are all extensions of the vSphere API for their respective programming language. While these
are languages that are supported today, additional languages may be supported in the future.

Getting Started with the vSAN Management SDK

To get started, the vSAN Management SDK, one of the previously mentioned packages needs to be
downloaded from the VMware Developer site. The starting page for SDKs can be found here:

http://code.vmware.com/sdks

This URL is a common launching point for SDKs for different VMware products including vSAN. The
vSAN Management SDKs can be found in the Storage section. Each of the different vSAN
Management SDK links leads to dedicated content for that SDK including documentation and
reference material and a download link.

While the vSAN Management SDK content is publicly available, a My VMware account is required.
Anyone can register for a My VMware account for free. Existing customers may already have accounts
if they have performed tasks such as license management, downloaded software, or worked with
VMware support.

From the download page, download the appropriate vSAN Management SDK for the language you
wish to use.

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

The vSAN Management SDKs are made available as a compressed .zip file. To use the SDK, the
contents of the .zip file will need to be extracted. While some Operating Systems provide native
support for extracting contents from .zip files, others do not.

Using the vSAN Management SDK for Python

To be able to use the vSAN Management SDK for Python, it is important to have Python installed on
the system that will be used to execute scripts that use the SDK.

Python can be easily installed on Windows, Linux, and Mac operating systems. This provides flexibility
for developers to use the platform of their choice, while providing portability of scripts across these
very different operating systems.

For the purpose of this document, Python for Windows 10 will be used. Python is available for free
from Python.org.

Python 2.7.12 was picked for the purpose of this paper. Python defaults to installing to C:\Python27.
After Python has been downloaded and installed, it is important to add pyVmomi. PyVmomi is the
Python SDK for the VMware vSphere API, allowing developers to access ESXi and vCenter using
Python. The vSAN Management API is built as an extension of the vSphere API therefore, pyVmomi is
also used to allow Python access to the vSAN Management API.

An easy way to add pyVmomi to the installation is through the use of pip. Pip is the Python Package
Manager. To install pyVmomi, simply type “pip install pyvmomi”

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Once pyVmomi has been installed, the bindings included in the vSAN Management SDK for Python
can be extracted from the vsan-sdk-python.zip file.

Copy vsanmgmtObjects.py from the bindings folder to Python’s Scripts folder (C:\Python27\Scripts).

Sample code scripts could also be copied to Python’s Scripts folder (C:\Python27\Scripts).

The resulting C:\Python27\Scripts folder will look something like this:

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Once the bindings and sample code have been copied to the Scripts folder, additional scripts can be
easily run from this location.

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

2. vSAN Recipes
Sample ‘Recipes’ are provided, to detail the process of how one would go about putting vSAN
Management API for Python scripts together.

These samples are not supported by VMware.

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

2.1 Deploying vSAN

Deploying vSAN can be accomplished easily using the Web Client using the Enable vSAN wizard. The
wizard prompts the administrator for some items like whether disks will be claimed manually or
automatically, if Deduplication and Compression are to be enabled, which disks will be used, and more.

Specific actions are required to be performed across hosts in the cluster, in a particular order. For
example, disks cannot be added to the vSAN datastore until vSAN has been enabled, and hosts have a
dedicated network to communicate between each other. Once hosts have been added to a vSphere
cluster, and settings like vSphere HA and DRS are configured, a vSAN cluster can be configured.

These tasks are easy individually, but when deploying many vSAN clusters very rapidly, the consistency
and repeatability of these tasks rely on the administrator. Manually executing these tasks is time
consuming, and is potentially subject to variance if performed inconsistently.

1. Selecting a Cluster for vSAN

vSAN is configured per cluster. Enabling vSAN using the vSphere Web Client is a very simple process.
In the Hosts and Clusters view, simply click the cluster that will have vSAN enabled, and go to Settings
under the Manage tab.

Because vCenter Server does not have a global option to enable vSAN on specific clusters, or all
clusters, the process is manual. Each cluster must be individually selected, and the vSAN Configuration
Wizard must be run.

When using the vSphere Web Client authentication is taken care of automatically, and provides a
secure session for performing tasks. When using code, the mechanism of connecting to a vCenter
Server, passing authentication, and establishing a session must be created.

if sys.version_info[:3] > (2, 7, 8):

context = ssl.create_default_context()
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE

# Connect to vCenter Server

si = SmartConnect(host=args.host, user=args.user, pwd=password, port=int(args.port),
sslContext=context)

# Disconnect from vCenter upon exit

atexit.register(Disconnect, si)

# Connect to a cluster that is passed as an argument

cluster = getClusterInstance(args.clusterName, si)

10

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

In the above code, a function, called getClusterInstance is called with the Cluster Name, which is
passed as a script argument, as well as the mechanism to securely connect to vCenter Server with
credentials, which were also passed as arguments.

Functions are snippets of code that are often valuable because they operate as smaller programs
within a larger program performing a specific action. Functions such as getClusterInstance below
could likely be reused across many scripts to connect to a specific cluster.

def getClusterInstance(clusterName, serviceInstance):

content = serviceInstance.RetrieveContent()
searchIndex = content.searchIndex
datacenters = content.rootFolder.childEntity

# Look for the cluster in each datacenter attached to vCenter

for datacenter in datacenters:
cluster = searchIndex.FindChild(datacenter.hostFolder, clusterName)
if cluster is not None:
return cluster return None

2. Using the vSAN Configuration Wizard

The vSAN Configuration Wizard is the starting point for enabling vSAN on a vSphere cluster. A lot of
work has been done by VMware Engineering to make the task of deploying vSAN very easy. The
wizard exposes the steps of choosing vSAN capabilities, ensuring that VMkernel adapters are
configured, the ability to choose storage devices, and then complete the setup.

Depending on which capabilities are selected, additional tasks can be performed using the wizard, like
claiming disks automatically, enabling Deduplication and Compression and configuring Fault Domains,
Stretched Clusters, or 2 Node configurations.

11

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

The vSAN Configuration Wizard is a launching point for configuring vSAN. For the purpose of this
document, we will break down the individual sections as they pertain to different functions.

3. Automatically Claiming Disks

vSAN can be configured to claim disks either automatically or manually. The default method is to
manually claim disks. This setting is valid both for the initial configuration, as well as normal operation
of vSAN. It is important to understand the behavior of this setting when deploying vSAN.

When disk claiming is set to automatic, disks will be claimed on the best effort of the vSAN Wizard to
choose the appropriate types for the Cache tier and the Capacity tier. This is easy for Hybrid
architectures, but could be problematic depending on capacity sizes in an All-Flash configuration if
disks that are desired to be used for the Cache tier are similar in capacity to those that are desired to
be used in the Capacity tier. In some cases it may be preferred to claim disks manually.

Here is sample code to Manually claim disks.

print 'Enable VSAN by claiming disks manually'

#Build vsanReconfigSpec step by step, it only takes effect after method VsanClusterReconfig is
called
vsanReconfigSpec = vim.VimVsanReconfigSpec( modify=True,
vsanClusterConfig=vim.VsanClusterConfigInfo( enabled=True,
defaultConfig=vim.VsanClusterConfigInfoHostDefaultInfo(autoClaimStorage=False)))

# Change the vSAN configuration to claim disks manually

task = vsanClusterSystem.VsanClusterReconfig(cluster, vsanReconfigSpec)
vsanapiutils.WaitForTasks([task], si)

4. Enabling Deduplication and Compression on All-Flash

It is often advantageous to enable Deduplication and Compression on All-Flash vSAN deployments.

Reduced capacity utilization can often make All-Flash vSAN more cost effective than Hybrid vSAN
deployments.

Enabling Deduplication and Compression is enabled either by a checkbox upon initially creating a
vSAN cluster, or can be enabled or disabled after the cluster has been created.

If a vSAN cluster is created without enabling Deduplication and Compression, the process of enabling
it performs a rolling upgrade, which can be time consuming, and in some cases require reduced
availability. Choosing to enable Deduplication and Compression at creation time can be beneficial to
mitigate the rolling upgrade process.

This can easily be done including code to enable Deduplication and Compression.

12

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

if isallFlash:
print 'Enable deduplication and compression for VSAN'
vsanReconfigSpec.dataEfficiencyConfig = vim.VsanDataEfficiencyConfig(
compressionEnabled = args.enabledc, deduplicationEnabled = args.enabledc)

# Enabled/Disable Deduplication and Compression

task = vsanClusterSystem.VsanClusterReconfig(cluster, vsanReconfigSpec)
vsanapiutils.WaitForTasks([task], si)

*Note Deduplication and Compression are not enabled independently using the vSphere Web Client.
There is no significant performance benefit enabling one and not the other. Enabling both is the only
supported configuration. The vSphere Web Client enables both simultaneously.

5. Networking: Multicast Configuration

vSAN requires Multicast to deliver metadata traffic among cluster nodes. Using the vSAN Wizard will
always set the default Multicast addressing. This process is not exposed by the vSAN Wizard.

In some situations, such as one where two independent vSAN clusters share the same Layer 2 network,
it is recommended to change the Multicast address for one of the two clusters. This can be
accomplished manually using the steps outlined in KB Article 2075451.

We can easily set the Multicast addressing using Python.

# Set the Multicast Addressing

configInfo = vim.VsanHostConfigInfo(
networkInfo=vim.VsanHostConfigInfoNetworkInfo(
port=[vim.VsanHostConfigInfoNetworkInfoPortConfig(
device=args.vmknic, ipConfig=vim.VsanHostIpConfig(
upstreamIpAddress='224.1.2.3',
downstreamIpAddress='224.2.3.4'
))]))
task = hostProps[host]['configManager.vsanSystem'].UpdateVsan_Task(configInfo)
# Execute the task
vsanapiutils.WaitForTasks(task, si)

This could be modified to accept arguments, rather than hard coded addresses, passed at runtime.

6. Networking: VMkernel Selection

Hosts must have a VMkernel interface tagged for vSAN traffic to be able to access a vSAN datastore.
This is relatively easy to perform on a single or few hosts, but can be challenging at scale.

To tag a VMkernel interface, a host must be selected, and then the Networking menu from the
Management tab selected. Once there the VMkernel interface that will be used for vSAN traffic must
be edited and tagged to include vSAN traffic.

13

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

While this is an easy task, it must be accomplished for each host in the cluster before vSAN is enabled.
Once each host has a VMkernel interface tagged for vSAN traffic, during setup the Wizard will indicate
that all hosts are properly configured.

Today, if hosts do not already have a VMkernel interface tagged for vSAN traffic, the Wizard must be
closed, VMkernel interfaces tagged, and the Wizard must be run again.

Selecting VMkernel NICs and enabling vSAN traffic can ensure proper configuration at script
execution.

# Update the configuration spec for VMkernet Networking

# Enumerate the selected VMkernel interface for each host, and add it to the list of tasks
for host in hosts:
print 'Enable VSAN traffic in host {} with {}'.format(hostProps[host]['name'], args.vmknic)
task = hostProps[host]['configManager.vsanSystem'].UpdateVsan_Task(configInfo)
tasks.append(task)

# Execute the tasks

vsanapiutils.WaitForTasks(tasks, si)

14

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

7. Disk Management and Claiming Disks

vSAN leverages storage devices of several types, depending on the architecture, for the purpose of
backing the vSAN datastore. There are different types of devices that vSAN uses, like storage
controllers, Solid-State Drives (SSDs), and traditional spinning Hard Disk Drives (HDDs).

Disks can be added to a vSAN cluster upon creation of the cluster, or can be added manually later
through Disk Management.

When using the vSphere Web Client, drives that have existing partitions are not listed as eligible to add
to the pool of disks used by vSAN. The task of listing eligible disks in the Web Client is done on the
backend, and not exposed to the administrator. Once selected, they will be claimed and added to the
vSAN cluster.

We can query each host in the cluster for eligible disks, and include the option to clear partitions on
disks that are not eligible. Once all eligible disks have been identified, they can be claimed.

for host in hosts:

# Enumerate find the ineligible disks
disks = [result.disk for result in
hostProps[host]['configManager.vsanSystem'].QueryDisksForVsan() if result.state ==
'ineligible']
print 'Find ineligible disks {} in host {}'.format([disk.displayName for disk in disks], hostProps
[host]['name'])

# For each disk, interactively ask the admin as to whether to individually wipe ineligible disks or not
for disk in disks:
if yes('Do you want to wipe disk {}?\nPlease Always check the partition table and the data
stored'
' on those disks before doing any wipe! (yes/no)?'.format(disk.displayName)):
hostProps[host]['configManager.storageSystem'].UpdateDiskPartitions(disk.deviceName,
vim.HostDiskPartitionSpec())

Also notice that in our wizard above that all disks are flash based devices. How do we determine which
devices to use for cache, and which devices to use for capacity? Because cache devices are normally
smaller than capacity devices, the wizard assigns Cache tier for the smaller devices and Capacity tier
for the larger devices. In a Hybrid configuration, flash devices would default to Cache tier and
traditional HDDs would be listed as Capacity tier devices.

15

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

diskmap = {host: {'cache':[],'capacity':[]} for host in hosts}

cacheDisks = []
capacityDisks = []

# For All Flash Architectures,

if isallFlash:
for host in hosts:
ssds = [result.disk for result in hostProps[host]
['configManager.vsanSystem'].QueryDisksForVsan() if
result.state == 'eligible' and result.disk.ssd]
smallerSize = min([disk.capacity.block * disk.capacity.blockSize for disk in ssds])
for ssd in ssds:
size = ssd.capacity.block * ssd.capacity.blockSize
if size == smallerSize:
diskmap[host]['cache'].append(ssd)
cacheDisks.append((ssd.displayName, sizeof_fmt(size), hostProps[host]['name']))
else:
diskmap[host]['capacity'].append(ssd)
capacityDisks.append((ssd.displayName, sizeof_fmt(size), hostProps[host]['name']))
else:
# For Hybrid Architectures
for host in hosts:
disks = [result.disk for result in hostProps[host]
['configManager.vsanSystem'].QueryDisksForVsan() if
result.state == 'eligible']
ssds = [disk for disk in disks if disk.ssd]
hdds = [disk for disk in disks if not disk.ssd]
for disk in ssds:
diskmap[host]['cache'].append(disk)
size = disk.capacity.block * disk.capacity.blockSize
cacheDisks.append((disk.displayName, sizeof_fmt(size), hostProps[host]['name']))
for disk in hdds:
diskmap[host]['capacity'].append(disk)
size = disk.capacity.block * disk.capacity.blockSize
capacityDisks.append((disk.displayName, sizeof_fmt(size), hostProps[host]['name']))

for host,disks in diskmap.iteritems():

if disks['cache'] and disks['capacity']:
dm = vim.VimVsanHostDiskMappingCreationSpec(
cacheDisks=disks['cache'], capacityDisks=disks['capacity'],
creationType='allFlash' if isallFlash else 'hybrid',
host=host)

# Execute the task

task = vsanVcDiskManagementSystem.InitializeDiskMappings(dm)
tasks.append(task)

8. Fault Domain Creation

vSAN 6 introduced the ability to logically group hosts into Fault Domains. This feature gives
administrators the ability to logically separate vSAN hosts much in the same way that hosts are
physically separated.

Grouping one or more hosts into a Fault Domain using the vSphere Web Client can be done upon
vSAN cluster creation, or after a cluster has been created

This is a process that must be accomplished by the Virtualization Admin manually.

Fault Domains and assigned hosts can easily be passed as script arguments.

16

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

# Perform these tasks if Fault Domains are passed as an argument

if args.faultdomains:
print 'Add fault domains in vsan'
faultDomains = []
#args.faultdomains is a string like f1:host1,host2 f2:host3,host4
for faultdomain in args.faultdomains.split():
fname, hostnames = faultdomain.split(':')
domainSpec = vim.cluster.VsanFaultDomainSpec(name=fname, hosts=[host for host in hosts
if hostProps[host]['name'] in hostnames.split(',')])
# Add Fault Domains to the Domain Specification
faultDomains.append(domainSpec)

# Apply the Domain Specification to the VSAN Config

vsanReconfigSpec.faultDomainsSpec = vim.VimClusterVsanFaultDomainsConfigSpec(
faultDomains=faultDomains)

# Configure Fault Domains

task = vsanClusterSystem.VsanClusterReconfig(cluster, vsanReconfigSpec)
vsanapiutils.WaitForTasks([task], si)

9. Finishing up

When completing a vSAN cluster setup using the vSAN Configuration Wizard, a nice summary screen
provides one last review of the proposed configuration before enabling vSAN with the selections a
Virtualization Administrator has made.

This review screen is the last opportunity to make sure that the cluster is configured properly. This is a
necessary step when manually configuring vSAN through the vSphere Web Client.

10. Enabling the Performance Service

vSAN 6.2 added a new Performance Service that maintains a database of performance metrics on the
vSAN datastore. It must be manually enabled for each vSAN cluster because it is not enabled during
initial setup.

To setup the Performance Service, select Health and Performance from the Manage tab of a vSAN
cluster.

17

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

It is even easier using code to enable the Performance Service on vSAN.

print 'Enable perf service on this cluster'

# Apply the Performance Service to the VSAN config
vsanPerfSystem = vcMos['vsan-performance-manager']

# Apply the config update

task = vsanPerfSystem.CreateStatsObjectTask(cluster)
vsanapiutils.WaitForTasks([task], si)

11. License Assignment

vSAN license assignment isn’t handled through the vSAN Configuration Wizard. vSAN clusters are
licensed through the vSphere Web Client. Multiple clusters can be licensed simultaneously through
the Web Client, but it is still a manual process.

The vSAN Management API allows includes the ability to assign licenses upon deployment of a vSAN
cluster.

if args.vsanlicense:
print 'Assign VSAN license'
lm = si.content.licenseManager
lam = lm.licenseAssignmentManager
lam.UpdateAssignedLicense(entity=cluster._moId, licenseKey=args.vsanlicense)

Using code to assign licenses to different clusters provides flexibility when assigning licenses to vSAN.
In cases where licensed features are upgraded across a large environment, using code to assign new
licensing could be significantly easier than manual license allocation.

Recipe Summary

Deploying vSAN manually is a fairly simple process. While there are some additional settings that
change the deployment options, it is still very simple. Despite the process being simplistic in nature,
there are always opportunities for error when manually performing processes at scale.

The above code snippets show how easy it is to automate the deployment of vSAN using code
consistently and repeatedly at both large and small scale.

18

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

The individual code snippets are available as a single Python script from the VMware Developer Center
at the following URL.

https://code.vmware.com/samples?id=1133

2.2 Configuring vSAN Stretched Clusters or 2 Node

Configuring vSAN Stretched Clusters or 2 Node

In vSAN 6.1, VMware introduced Stretched Clusters and 2 Node configurations. When deploying
Stretched Cluster or 2 Node configurations, one or more hosts are located in one fault domain, an
equal number of hosts in a second fault domain, and host that provides the Witness component for
the cluster.

Stretched Clusters and 2 Node configurations have a few distinct characteristics from traditional vSAN
clusters.

Effectively 2 comprised of between 1 and 15 nodes per site

Require a Witness Host to make up the 3rd site for metadata content
This can be a physical host or the freely available vSAN Witness Appliance

Important Note: The previous recipe is used starting with Step 3.

Again, this is sample code, not supported by VMware.

1. Deploying a vSAN Witness Appliance

The first step required in supporting vSAN Stretched Clusters or 2 Node configurations, is to either
configure a physical host, or a vSAN Witness Appliance to perform the Witness responsibilities.

The vSAN Witness Appliance is great alternative to using a physical ESXi host due to not requiring a
dedicated license or dedicated physical disks for vSAN metadata. Virtual Appliances are often
provided as an OVA file, or an OVF file, along with additional components such as a manifest and
virtual disk files.

There are several methods to import Virtual Appliances into a vSphere environment. These methods
include options to import Virtual Appliances as a menu item in the vSphere Web Client (with the help
of the vSphere Client Integration Plugin), or the legacy vSphere Client. There is also an OVF Tool,
provided for various operating systems, that allows for importing Virtual Appliances from a command
line.

The vSAN Witness Appliance can be downloaded from VMware.com:

https://my.vmware.com/web/vmware/info/slug/datacenter_cloud_infrastructure/
vmware_virtual_san/6_2#drivers_tools

To import a Virtual Appliance from the vSphere Web Client, an administrator only needs to right click
on the cluster the appliance will be deployed to, designate the Virtual Appliance source, select
additional items (such as host, datastore, network, etc.), as well as any prompts for settings specific to
the Virtual Appliance.

19

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Remember that previously mentioned tools can natively, or through the use of a plugin, upload Virtual
Appliances. Deploying the vSAN Witness Appliance can also be accomplished as part of a Python
script, but requires code to handle the upload process.

The vSAN Witness Appliance is available as an OVA. And OVA is a self-contained file that is comprised
of several files. Each of these files must be uploaded to vCenter Server during the OVF Deployment
process.

To upload multiple files, we first must be able to upload a single file.

20

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

def uploadFile(srcURL, dstURL, create, lease, minProgress, progressIncrement, vmName=None,

log=None):
'''
This function will upload vmdk file to vc by using http protocol
@param srcURL: source url
@param dstURL: destnate url
@param create: http request method
@param lease: HttpNfcLease object
@param minProgress: file upload progress initial value
@param progressIncrement: file upload progress update value
@param vmName: imported virtual machine name
@param log: log object
@return:
'''
srcData = urllib2.urlopen(srcURL)
length = int(srcData.headers['content-length'])
ssl._create_default_https_context = ssl._create_unverified_context
protocol, hostPort, reqStr = splitURL(dstURL)
dstHttpConn = createHttpConn(protocol, hostPort)
reqType = create and 'PUT' or 'POST'
dstHttpConn.putrequest(reqType, reqStr)
dstHttpConn.putheader('Content-Length', length)
dstHttpConn.endheaders()

bufSize = 1048768 # 1 MB
total = 0
progress = minProgress
if log:
# If args.log is available, then log to it
log = log.info
else:
log = sys.stdout.write
log("%s: %s: Start: srcURL=%s dstURL=%s\n" % (time.asctime(time.localtime()), vmName, srcURL,
dstURL))
log("%s: %s: progress=%d total=%d length=%d\n" % (time.asctime(time.localtime()), vmName,
progress, total, length))
while True:
data = srcData.read(bufSize)
if lease.state != vim.HttpNfcLease.State.ready:
break
dstHttpConn.send(data)
total = total + len(data)
progress = (int)(total * (progressIncrement) / length)
progress += minProgress
lease.Progress(progress)
if len(data) == 0:
break
log("%s: %s: Finished: srcURL=%s dstURL=%s\n" % (time.asctime(time.localtime()), vmName,
srcURL, dstURL))
log("%s: %s: progress=%d total=%d length=%d\n" % \ (time.asctime(time.localtime()), vmName,
progress, total, length))
log("%s: %s: Lease State: %s\n" % \
(time.asctime(time.localtime()), vmName, lease.state))
if lease.state == vim.HttpNfcLease.State.error:
raise lease.error
dstHttpConn.getresponse()
return progress

21

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Once the function is defined to upload a single file, another can be created to upload multiple files.

def uploadFiles(fileItems, lease, ovfURL, vmName=None, log=None):

'''
Upload witness vm's vmdk files to vCenter by using http protocol
@param fileItems: the source vmdks read from ovf file
@param lease: Represents a lease on a VM or a vApp, which can be used to import or export
disks for the entity
@param ovfURL: witness vApp ovf url
@param vmName: The name of witness vm
@param log:
@return:
'''
uploadUrlMap = {}
for kv in lease.info.deviceUrl:
uploadUrlMap[kv.importKey] = (kv.key, kv.url)

progress = 5
increment = (int)(90 / len(fileItems))
for file in fileItems:
ovfDevId = file.deviceId
srcDiskURL = urlparse.urljoin(ovfURL, file.path)
(viDevId, url) = uploadUrlMap[ovfDevId]
if lease.state == vim.HttpNfcLease.State.error:
raise lease.error
elif lease.state != vim.HttpNfcLease.State.ready:
raise Exception("%s: file upload aborted, lease state=%s" % \
(vmName, lease.state))
progress = uploadFile(srcDiskURL, url, file.create, lease, progress, increment, vmName, log)

Uploading files is only a small portion of Deploying a Virtual Appliance. Additional tasks such as
cluster, host, network, and datastore placement are required, as well as passing any parameters that
the Virtual Appliance requires.

22

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

def DeployWitnessOVF(ovfURL, si, host, vmName, dsRef, vmFolder, vmPassword=None,

network=None, log=None):
'''
Deploy witness VM to vCenter, the import process consists of two steps:
1>Create the VMs and/or vApps that make up the entity.
2>Upload virtual disk contents.
@param ovfURL: ovf source url
@param si: Managed Object ServiceInstance
@param host: HostSystem which the VM located
@param vmName: VM name
@param dsRef: Datastore which the VM located
@param vmFolder: Folder which the VM belong to
@param vmPassword: Password for the VM
@param network: Managed Object Network of the VM
@param log:
@return:
'''
rp = host.parent.resourcePool
params = vim.OvfManager.CreateImportSpecParams()
params.entityName = vmName
params.hostSystem = host
params.diskProvisioning = 'thin'

f = urllib.urlopen(ovfURL)
ovfData = f.read()

import xml.etree.ElementTree as ET

params.networkMapping = []
if vmPassword:
params.propertyMapping = [vim.KeyValue(key='vsan.witness.root.passwd',
value=vmPassword)]
ovf_tree = ET.fromstring(ovfData)

for nwt in ovf_tree.findall('NetworkSection/Network'):

nm = vim.OvfManager.NetworkMapping()
nm.name = nwt.attrib['name']
if network != None:
nm.network = network
else:
nm.network = host.parent.network[0]
params.networkMapping.append(nm)

res = si.content.ovfManager.CreateImportSpec(ovfDescriptor = ovfData,

resourcePool = rp, datastore = dsRef, cisp = params)
if isinstance(res, vim.MethodFault):
raise res
if res.error and len(res.error) > 0:
raise res.error[0]
if not res.importSpec:
raise Exception("CreateImportSpec raised no errors, but importSpec is not set")

lease = rp.ImportVApp(spec = res.importSpec, folder = vmFolder, host = host)

while lease.state == vim.HttpNfcLease.State.initializing:
time.sleep(1)

if lease.state == vim.HttpNfcLease.State.error:
raise lease.error

# Upload files
uploadFiles(res.fileItem, lease, ovfURL, vmName, log)
lease.Complete()
23
return lease.info.entity
Copyright © 2019 VMware, Inc. All rights reserved.
 vSAN API Cookbook for Python

The DeployWitnessOVF function takes care of setting networking, configuring the supplied password
as one of vApp options, and proper placement of the appliance on a specific host or resource pool.
Fortunately, the vSAN Witness Appliance only requires a password as an additional argument that
needs to be passed.

The DeployWitnessOVF function will parse the contents of the OVF, but does not have the capability
to parse an OVA, which the Witness Appliance is downloaded as. The Witness OVA file will have to be
extracted from an OVA to a folder containing the OVF and other required files.

The OVA file is essentially a .tar archive, that can be extracted easily using a wide variety of tools. For
the purpose of this paper, TarTool will be used due to its simplicity and command line availability. *No
specific recommendation of this tool should be inferred by the reader, as many tools provide this
functionality.

After deploying the vSAN Witness Appliance, the VM must be powered on and some tasks need to
take place. Putting these together, the process of deploying a Witness appliance could look something
like this:

24

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

print 'Start to add virtual witness host'

'''
Steps add virtual witness, rather than a dedicated physical ESXi host to be a witness host,
VMware has developed the VSAN witness appliance to take care of the witness requirements
1) Deploy witness VM specifying host, storage, & network for the Witness VM
2) Get the witness VM and add it to the data center as a witness host
'''
dc = searchIndex.FindChild(entity = si.content.rootFolder, name = args.datacenter)
#specify the host for the witness VM
hostSystem = getHostSystem(args.vmhost, dc, si)
#specify the storage for the witness VM
ds = searchIndex.FindChild(entity = dc.datastoreFolder, name = args.datastore)
#specify the network for the witness VM
if args.network:
network = [net for net in dc.networkFolder.childEntity
if net.name == args.network][0]
else:
network = dc.networkFolder.childEntity[0]

witnessVm = DeployWitnessOVF(args.ovfurl, si, hostSystem, args.name, ds, dc.vmFolder,

vmPassword=args.vmpassword,
network=network)
task = witnessVm.PowerOn()
vsanapiutils.WaitForTasks([task], si)

# Wait for vm to power on and become available

beginTime = time.time()
while True:
try:
# Connect to the Witness Host
SmartConnect(host=witnessVm.guest.ipAddress,
user='root',
pwd=args.vmpassword,
port=443,
sslContext=context)
except:
time.sleep(10)
timeWaiting = time.time() – beginTime
if timeWaiting > (15 * 60):
raise Exception("Timed out waiting (>15min) for VM to up!")
else:
break

2. Adding the Witness Appliance to vCenter Server

The vSAN Witness Host, either in physical or Virtual Appliance form, must be added to the vCenter
environment where the vSAN Stretched Cluster or 2 Node configuration is running. It is important to
remember that the Witness Host cannot be a member of the vSAN cluster.

Hosts can be manually added to a vSphere Datacenter, or clusters within that datacenter, using the
vSphere Web Client, vSphere Client, or other scripting tools. It is important to note that when using the
legacy vSphere Client, adding a vSAN Witness Appliance as a host will not properly assign the vSAN
Witness license. As a result, if manually adding a Witness, using the vSphere Web Client is a preferred
method versus the legacy vSphere Client.

25

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

A function is required to facilitate the process of adding the Witness Appliance as a host in vCenter.

def AddHost(host, user='root', pwd=None, dcRef=None, si=None, sslThumbprint=None,

port=443):
''' Add a host to a data center Returns a host system '''
cnxSpec = vim.HostConnectSpec(
force=True, hostName=host, port=port, userName=user, password=pwd,
vmFolder=dcRef.vmFolder)
if sslThumbprint:
cnxSpec.sslThumbprint = sslThumbprint
hostParent = dcRef.hostFolder
try:
task = hostParent.AddStandaloneHost(addConnected = True, spec = cnxSpec)
vsanapiutils.WaitForTasks([task], si)
return getHostSystem(host, dcRef, si)
except vim.SSLVerifyFault as e:
#By catching this exception, user doesn't need input the host's thumbprint of the SSL certificate,
the logic below
#will do this automatically
cnxSpec.sslThumbprint = e.thumbprint
task = hostParent.AddStandaloneHost(addConnected = True, spec = cnxSpec)
vsanapiutils.WaitForTasks([task], si)
return getHostSystem(host, dcRef, si)
except vim.DuplicateName as e:
raise Exception("AddHost: ESX host %s has already been added to VC." % host)

Adding the host becomes relatively easy using the AddHost function created above.

print 'Add witness host {} to datacenter {}'.format(witnessVm.name, args.witnessdc)

dcRef = searchIndex.FindChild(entity = si.content.rootFolder, name = args.witnessdc)
witnessHost = AddHost(witnessVm.guest.ipAddress, pwd=args.vmpassword, dcRef=dcRef,
si=si)

26

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

3. Configuring an existing cluster as a Stretched Cluster or 2 Node

The process of configuring a vSAN cluster as either a Stretched Cluster or 2 Node configuration can be
done upon initial creation, or after a vSAN cluster has been initially setup.

To convert an existing cluster to a Stretched or 2 Node cluster, the process is as follows.

• Choose which host(s) will be placed in the Preferred Fault Domain

• Choose which host(s) will be placed in the Secondary Fault Domain
• Select a Witness host
◦ Choose cache and capacity disks for the Witness host
• Complete the setup

Reusing much of the code from the Deploy recipe, some more tasks are still required, including setting
up Fault Domains, designating a Witness Host, and selecting the Witness Host disks that will be used.

Configuring Fault Domains

Selecting a Witness Host

Selecting Witness Host disks

27

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

To setup Fault Domains, we can enumerate the hosts in the cluster, answer yes or no for which Fault
Domain we choose to place the hosts into, saving that information in an array.

preferedFd = args.preferdomain
secondaryFd = args.seconddomain
firstFdHosts = []
secondFdHosts = []
for host in hosts:
if yes('Add host {} to preferred fault domain ? (yes/no)'.format(hostProps[host]['name'])):
firstFdHosts.append(host)
for host in set(hosts) - set(firstFdHosts):
if yes('Add host {} to second fault domain ? (yes/no)'.format(hostProps[host]['name'])):
secondFdHosts.append(host)
faultDomainConfig = vim.VimClusterVSANStretchedClusterFaultDomainConfig(
firstFdHosts = firstFdHosts,
firstFdName = preferedFd,
secondFdHosts = secondFdHosts,
secondFdName = secondaryFd )

Like in the previous recipe, only eligible disks can be used for vSAN devices. Putting these in an array
will allow for them to be claimed at the time of cluster configuration.

28

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

disks = [result.disk for result in witnessHost.configManager.vsanSystem.QueryDisksForVsan() if

result.state == 'eligible']
diskMapping = None
if disks:
ssds = [disk for disk in disks if disk.ssd]
nonSsds = [disk for disk in disks if not disk.ssd]
#host with hybrid disks
if len(ssds) > 0 and len(nonSsds) > 0:
diskMapping = vim.VsanHostDiskMapping(
ssd = ssds[0],
nonSsd = nonSsds
)
#host with all-flash disks,choose the ssd with smaller capacity for cache layer.
if len(ssds) > 0 and len(nonSsds) == 0:
smallerSize = min([disk.capacity.block * disk.capacity.blockSize for disk in ssds])
smallSsds = []
biggerSsds = []
for ssd in ssds:
size = ssd.capacity.block * ssd.capacity.blockSize
if size == smallerSize:
smallSsds.append(ssd)
else:
biggerSsds.append(ssd)
diskMapping = vim.VsanHostDiskMapping(
ssd = smallSsds[0],
nonSsd = biggerSsds
)

Once hosts have been put into two Fault Domain arrays, and eligible disks have been determined for
the Witness host, reconfiguring the Cluster can occur.

print 'start to create stretched cluster'

task = vsanScSystem.VSANVcConvertToStretchedCluster(
cluster=cluster,
faultDomainConfig=faultDomainConfig,
witnessHost=witnessHost, preferredFd=preferedFd,
diskMapping=diskMapping)
vsanapiutils.WaitForTasks([task], si)

Recipe Summary

Just as deploying vSAN was in the first recipe, Stretched Cluster and 2 Node configurations can also
be easily and repeatedly automated with code.

The above code snippets show how easy it is to automate Stretched Cluster and 2 Node configurations
of vSAN using code consistently and repeatedly at both large and small scale.

The individual code snippets are available as a single Python script from the VMware Developer Center
at the following URL.

https://code.vmware.com/samples?id=1134

2.3 Performing a vSAN On-Disk Upgrade

29

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

Performing a vSAN On-Disk Upgrade

Upgrading vSphere hosts is a process that has been around for quite some time. There are several
methods of upgrading vSphere hosts like Upgrade Manager, esxcli, and more. These existing methods
will upgrade ESXi hosts that are running vSAN, but will not upgrade the vSAN On-Disk format.

The VSAN-FS On-Disk format initially introduced in vSAN 5.5 is version 1.0. vSAN 6 introduced version
2.0. Version 2.0 of the vSAN On-Disk format brought about changes, allowing for configurations such
as All-Flash architectures, Stretched Clusters, support of the vsanSparse format for snapshots, and
more. With the introduction of vSAN 6.2, version 3.0 was introduced. Version 3.0 supports even more
functionality with Deduplication and Compression as well as Erasure Coding for All-Flash
architectures, and features including software checksums and IOPS limits for all architectures.

Depending on the configuration and state of a vSAN cluster, performing an On-Disk Format upgrade
can have different requirements. This is because the On-Disk format upgrade process essentially
evacuates data from each Disk Group in a host, removes and recreates the Disk Group in the new
format, and migrates data back, all in a rolling upgrade process.

This recipe will cover some of the things to consider when programmatically updating the On-Disk
format of vSAN.

1. Determining the current On-Disk format

Before attempting to perform a vSAN On-Disk upgrade, it is important to know what the current
format is, as well as the latest supported format for the ESXi build that vSAN cluster is running.

This is automatically visible within the vSphere Web Client from the Manage tab under Settings for a
vSphere cluster running vSAN.

Notice in the graphic that the Disk format version is 2.0, yet upgradeable to 3.0. Additionally, all the
disks, 6 of 6, are running the outdated version.

The vSphere Web Client easily shows this information, but requires a Virtualization Admin to manually
check the status of the On-Disk format for the cluster.

We’ll connect to the cluster, and determine the highest On-Disk Format supported by the cluster.

30

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

cluster = getClusterInstance(args.clusterName, si)

vcMos = vsanapiutils.GetVsanVcMos(si._stub, context=context)
vsanUpgradeSystem = vcMos['vsan-upgrade-systemex']
supportedVersion = vsanUpgradeSystem.RetrieveSupportedVsanFormatVersion(cluster)
print 'The highest vSAN disk format version that given cluster supports is {}'.format
(supportedVersion)

A function, provides the ability to compare the existing On-Disk version to the latest supported
version.

def hasOlderVersionDisks(hostDiskMappings, supportedVersion):

for hostDiskMappings in hostDiskMappings:
for diskMapping in hostDiskMappings:
if diskMapping.ssd.vsanDiskInfo.formatVersion < supportedVersion:
return True
for disk in diskMapping.nonSsd:
if disk.vsanDiskInfo.formatVersion < supportedVersion:
return True
return False

We can gather each of the disk group member devices into diskMappings, then pass them into the
hasOlderVersionDisks function to determine if an upgrade is necessary or not.

vsanSystems = CollectMultiple(si.content, cluster.host,

['configManager.vsanSystem']).values()
vsanClusterSystem = vcMos['vsan-cluster-config-system']
diskMappings = CollectMultiple(si.content, [vsanSystem['configManager.vsanSystem'] for
vsanSystem in vsanSystems],
['config.storageInfo.diskMapping']).values()

diskMappings = [diskMapping['config.storageInfo.diskMapping'] for diskMapping in

diskMappings]
needsUpgrade = hasOlderVersionDisks(diskMappings, supportedVersion)

In cases where a cluster is already at the highest level support, there is obviously no need to perform
an upgrade. In cases where an upgrade is necessary, there are some additional tasks that need to be
performed, depending on the cluster’s configuration.

2. Role of Disk Claiming in vSAN On-Disk Format Upgrades

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

31

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

A Virtualization Admin will have to go and change the Disk Claiming method from automatic to
manual.

Having to make this change across more than one cluster can be very time consuming process. This
can be better accomplished through an upgrade script checking for automatic disk claiming, and
changing the cluster’s disk claiming method to manual.

autoClaimChanged = False
if vsanConfig.defaultConfig.autoClaimStorage:
print 'autoClaimStorage should be set to false before upgrade VSAN disks'
autoClaimChanged = True
vsanReconfigSpec = vim.VimVsanReconfigSpec(
modify = True,
vsanClusterConfig = vim.VsanClusterConfigInfo(
defaultConfig = vim.VsanClusterConfigInfoHostDefaultInfo(
autoClaimStorage = False)))

task = vsanClusterSystem.VsanClusterReconfig(cluster, vsanReconfigSpec)

vsanapiutils.WaitForTasks([task], si)

32

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

3. On-Disk Format Upgrade Preflight Check

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

When an upgrade is selected from the vSphere Web Client, a “preflight” check occurs. While this is
handled by the vSphere Web Client, it must also be done in script.

print 'Perform VSAN upgrade preflight check'

upgradeSpec = vim.VsanDiskFormatConversionSpec(
dataEfficiencyConfig = vim.VsanDataEfficiencyConfig(
compressionEnabled = args.enabledc, deduplicationEnabled = args.enabledc))

**Note Deduplication and Compression are not enabled independently using the vSphere Web Client.
There is no significant performance benefit enabling one and not the other. Enabling both is the only
supported configuration. The vSphere Web Client enables both simultaneously.

If there are any issues with the preflight check, they must be resolved beforehand. We’ll need to
indicate this in our script so the Virtualization Admin can address accordingly.

issues = vsanUpgradeSystem.PerformVsanUpgradePreflightCheckEx(cluster, spec =

upgradeSpec).issues
if issues:
print 'Please fix the issues before upgrade VSAN'
for issue in issues:
print issue.msg
return

4. Reduced Redundancy During On-Disk Format Upgrades

vSAN On-Disk format upgrades require the existing VM storage policies to be satisfied during the
upgrade process.

In a 3 node cluster, a Failure To Tolerate =1 policy requires 3 nodes. Bringing a node offline to perform
an upgrade would create a situation of reduced redundancy.

This doesn’t mean that 3 node clusters can never be upgraded. It only means that they can only be
upgraded with reduced redundancy.

By default, the upgrade process does not allow reduced redundancy. Attempting to perform an On-
Disk format upgrade without sufficient spare resources will fail.

In the case where there are not enough vSAN resources to satisfy a VM storage policy, such as a 3
node cluster with FTT=1 using mirroring, a reduced redundancy flag must be set. There is no way to

33

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

accomplish this through the vSphere Web Client, and Virtualization Admins are required to perform
the upgrade from the Ruby vSphere Console (RVC). This process is detailed in KB 2113221 for the
Version 1 to Version 2 upgrade. The Version 3 upgrade is similar.

If performing an On-Disk format upgrade using a Python script, allowing the upgrade to occur with the
reduced redundancy flag can be set as part of initiating the upgrade.

print 'call PerformVsanUpgradeEx to upgrade disk versions'

task = vsanUpgradeSystem.PerformVsanUpgradeEx(cluster=cluster,
performObjectUpgrade=args.objupgrade,
allowReducedRedundancy=args.reduceredundancy)

5. Enabling Deduplication and Compression during an On-Disk Format Upgrade

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

When vSAN 6.2 hosts continue to operate with the Version 2.0 On-Disk format, changing the
deduplication and compression setting to enabled will automatically initiate the upgrade process to
the Version 3.0 On-Disk format.

With that being stated, when upgrading from the Version 2.0 format to Version 3.0, deduplication and
compression may or may not be enabled simultaneously. However, if an administrator simply chooses
to upgrade to the new Version 3.0 format, the process to enable deduplication and compression would
then have to be run afterwards, as a separate rolling upgrade.

This process is not explicitly detailed, and scripting the process could easily accommodate both
upgrading and enabling deduplication and compression simultaneously.

print 'call PerformVsanUpgradeEx to upgrade disk versions'

task = vsanUpgradeSystem.PerformVsanUpgradeEx(cluster=cluster,
performObjectUpgrade=args.objupgrade,
allowReducedRedundancy=args.reduceredundancy)

Recipe Summary

In some cases, it may be significantly easier to execute a simple script to accomplish more tasks than
to manually execute equivalent tasks from the vSphere Web Client, such as upgrading the On-Disk
format with reduced redundancy, while enabling deduplication and compression simultaneously.

The above code snippets show how easy it is to upgrade the On-Disk format of a vSAN cluster using a
simple script along with specific parameters like “--enabledc” and “--reduced-redundancy”.

The individual code snippets are available as a single Python script from the VMware Developer Center
at the following URL.

https://code.vmware.com/samples?id=1135

Performing a vSAN On-Disk Upgrade

Upgrading vSphere hosts is a process that has been around for quite some time. There are several
methods of upgrading vSphere hosts like Upgrade Manager, esxcli, and more. These existing methods
will upgrade ESXi hosts that are running vSAN, but will not upgrade the vSAN On-Disk format.

The VSAN-FS On-Disk format initially introduced in vSAN 5.5 is version 1.0. vSAN 6 introduced version
2.0. Version 2.0 of the vSAN On-Disk format brought about changes, allowing for configurations such
as All-Flash architectures, Stretched Clusters, support of the vsanSparse format for snapshots, and
more. With the introduction of vSAN 6.2, version 3.0 was introduced. Version 3.0 supports even more

34

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

functionality with Deduplication and Compression as well as Erasure Coding for All-Flash
architectures, and features including software checksums and IOPS limits for all architectures.

Depending on the configuration and state of a vSAN cluster, performing an On-Disk Format upgrade
can have different requirements. This is because the On-Disk format upgrade process essentially
evacuates data from each Disk Group in a host, removes and recreates the Disk Group in the new
format, and migrates data back, all in a rolling upgrade process.

This recipe will cover some of the things to consider when programmatically updating the On-Disk
format of vSAN.

Determining the current On-Disk format

Before attempting to perform a vSAN On-Disk upgrade, it is important to know what the current
format is, as well as the latest supported format for the ESXi build that vSAN cluster is running.

This is automatically visible within the vSphere Web Client from the Manage tab under Settings for a
vSphere cluster running vSAN.

Notice in the graphic that the Disk format version is 2.0, yet upgradeable to 3.0. Additionally, all the
disks, 6 of 6, are running the outdated version.

The vSphere Web Client easily shows this information, but requires a Virtualization Admin to manually
check the status of the On-Disk format for the cluster.

We’ll connect to the cluster, and determine the highest On-Disk Format supported by the cluster.

cluster = getClusterInstance(args.clusterName, si)

vcMos = vsanapiutils.GetVsanVcMos(si._stub, context=context)
vsanUpgradeSystem = vcMos['vsan-upgrade-systemex']
supportedVersion = vsanUpgradeSystem.RetrieveSupportedVsanFormatVersion(cluster)
print 'The highest vSAN disk format version that given cluster supports is {}'.format
(supportedVersion)

A function, provides the ability to compare the existing On-Disk version to the latest supported
version.

def hasOlderVersionDisks(hostDiskMappings, supportedVersion):

for hostDiskMappings in hostDiskMappings:
for diskMapping in hostDiskMappings:
if diskMapping.ssd.vsanDiskInfo.formatVersion < supportedVersion:
return True
for disk in diskMapping.nonSsd:
if disk.vsanDiskInfo.formatVersion < supportedVersion:
return True
return False

We can gather each of the disk group member devices into diskMappings, then pass them into the
hasOlderVersionDisks function to determine if an upgrade is necessary or not.

35

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

vsanSystems = CollectMultiple(si.content, cluster.host,

['configManager.vsanSystem']).values()
vsanClusterSystem = vcMos['vsan-cluster-config-system']
diskMappings = CollectMultiple(si.content, [vsanSystem['configManager.vsanSystem'] for
vsanSystem in vsanSystems],
['config.storageInfo.diskMapping']).values()

diskMappings = [diskMapping['config.storageInfo.diskMapping'] for diskMapping in

diskMappings]
needsUpgrade = hasOlderVersionDisks(diskMappings, supportedVersion)

In cases where a cluster is already at the highest level support, there is obviously no need to perform
an upgrade. In cases where an upgrade is necessary, there are some additional tasks that need to be
performed, depending on the cluster’s configuration.

Role of Disk Claiming in vSAN On-Disk Format Upgrades

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

A Virtualization Admin will have to go and change the Disk Claiming method from automatic to
manual.

Having to make this change across more than one cluster can be very time consuming process. This
can be better accomplished through an upgrade script checking for automatic disk claiming, and
changing the cluster’s disk claiming method to manual.

autoClaimChanged = False
if vsanConfig.defaultConfig.autoClaimStorage:
print 'autoClaimStorage should be set to false before upgrade VSAN disks'
autoClaimChanged = True
vsanReconfigSpec = vim.VimVsanReconfigSpec(
modify = True,
vsanClusterConfig = vim.VsanClusterConfigInfo(
defaultConfig = vim.VsanClusterConfigInfoHostDefaultInfo(
autoClaimStorage = False)))

task = vsanClusterSystem.VsanClusterReconfig(cluster, vsanReconfigSpec)

vsanapiutils.WaitForTasks([task], si)

On-Disk Format Upgrade Preflight Check

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

When an upgrade is selected from the vSphere Web Client, a “preflight” check occurs. While this is
handled by the vSphere Web Client, it must also be done in script.

print 'Perform VSAN upgrade preflight check'

upgradeSpec = vim.VsanDiskFormatConversionSpec(
dataEfficiencyConfig = vim.VsanDataEfficiencyConfig(
compressionEnabled = args.enabledc, deduplicationEnabled = args.enabledc))

**Note Deduplication and Compression are not enabled independently using the vSphere Web Client.
There is no significant performance benefit enabling one and not the other. Enabling both is the only
supported configuration. The vSphere Web Client enables both simultaneously.

36

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

If there are any issues with the preflight check, they must be resolved beforehand. We’ll need to
indicate this in our script so the Virtualization Admin can address accordingly.

issues = vsanUpgradeSystem.PerformVsanUpgradePreflightCheckEx(cluster, spec =

upgradeSpec).issues
if issues:
print 'Please fix the issues before upgrade VSAN'
for issue in issues:
print issue.msg
return

Reduced Redundancy During On-Disk Format Upgrades

vSAN On-Disk format upgrades require the existing VM storage policies to be satisfied during the
upgrade process.

In a 3 node cluster, a Failure To Tolerate =1 policy requires 3 nodes. Bringing a node offline to perform
an upgrade would create a situation of reduced redundancy.

This doesn’t mean that 3 node clusters can never be upgraded. It only means that they can only be
upgraded with reduced redundancy.

By default, the upgrade process does not allow reduced redundancy. Attempting to perform an On-
Disk format upgrade without sufficient spare resources will fail.

In the case where there are not enough vSAN resources to satisfy a VM storage policy, such as a 3
node cluster with FTT=1 using mirroring, a reduced redundancy flag must be set. There is no way to
accomplish this through the vSphere Web Client, and Virtualization Admins are required to perform
the upgrade from the Ruby vSphere Console (RVC). This process is detailed in KB 2113221 for the
Version 1 to Version 2 upgrade. The Version 3 upgrade is similar.

If performing an On-Disk format upgrade using a Python script, allowing the upgrade to occur with the
reduced redundancy flag can be set as part of initiating the upgrade.

print 'call PerformVsanUpgradeEx to upgrade disk versions'

task = vsanUpgradeSystem.PerformVsanUpgradeEx(cluster=cluster,
performObjectUpgrade=args.objupgrade,
allowReducedRedundancy=args.reduceredundancy)

Enabling Deduplication and Compression during an On-Disk Format Upgrade

If vSAN is configured to automatically claim disks, an On-Disk format upgrade cannot occur. In the
vSphere Web Client, if this is the case, an error will occur.

When vSAN 6.2 hosts continue to operate with the Version 2.0 On-Disk format, changing the
deduplication and compression setting to enabled will automatically initiate the upgrade process to
the Version 3.0 On-Disk format.

With that being stated, when upgrading from the Version 2.0 format to Version 3.0, deduplication and
compression may or may not be enabled simultaneously. However, if an administrator simply chooses
to upgrade to the new Version 3.0 format, the process to enable deduplication and compression would
then have to be run afterwards, as a separate rolling upgrade.

This process is not explicitly detailed, and scripting the process could easily accommodate both
upgrading and enabling deduplication and compression simultaneously.

37

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

print 'call PerformVsanUpgradeEx to upgrade disk versions'

task = vsanUpgradeSystem.PerformVsanUpgradeEx(cluster=cluster,
performObjectUpgrade=args.objupgrade,
allowReducedRedundancy=args.reduceredundancy)

Recipe Summary

In some cases, it may be significantly easier to execute a simple script to accomplish more tasks than
to manually execute equivalent tasks from the vSphere Web Client, such as upgrading the On-Disk
format with reduced redundancy, while enabling deduplication and compression simultaneously.

The above code snippets show how easy it is to upgrade the On-Disk format of a vSAN cluster using a
simple script along with specific parameters like “--enabledc” and “--reduced-redundancy”.

The individual code snippets are available as a single Python script from the VMware Developer Center
at the following URL.

https://developercenter.vmware.com/samples?id=1135

38

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

3. References
.

39

Copyright © 2019 VMware, Inc. All rights reserved.

 vSAN API Cookbook for Python

3.1 Additional Documentation

For more information about VMware vSAN, please visit the product pages at http://www.vmware.com/
products/virtual-san

Below are some links to online documentation:

vSAN Product Documentation

vSAN 6.2 Design and Sizing Guide

vSAN 6.2 Network Design Guide

vSAN 6.2 Stretched Cluster & 2 Node Guide

VMware vSAN Community

Support Knowledge base

VMware vSphere and VMware vCenter Server Resources:

Product Overview

Product Documentation

3.2 VMware Contact Information

For additional information or to purchase VMware vSAN, VMware’s global network of solutions
providers is ready to assist. If you would like to contact VMware directly, you can reach a sales
representative at 1-877-4VMWARE (650-475-5000 outside North America) or email
sdssales@vmware.com. When emailing, please include the state, country, and company name from
which you are inquiring.

3.3 About the Author

This cookbook was put together using content from various resources from vSAN Engineering.

Jase McCarty is a Staff Technical Marketing Architect at VMware with a focus on storage solutions. He
has been in the Information Technology field for over 25 years, with roles on both the customer and
vendor side. Jase has Co-Authored two books on VMware virtualization, routinely speaks technology
focused user group meetings, and has presented at VMworld and EMC World.
Follow Jase on Twitter: @jasemccarty

40

Copyright © 2019 VMware, Inc. All rights reserved.