VMWARE CLOUD FOUNDATION -

REDEFINING SOFTWARE-DEFINED
DATACENTERS

Anuj Sharma
Sr. Principal Engineer
Dell
Anuj.sharma@dell.com

Benedikt Mayer
Sr. Principal Engineer
Dell
Benedikt.mayer@dell.com

Knowledge Sharing Article

© 2020 Dell Inc. or its subsidiaries.
The Dell Technologies Proven Professional Certification program validates a wide range of skills and
competencies across multiple technologies and products.

From Associate, entry-level courses to Expert-level, experience-based exams, all professionals in or

looking to begin a career in IT benefit from industry-leading training and certification paths from one of
the world’s most trusted technology partners.

Proven Professional certifications include:

• Cloud
• Converged/Hyperconverged Infrastructure
• Data Protection
• Data Science
• Networking
• Security
• Servers
• Storage
• Enterprise Architect

Courses are offered to meet different learning styles and schedules, including self-paced On Demand,
remote-based Virtual Instructor-Led and in-person Classrooms.

Whether you are an experienced IT professional or just getting started, Dell Technologies Proven
Professional certifications are designed to clearly signal proficiency to colleagues and employers.

Learn more at www.dell.com/certification

2020 Dell Technologies Proven Professional Knowledge Sharing 2

Table of Contents
Overview............................................................................................................................... 4
Architecture .......................................................................................................................... 5
Standard Architecture Model ................................................................................................................... 5
Consolidated Architecture Model ............................................................................................................. 9
Stretched vSAN Cluster with VMware Cloud Foundations ..................................................................... 10
NSX-V Integration with VMware Cloud Foundation ............................................................................... 10
NSX-T Integration with VMware Cloud Foundation ............................................................................... 10
VMware Validated Design with VMware Cloud Foundation .................................................................. 10
VMware Enterprise PKS on VMware NSX-T Workload Domain ............................................................. 10
VMware Cloud Foundation Deployment Overview ................................................................................ 11
Why VMware Cloud Foundation on VxRAIL ........................................................................................... 12
Case Study........................................................................................................................... 13
Engagement Background ........................................................................................................................ 13
Conceptual Design .................................................................................................................................. 14
Overview - Logical Design ....................................................................................................................... 17
Overview - Physical Design ..................................................................................................................... 18
Troubleshooting .................................................................................................................. 24
References .......................................................................................................................... 25

Disclaimer: The views, processes or methodologies published in this article are those of the authors.
They do not necessarily reflect Dell Technologies’ views, processes or methodologies.

Dell.com/certification 3
Overview
Industries are undergoing digital transformation on a path to innovate and adapt to new customer
behavior. This rapid pace of change requires a cloud platform that can be quickly deployed and
consumed. VMware Cloud Foundation combines VMware’s leading technologies to deliver a
standardized software defined datacenter (SDDC) platform to environments of all size.

VMware Cloud Foundation provides automated end to end lifecycle management combines compute,
network and storage virtualization as well as cloud management into as single solution.

VMware Cloud Foundation (VCF) on VxRail provides the simplest path to a multi-cloud solution through
automated deployment of a fully integrated platform that leverages native VxRail hardware and
software capabilities paired with VMware’s stack. These components work together to deliver a new
turnkey solution.

VCF on VxRail is engineered to standardize the architecture of all SDDC components like vSphere, VSAN,
and NSX to provide time to value from the large-scale data center to edge locations. Another important
aspect is lifecycle automation provided by Cloud Foundation, that streamlines the upgrade experience
for the full SDDC software and hardware stack.

2020 Dell Technologies Proven Professional Knowledge Sharing 4

Architecture
VMware Cloud Foundations support two Architecture Models – standard and consolidated. Both
architecture models are based on the concept of workload domains. A workload domain can be visualized
as carved out pooled container of compute, storage and networking resources from the available
resources within VMware Cloud Foundation System. A workload domain consists of one or more ESXi
vSAN Clusters with NSX Networking. In VCF, there are different types of workload domains

• Management Workload Domain

Management Workload Domain is created during the initial bring-up of VMware Cloud
Foundations system. Management workload domain hosts the Management Components of
VMware Cloud Foundation system like Management vCenter Server, VI Workload Domains
vCenter, SDDC Manager, and NSX Manager for all the Workload Domains. Minimum of four
Nodes are required in Management Workload Domain.
• VI Workload Domain
VI Workload Domain can also be referred to as Production Workload Domains as
customer/tenant workload will be hosted in these workload domains. Minimum of three nodes
are required in VI Workload Domain.
• Horizon Workload Domain
Horizon Workload Domain automates deployment of VMware Horizon components which
enables delivery of Virtual Desktop Infrastructure (VDI) and Remote Desktop Session Host
desktops and applications.

This chapter will discuss VMware Cloud Foundations Architecture models along with other relevant topics
related to architecture of VMware Cloud Foundations.

Standard Architecture Model

Standard Architecture Model is the preferred model of deployment and only model supported with
VMware Cloud Foundations on VxRail. Standard Architecture Model has dedicated servers for
Management Workload Domain and VI Workload Domain. This segregates Management Components of
VCF from Tenant Workloads on Physical Layer as well. All domains – vCenter, NSX Manager, and Horizon
View Management VM’s – will reside on Management Workload Domain and Tenant Workload
VM’s/VDI’s will reside on Tenant Workload Domain along with NSX-V Controllers in the case of NSX-V.
Figure 1 shows examples of Standard Architecture Model.

Dell.com/certification 5
 TOR-1
TOR-2

NSX-V NSX-T

MANAGEMENT WORKLOAD DOMAIN

MGMT SDDC
MGR MGR

Management ESXi Cluster

VCR MGR
MWD TWD
NSX-V
VRLI
PSC1 CNTL 1
VSAN MN
MWD

NSX-V
VRLI
PSC2 CNTL 2
WN1
MWD
NSX-V
TENANT VRLI
CNTL 3
Network
Security

VCR WN2
MWD
TENANT
TENANT
NSX
VM

TENANT WORKLOAD DOMAIN

CNTL 1
TENAT ESXi CLUSTER

TENANT
TENANT
NSX
VM
CNTL 2
VSAN

TENANT
TENANT
NSX
VM
CNTL 3

TENANT
VM

Figure 1

Figure 1 shows a single rack deployment with one Management Workload Domain with four Nodes and
one Tenant Workload Domain with four Nodes. Minimum recommended nodes for Management Cluster
are four.

In Management Workload Domain we have the following components deployed:

• Management Workload Domain vCenter

• Platform Service Controllers which server as a single sign on domain for Tenant vCenter Servers
and Management Workload Domain vCenter Server.
• Tenant Workload Domain vCenter
• NSX-V Manager for Management Workload Domain
• NSX-V Controllers for Management Workload Domain
• NSX-T Manager for Tenant Workload Domain
• SDDC Manager
• vRealize Log Insight (Log Insight License for Management Workload Domain is included. Tenant
Workload Domains can also be added with separate license)

2020 Dell Technologies Proven Professional Knowledge Sharing 6

In Tenant Workload Domain we have the following components deployed:

• Tenant Workload VM’s

SPINE-1

SPINE-2

TOR-1 TOR-1

TOR-2 TOR-2

NSX-V NSX-T TENANT

MANAGEMENT WORKLOAD DOMAIN

MGMT SDDC NSX-V
MGR MGR
Management ESXi Cluster

CNTL1 T2

TENANT 2 ESXi CLUSTER

2 VM

TENANT 2 WORKLOAD DOMAIN

VCR MGR
MWD T1
NSV-V NSX-V TENANT
TENANT VRLI
PSC1 CNTL 1 CNTL2 T2 2 VM

VSAN
2 VCR MN
VSAN

MWD

NSX-V
NSV-V NSX-V NSX-V TENANT
VRLI
PSC2 CNTL 2 MGR CNTL3 T2 2 VM
T2 WN1
MWD
NSV-V TENANT TENANT
TENANT 1 HORIZON VRLI

Network
CNTL 3

Security
VIEW VMs
2 VM 2 VM
Network
Security

VCR TENANT 3 WN2

MWD

TENANT 1 TENANT 3 NSX-V

VDI CNTL1 T3
TENANT 1 WORKLOAD DOMAIN

TENANT 3 WORKLOAD DOMAIN

VM

TENAT 3 ESXi CLUSTER

TENAT ESXi CLUSTER

TENANT 3 NSX-V
TENANT 1
VDI CNTL2 T3
HORIZON

VM

VSAN
NSX-T
VSAN

TENANT 1 TENANT 3 NSX-V

VDI CNTL3 T3
VM

TENANT 1 TENANT 3
VDI
VM

Figure 2

Figure 2 depicts a multi-rack deployment in Leaf Spine Network Topology where each rack has two Top
of the Rack Switches, with each rack connecting to Spine Switches. Also in the above deployment we can
see that we have one Management Workload Domain, one NSX-V Tenant Workload Domain, one NSX-T
Tenant Workload Domain and one Horizon Workload Domain. This shows that multiple tenants with
different configurations can be accommodated in a Single VMware Cloud Foundation Instance.

Dell.com/certification 7
Management Workload Domain Components

• Management Workload Domain vCenter

• Platform Service Controllers which server as a single sign on domain for Tenant vCenter Servers
and Management Workload Domain vCenter Server
• NSX-V Manager for Management Workload Domain
• NSX-V Controllers for Management Workload Domain
• SDDC Manager
• vRealize Log Insight (Log Insight License for Management Workload Domain is included. Tenant
Workload Domains can also be added with separate license)
• Tenant 1 Workload Domain vCenter
• NSX-T Manager for Tenant Workload Domain 1
• Tenant 2 Workload Domain vCenter
• NSX-V Manager for Tenant Workload Domain 2
• Tenant 3 Workload Domain vCenter.
• NSX-V Manager for Tenant Workload Domain 3
• Horizon View VM’s for Tenant Workload Domain 3 (Composers, Connection Servers, App
Volume Managers, User Environment Manager, Unified Access Gateway, Load Balancers)

Tenant 1 Workload Domain Components

• Tenant Workload VM’s

Tenant 2 Workload Domain Components

• Tenant Workload VM’s

• NSX-V Controllers

Tenant 3 Workload Domain Components

• Tenant Workload VDI’s

• NSX-V Controllers.

2020 Dell Technologies Proven Professional Knowledge Sharing 8

Consolidated Architecture Model
Consolidated Model is supported by VMware for smaller environments, i.e. less than six nodes. In
Consolidated Deployment Model there is no segregation of physical hardware for Management
Workload Domain and Tenant Workload Domain. All resources are shared by both Management
Components and Tenant Workloads. Horizon and NSX-T are not supported with this model.

TOR-1
TOR-2

NSX-V SDDC
vCenter
MGR MGR

NSX-V VRLI
Management and Tenant ESXi Cluster

TENANT
PSC1 VM
CNTL-1 MN

CONSOLIDATED WORKLOAD DOMAIN

NSX-V TENANT VRLI
PSC2 VM
CNTL-2 WN1
Network
Security
VSAN

TENANT NSX-V TENANT VRLI

VM CNTL-3 VM WN2

TENANT TENANT TENANT

VM VM VM

TENANT TENANT TENANT

VM VM VM

6 or Less Nodes

Figure 3

Figure 9 depicts a single ESXi Cluster with six Nodes with all the Management VM’s as well as Tenant
VM’s. Single vCenter manages the environment and resource pools are used to provide isolation
between management and tenant workloads. The environment is managed from a single vCenter Server
and vSphere resource pools provide isolation between management and user workloads. Consolidate
Architecture can be converted to Standard Deployment Model by adding nodes and creating a new
Tenant Workload Domain followed by migration Tenant Workload VM’s from Management Domain to
New Tenant Workload Domain. This deployment model is not supported with VxRail.

Dell.com/certification 9
Stretched vSAN Cluster with VMware Cloud Foundations
vSAN Stretched Cluster with VMware Cloud Foundation works similarly as a normal vSAN Stretched
Cluster. With VMware Cloud Foundation we get the benefit of automated deployment of the Stretched
Workload Domain and using NSX-V or NSX-T. With VCF on VxRail we recommend physical and logical
configuration based on the VVD standardized architecture design which includes a Management Cluster
(4 node cluster minimum) and Compute/Edge Workload Domain Cluster (4 node cluster minimum).

NSX-V Integration with VMware Cloud Foundation

With VMware Cloud Foundations we get the automated deployment of NSX-V. Whenever we deploy a
new Workload Domain a new NSX-Manager Instance gets deployed on Management Workload Domain
which is integrated with Tenant vCenter. 3-NSX Controllers get deployed on the Tenant Workload
Domain. This way, as soon as the Workload Domain is deployed NSX-V is ready to use and VMware
Validate Design (VVD) guidelines can be followed for deploying additional components.

NSX-T Integration with VMware Cloud Foundation

NSX-T components are deployed with the First NSX-T workload domain. It deploys three NSX-T Manager
Virtual Machines in Management Workload Domain. Any further NSX-T workload domains created will
share NSX-T Management resources already deployed. This way, as soon as the Workload Domain is
deployed NSX-T is ready to use and VMware Validate Design (VVD) guidelines can be followed for
deploying additional components.

VMware Validated Design with VMware Cloud Foundation

VMware Validated Design provides blueprints and guidelines for deploying a Software Defined Data
Center. Similarly, VMware has VMware Validated Design for VMware Cloud Foundation which can be
followed while deploying VCF on VxRail. Following VVD guidelines will reduce design flaws and increase
efficiency. VMware regularly updates the guidelines. We can refer to
https://docs.vmware.com/en/VMware-Validated-Design/index.html for more details.

VMware Enterprise PKS on VMware NSX-T Workload Domain

Kubernetes is a leading container orchestration system. Starting with VCF 3.8.1, we have the ability now
to automate the deployment of VMware Enterprise PKS through SDDC Manager. Steps for automated
deployment of VMware Enterprise PKS:

• We deploy the NSX-T workload domain using SDDC and configure NSX-T.
• Prepare the IP addresses and forward/reserve DNS records for the PKS API, Pivotal Operations
Manager and the Harbor Registry (optional)
• Generate the certificates and private keys from a trusted certificate authority that include the
fully qualified domain names for each PKS management component.
• Prepare the NSX-T Tier-0 router, node and pod IP blocks and a floating IP pool for Kubernetes
cluster resources.
• Prepare the network settings and resources for the availability zones. This includes the network
CIDR, gateway, reserved IP ranges, target logical switch and vSphere cluster for management
and Kubernetes availability zone.

2020 Dell Technologies Proven Professional Knowledge Sharing 10

Once we have all these done, we use SDDC Manager for automated PKS Components deployment.
Figure 4 represents a sample environment. VMware Enterprise PKS Control Plane VM’s are
automatically deployed.

Figure 4

VMware Cloud Foundation Deployment Overview

• Validate the Compatibility of ESXi Nodes as per VMware VCF Compatbility Matrix.
• Rack , Stack and Cable the nodes.
Prepare Phsyical Infrastructure For Deployment • Upgarde the Hardware Firmware Components .
• Prepare ESXi Hosts.
• Deploy Cloud Builder VM

• Download the VCF Cloud Builder Workbook.

Deploy VCF Cloud Builder VM. • Fill up the Environment Details in the workbook .

• Upload the Cloud Builder Workbook

Upload the Workbook • Validate the Environment

Begin the Automated Bringup of Management • Using SDDC manager start Automated Bring Up of Management work load domain.
Workload Domain • This will install SDDC Manager , vCenter , PSC, NSX Manager , NSX Controllers and Log Insight

Initiate Automated Compute Workload Domain • Once the Management Workload is deployed we can deploy Compute/Horizon/PKS workload Domain as and when required .
Deployment as per requirements
• If the deployed Tenant Workload Domain is NSX-V Compute Workload domain then SDDC Manager will deploy vCenter in
NSX-V Workload Domain Management Workload Domain , Deploy NSX Manager in Management workload Domain , Registerr NSX Manager with Tenant
vCenter and Deploy 3 NSX Controller in Tennat Workload Domain.
• If we want to deploy Horizon Workload Domain then first we deploy a normal VI compute worklad domain and then use that
compute workload domain to deploy SDDC VDI's. Horizon View Management VM's like Composers , Connection Servers, App
NSX-V Horizon Workload Domain Volume Managers, User Environment Manager , Unified Access Gateway , Load Balancers are deployed on Management Workload
Domain. Also . There are some pre-requistes before deploying that should be met. We should refer VCF Administratior Guide for
more details on this.

• The first NSX-T Workload Domain that we will deploy will lead to deployment of 3 NSX Managers on the Management Workload
NSX-T VI Workload Domain Domain. Any future NSX-T workload domains will also use the existing NSX-Managers.

VMware PKS Workload Domain • We can deploy VMware PKS Workload Domain with NSX-T as defined in previous section .

Dell.com/certification 11
Why VMware Cloud Foundation on VxRAIL
• Co-engineered with VMware and only VMware Co-engineered Solution available.
• Single point of contact for support-related issues.
• Proactive Dial Home Support.
• Automated end-to-end lifecycle management including automated VMware Components
Upgrades along with VxRail Node Hardware firmware upgrades.
• Standardized on VMware SDDC Architecture.
• Tighter VxRail Integration within vSphere Client enables management of VxRail nodes from a
single pane of glass.
• VxRail Software Bundles alignment with VMware Cloud Foundations Release.
• Dell Smart Fabric Integration for automation of Switch Configuration.
• Industry-Leading Compute Infrastructure.

2020 Dell Technologies Proven Professional Knowledge Sharing 12

Case Study
Engagement Background
African Airlines is the flag carrier of Ethiopia. Over the past seventy years, it has become one of the
continent's leading carriers, unrivalled in Africa for efficiency and operational success, turning profits for
nearly every year of its existence.

Operating at the forefront of technology, the airline has also become one of Ethiopia's major industries
and a veritable institution in Africa. It commands a lion's share of the pan-African network including
multiple daily east-west flights across the continent and currently serves 81 international destinations
operating the newest and youngest fleets.

African Airlines’ current vSphere infrastructure is based on IBM hardware acquired in 2014. The vSphere
environment based on vSphere 6.5 has reached maximum capacity. The perception from African Airlines
staff is that the current support is unreliable and costly. Currently, there is no disaster recovery plan in
place to deal with a complete site failure. As a main driver for African Airlines’ IT transformation strategy
an agile approach that includes automation and self-service shall be adopted.

African Airlines engaged Dell Technologies to design and implement a new infrastructure to replace the
current solution. The solution needs to address the current resource constraints and lacking site failover
capabilities.

Dell.com/certification 13
Conceptual Design
Initial design workshops with the project sponsors and all involved stakeholders of African Airlines
defined their business goals along with their business requirements. In these meetings requirements,
constraints, assumptions and risks (shown in the sections below) were also identified that influenced the
design.

Business Requirements
Item Design Quality Description

R01 Scalability Accommodate existing virtual machines in the new environment.

R02 Scalability Account for 40% growth in the number of workloads based on
the existing metrics.

R03 Availability Design must provide a highly available solution to sustain

operations during system failures.

R04 Availability Design must provide failover capabilities to sustain two server
failures at the same time.

R05 Availability Service Level Agreement of 99.95% uptime per annum.

R06 Availability Solution to span two data center in the main location and a
second site within the same metro area.

R07 Recoverability Recovery Time Objective (RTO) of 1 hours.

R08 Recoverability Recovery Point Objective (RPO) of 30 minutes.

R09 Recoverability Site failover capabilities for all virtual machines.

R10 Manageability Management workload should be at least physically separated

from production workload.

R11 Manageability Automated upgrade and expansion processes must be in place.

R12 Security Adopt software defined networking approach.

R13 Performance, The design must provide configuration parameters for best
Availability performance and resiliency.

R14 Manageability Design must maintain simplicity where possible to allow existing
operations teams to manage the new environments.

R15 Scalability Solution should be able to be expanded at a later stage.

2020 Dell Technologies Proven Professional Knowledge Sharing 14

Design Assumptions
Item Type Description Validated

A01 Infrastructure DNS infrastructure is available including forward and reverse

lookup for all vSphere components.

A02 Infrastructure NTP server is available for time synchronization of all vSphere
components.

A03 Infrastructure Sufficient power, cooling and rack space is available to support
the solution.

A04 Security African Airlines is responsible for creating all required

certificates for the solution.

A05 Operations African Airlines has sufficient knowledge of how to manage the
vSphere environment.

A06 Network Connectivity between sites in place.

A07 Network African Airlines is responsible for creating all VLANs that will be
used in this solution at the core level as well as providing routing
capabilities if required.

A08 Implementation African Airlines is responsible for assigning internal resources to

assist during the implementation.

A09 Network IP address space is available for all components in this solution.

Dell.com/certification 15
Design Constraints
Item Design Quality Description

C01 Infrastructure Reuse of current data center facilities.

C02 General During normal operations no production workload should run in the DR
site. Secondary site should only be utilized in a DR case.

C03 Infrastructure Reuse of existing networking equipment.

C04 Recoverability Avamar backup solution must be leveraged.

C05 Manageability Integration into existing monitoring solutions - IBM Tivoli Monitoring for
application monitoring, VMware vRealize Operations Manager for
vSphere monitoring.

Resources are pooled together while adhering to the requirements and constraints of this design to run
the workload. The environment is separated into management and production. Replication between
sites is facilitating a complete site failover. All production VM on the current vSphere environment will
be migrated onto the target solution as part of a migration project. Figure 5 illustrates the conceptual
design of the environment.

Region A Region B

Availability Zone 1 Availability Zone 2

Management Pod Production Pod Management Pod Production Pod

VM VM VM VM VM VM VM VM VM VM VM VM

Compute Compute
Replication
Storage Storage

Network Network

Security Security

Figure 5

2020 Dell Technologies Proven Professional Knowledge Sharing 16

Overview - Logical Design
Decisions taken to translate African Airlines’ conceptional design to a logical design.

Design Decision ID DD01

Design Decision Adopt VMware Cloud Foundation.
Design Justification VMware Cloud Foundation provides an engineered framework
to deploy a SDDC in a rapid way. VCF allows for automated
lifecycle management of the full stack.

Design Decision ID DD02

Design Decision Use two availability zones in region 1 and region 2 as disaster
recovery.
Design Justification To achieve availability and recoverability requirements the
solution will be deployed across multiple availability zones and
regions.
Region 1:
Availability Zone 1: Stretched Cluster
Availability Zone 2: Stretched Cluster
Region 2:
Individual Cluster

Design Decision ID DD03

Design Decision Create separate Workload Domain for management and
production.
Design Justification Separate production from production workload to comply with
African Airlines business requirements and provides further
scalability.

Design Decision ID DD04

Design Decision Use Hyperconverged nodes as standard building block.
Design Justification Using vSAN ReadyNodes ensures seamless compatibility with
vSAN at deployment, support and future expandability.

Design Decision ID DD05

Design Decision Use NSX as software defined networking solution.
Design Justification Allow for transparent cross-region VM mobility, communication
and micro segmentation.

Design Decision ID DD06

Design Decision Replication between regions will be used for disaster recovery.
Design Justification Allow for seamless disaster recovery to comply with African
Airlines’ requirement.

Dell.com/certification 17
Figure 6 outlines the logical design.
Region A Region B

Availability Zone 1 Availability Zone 2

External and
Inter-site Connectivity

Stretched Management Stretched Management

Workload Domain Workload Domain

VM VM VM VM VM VM

Hypervisor Hypervisor
IP Based Replication
Hyperconverged Nodes Hyperconverged Nodes

Physical and Software Defined Networking Physical and Software Defined Networking

Stretched Production Stretched Production

Workload Domain Workload Domain

VM VM VM VM VM VM

Hypervisor Hypervisor
IP Based Replication
Hyperconverged Nodes Hyperconverged Nodes

Physical and Software Defined Networking Physical and Software Defined Networking

Figure 6

Overview - Physical Design

Sizing Rationale
The design needs to accommodate the workload running in the current virtual environment plus 40%
growth. An assessment has been carried out to collect the required performance metrics to size the new
solution accordingly. Data has been extracted from vCenter and RVTools over a period of one week in
the current environment.

The following table summarizes the results of the analysis. It details the overall requirements for the
vSphere environment to support the workloads of the existing workload. All values have been rounded
up to ensure sufficient resources are available during peak times.

Metric Value

VMs Number of VMs 196

CPU Total number of vCPUs 631

Average CPU usage in MHz 217085

Peak CPU usage in MHz 409797

Memory Allocated Memory in GB 1138

Average Memory usage in GB 509

2020 Dell Technologies Proven Professional Knowledge Sharing 18

 Storage Allocated Virtual Disk space in TB 61

Average IOs per second 10192

Average throughput in Mbps 1176

Peak throughput in Mbps 1872

Network Average throughput in Mbps 982

Peak throughput in Mbps 1421

A total of 40% growth needs to be accounted for in the environment based on the current resource
usage. The table below lists the resource requirements including growth.

Metric Value

VMs Number of VMs 274

CPU Number of vCPUs 883

Average CPU usage in MHz 303919

Peak CPU usage in MHz 552715

Memory Allocated Memory in GB 1898

Average Memory usage in GB 712

Storage Allocated Virtual Disk space in TB 85

Average IOs per second 14269

Average throughput in Mbps 1647

Peak throughput in Mbps 2621

Network Average throughput in Mbps 1375

Peak throughput in Mbps 1989

Dell.com/certification 19
Host Design
Design Decision ID DD07
Design Decision Use Dell AF-4 Ready Node as building block for the SDDC.
Design Justification To account for the resource requirements, the following VSAN-
ready nodes have been selected, while aligning to VMware
recommendation in terms of sizing.
Region A:
Management Workload Domain
AZ1: 4x AF-4 Nodes
AZ2: 4x AF-4 Nodes
Production Workload Domain
AZ1: 12x AF-4 Nodes
AZ2: 12x AF-4 Nodes

Region B:
Management Workload Domain
4x AF-4 Nodes
Production Workload Domain
12x AF-4 Nodes

The table below outlines the individual node configuration per Workload Domain.

Management Workload Domain

Cores per Node 24

Memory per Node 128 GB
Capacity drives per Node 3x 3.84 TB SSD
Raw Cache per Node 900 GB SSD
Network Cards 4x 10 Gbit/s

Production Workload Domain

Cores per Node 24

Memory per Node 256 GB
Capacity drives per Node 3x 3.84 TB SSD
Raw Cache per Node 900 GB SSD
Network Cards 4x 10 Gbit/s

2020 Dell Technologies Proven Professional Knowledge Sharing 20

Figure 7 provides a visual overview of the host distribution.
Region A Region B

Availability Zone 1 Availability Zone 2

ToR ToR ToR ToR ToR ToR

Switch Switch Switch Switch Switch Switch

Stretched Stretched
Management
management cluster management cluster
Cluster
Availability Zone 1 Availability Zone 2
(4 ESXi hosts)
(4 ESXi hosts) (4 ESXi hosts)

Stretched compute cluster Stretched compute cluster

Availability Zone 1 Availability Zone 2 Compute cluster
(12 ESXi hosts) (12 ESXi hosts) (12 ESXi hosts)

Figure 7

Network Design
Design Decision ID DD08
Design Decision Reuse existing Nexus 9236C switches.
Design Justification Customer has an existing investment in Nexus 9236C switches
that shall be reused. Each rack will include a pair of TOR
switches.
Each Hyperconverged node will be connected to each switch for
redundancy.

Design Decision ID DD09

Design Decision NSX-v will be deployed in a multi-site configuration.
Design Justification Allow for cross-site migrations and disaster recovery without the
need to change any IP addressing.

Figure 8 provides an overview of the NSX architecture.

Dell.com/certification 21
 Management Cluster - Region A Management Cluster - Region B

Shared vCenter Single Sign-on Domain

Management Management
vCenter Server vCenter Server

NSX Manager NSX Manager Pairing NSX Manager

Primary Secondary

Production Production
vCenter Server vCenter Server
NSX Manager Pairing NSX Manager
NSX Manager
Primary Secondary

NSX Controller 1

NSX Controller 2

Universal
NSX Controller 3 Controller
Cluster

Production Cluster - Region A Production Cluster - Region B

NSX Controller 1

NSX Controller 2

Universal
NSX Controller 3 Controller
Cluster

Figure 8

Disaster Recovery & Business Continuity

Design Decision ID DD10
Design Decision RecoverPoint for Virtual Machines will be used for Disaster
Recovery.
Design Justification Provide automated failover runbooks between both regions.

Figure 9 highlights the RecoverPoint for VM architecture.

2020 Dell Technologies Proven Professional Knowledge Sharing 22

 VMDK

Figure 9

The table below provides an overview of business continuity and disaster recovery.

Component Business Continuity Disaster Recovery

Compute Redundant physical components, Hardware replacement, restore from
vSphere HA config backup

Networking Redundant physical components, Hardware replacement, Host profiles,

NSX ICMP restore from config backup

Storage Redundant physical components Data replication, Hardware replacement,

and VSAN policies restore from VSAN

Data Center Redundant power, cooling and Site failover via RP4VM
backup generators

Dell.com/certification 23
Troubleshooting
As with VMware Cloud Foundations on VxRail we have virtualized Compute, Storage and Network along
with Automation Layer. With so many abstraction layers troubleshooting becomes difficult. As per our
experience we follow the approach below for troubleshooting issues and found it to be very helpful.

Define the •Define the problem statement.

•Document the time stamps.
•Document the problem symptoms.

Problem •Document the recent environment changes.

•Gather the logs.

Identify the •Navigate through the various component logs and correlate with time stamps.
•Validate if there is any relation with regard to recent environment changes.
•Navigate existing Knowledgebase .

root cause •Narrow down possible causes.

•Validate the root cause and document.

Implement •Implement the solution.

•Monitor the environment.
the Solution
As VMware Cloud Foundations comprises many components we will highlight important log files that
help us to troubleshoot issues related to environment. For individual components like vCenter, NSX, etc.
we can refer to their respective log files.

Location Purpose Component

Logs\marvin It’s the main log for VxRail VxRail Manager VM
Tomcat\logs\marvin.log Operations.
/opt/vmware/bringup/logs/vcf- It’s the main log for SDDC SDDC Manager VM/Cloud
bringup.log Manager Workload Domain Foundation Builder VM
/opt/vmware/bringup/logs/vcf- operations.
bringup-debug.log

/var/log/vmware/vcf/bringup/vcf- Log to be referred to for SDDC Manager VM

bringup-debug.log bringing up tasks.
/var/log/vmware/vcf/lcm Logs related to VMware Cloud SDDC Manager VM
Foundation Component
Upgrades.

VMware Cloud Foundations includes very helpful Supportablity and Serviceability (SoS) CLI utility. This
utility can be used for log bundle collection, detailed health checks and other maintenance related tasks.
Location of the utility is /opt/vmware/sddc-support/sos on SDDC Manager VM.

2020 Dell Technologies Proven Professional Knowledge Sharing 24

Examples of the utility

./sos –log-dir /tmp –domain-name MGMT –sddc-manager-logs

Above command will collect SDDC Manager logs along with VxRail Manager Logs.

./sos - - health-check

Above command will perform detailed health check of the environment.

sos –help

Above command will display all the available command options.

We also have an important command to see the password of the environment. Below command needs
to be run from SDDC Manager.

/use/bin/lookup_password

References
https://docs.vmware.com/en/VMware-Cloud-Foundation/

Dell.com/certification 25
Dell Technologies believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” DELL TECHNOLOGIES MAKES NO
RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.

Use, copying and distribution of any Dell Technologies software described in this publication requires an
applicable software license.

Copyright © 2020 Dell Inc. or its subsidiaries. All Rights Reserved. Dell Technologies, Dell, EMC, Dell EMC
and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be
trademarks of their respective owners.

2020 Dell Technologies Proven Professional Knowledge Sharing 26