VMware Cloud Foundation

Deployment Guide
VMware Cloud Foundation 5.2
VMware Cloud Foundation Deployment Guide

You can find the most up-to-date technical documentation on the VMware by Broadcom website at:

https://docs.vmware.com/

VMware by Broadcom
3401 Hillview Ave.
Palo Alto, CA 94304
www.vmware.com

©
Copyright 2015-2024 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc.
and/or its subsidiaries. For more information, go to https://www.broadcom.com. All trademarks, trade
names, service marks, and logos referenced herein belong to their respective companies.

VMware by Broadcom 2
 Contents

About the VMware Cloud Foundation Deployment Guide 4

1 Preparing your Environment for VMware Cloud Foundation 6

2 Deploying VMware Cloud Foundation 7

Deploy VMware Cloud Builder Appliance 8
Prepare ESXi Hosts for VMware Cloud Foundation 10
Create a Custom ISO Image for ESXi 11
Create a Custom ESXi ISO Image Using VMware PowerCLI 11
Create a Custom ESXi ISO Image Using vSphere Lifecycle Manager 13
Install ESXi Interactively and Configure Hosts for VMware Cloud Foundation 13
Install ESXi on VMware Cloud Foundation Hosts Using the ISO 14
Configure the Network on VMware Cloud Foundation Hosts 15
Configure the Virtual Machine Network Port Group on VMware Cloud Foundation Hosts
16
Configure NTP on VMware Cloud Foundation Hosts 17
Regenerate the Self-Signed Certificate on All Hosts 18
Configure ESXi Hosts with Signed Certificates 18
Deploy the Management Domain Using VMware Cloud Builder 20
About the Deployment Parameter Workbook 21
Credentials Worksheet 22
Hosts and Networks Worksheet 24
Deploy Parameters Worksheet: Existing Infrastructure Details 28
Deployment Parameters Worksheet: License Keys 29
Deploy Parameters Worksheet: vSphere Infrastructure 30
Deploy Parameters Worksheet: VMware NSX 34
Deploy Parameters Worksheet: SDDC Manager 34
Deploy the Management Domain Using ESXi Hosts with External Certificates 35

3 Troubleshooting VMware Cloud Foundation Deployment 37

VMware Cloud Builder Log Files 37
Using the SoS Utility on VMware Cloud Builder 38

4 VMware Cloud Foundation Glossary 43

VMware by Broadcom 3
About the VMware Cloud Foundation
Deployment Guide

The VMware Cloud Foundation Deployment Guide provides information about installing VMware
ESXi™ software on VMware Cloud Foundation™ servers and deploying the management domain
using the VMware Cloud Builder appliance™. Starting with VMware Cloud Foundation 5.2, you
can also use the VCF Import Tool to convert an existing vSphere cluster into a management
domain.

Intended Audience
The VMware Cloud Foundation Deployment Guide is intended for data center cloud
administrators who deploy a VMware Cloud Foundation system in their organization's data
center. The information in this guide is written for experienced data center cloud administrators
who are familiar with:

n Concepts of virtualization and software-defined data centers

n Networking and concepts such as uplinks, NICs, and IP networks

n Hardware components such as top-of-rack (ToR) switches, inter-rack switches, servers with
direct attached storage, cables, and power supplies

n Methods for setting up physical racks in your data center

®
n Using the VMware vSphere Client™ to work with virtual machines

Related Publications
Getting Started with VMware Cloud Foundation document provides a high-level overview of the
VMware Cloud Foundation product

The Planning and Preparation Workbook provides detailed information about the software, tools,
and external services that are required for Cloud Foundation.

The VMware Cloud Foundation Administration Guide contains detailed information about how to
administer and operate a VMware Cloud Foundation system in your data center. It also contains
information about using the VCF Import Tool to convert an existing vSphere environment to the
VMware Cloud Foundation management domain.

Your VMware Cloud Foundation system includes various VMware software products and
components. You can find the documentation for those VMware software products at
docs.vmware.com.

VMware by Broadcom 4
VMware Cloud Foundation Deployment Guide

VMware Cloud Foundation Glossary

The VMware Cloud Foundation Glossary defines terms specific to VMware Cloud Foundation.

VMware by Broadcom 5
Preparing your Environment for
VMware Cloud Foundation 1
Before you start the automated deployment of the management domain using VMware Cloud
Builder, your environment must meet target prerequisites and be in a specific starting state.

Prepare the platform by deploying and configuring the necessary infrastructure components. For
detailed prerequisites, see the Planning and Preparation Workbook.

VMware by Broadcom 6
Deploying VMware Cloud
Foundation 2
You begin the VMware Cloud Foundation deployment process by deploying the VMware Cloud
Builder appliance. After imaging your servers, you download and complete the deployment
parameters workbook from the VMware Cloud Builder appliance to define your network
information, host details, and other required information. During the deployment process, this
workbook is uploaded to the VMware Cloud Builder appliance, where a JSON file is generated to
drive the bring-up process. The provided information is validated, and the automated phase of
the bring-up process begins.

You can perform bring-up with certificates generated by an external CA, in which case
ESXi certificates are not replaced with vCenter Server signed certificates. If you use external
certificates for ESXi hosts in the management domain, hosts added after bring-up must also
be added with external certificates. This feature is supported only through APIs. For more
information, see Deploy the Management Domain Using ESXi Hosts with External Certificates.

Prerequisites

You must prepare your environment for deploying VMware Cloud Foundation. See the Planning
and Preparation Workbook.

Procedure

1 Deploy VMware Cloud Builder Appliance

VMware Cloud Builder is a virtual appliance that is used to deploy and configure the first
cluster of the management domain and transfer inventory and control to SDDC Manager.
During the deployment process, the VMware Cloud Builder appliance validates network
information you provide in the deployment parameter workbook such as DNS, network
(VLANS, IPs, MTUs), and credentials.

2 Prepare ESXi Hosts for VMware Cloud Foundation

3 Deploy the Management Domain Using VMware Cloud Builder

The VMware Cloud Foundation deployment process is referred to as bring-up. You specify
deployment information specific to your environment such as networks, hosts, license keys,
and other information in the deployment parameter workbook and upload the file to the
VMware Cloud Builder appliance to initiate bring-up of the management domain.

VMware by Broadcom 7
VMware Cloud Foundation Deployment Guide

Deploy VMware Cloud Builder Appliance

VMware Cloud Builder is a virtual appliance that is used to deploy and configure the first cluster
of the management domain and transfer inventory and control to SDDC Manager. During the
deployment process, the VMware Cloud Builder appliance validates network information you
provide in the deployment parameter workbook such as DNS, network (VLANS, IPs, MTUs), and
credentials.

You must deploy the VMware Cloud Builder appliance on a suitable platform. This can be on a
laptop running VMware Workstation or VMware Fusion, or on an ESXi host. The VMware Cloud
Builder appliance must have network access to all hosts on the management network.

This procedure describes how to deploy the VMware Cloud Builder appliance directly to an ESXi
host.

Prerequisites

Before you deploy the VMware Cloud Builder appliance, verify that your environment fulfills the
requirements for this process.

Prerequisite Value

Environment n Verify that your environment is configured for

deployment of VMware Cloud Builder and the
management domain.
n Verify that you have available virtual infrastructure that
has access to the management network that will be
used by the management domain. You deploy VMware
Cloud Builder on that virtual infrastructure.

Resource Requirements n 4 CPUs

n 4 GB of Memory
n 279 GB of Storage
n 25.1 GB (thin provisioned)
n 253.8 GB (thick provisioned)

Installation Packages Verify that you download the OVA file(s) for VMware
Cloud Builder.

Network n Verify that the static IP address and FQDN for the
VMware Cloud Builder appliance are available.
n Verify that connectivity is in place from the VMware
Cloud Builder appliance and the management VLAN
used in the deployment.

To automate the deployment, the VMware Cloud Builder appliance must be on the same
management network as the hosts to be used. It must also be able to access all required external
services, such as DNS and NTP.

Procedure

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 In the navigation pane, select Host, and click Create/Register VM.

VMware by Broadcom 8
VMware Cloud Foundation Deployment Guide

3 On the Select creation type dialog box, select Deploy a virtual machine from an OVF or OVA
file and click Next.

4 On the Select OVF and VMDK files page, enter a name for the virtual machine, select the
VMware Cloud Builder.ova file, and click Next.

5 On the Select Storage page, select a datastore and click Next.

6 On the License agreements dialog box, click I agree and then click Next.

7 On the Select networks dialog box, enter the following values and click Next.

Setting Value

Network mappings your_portgroup

Disk provisioning Thin

Power on automatically Selected

8 On the Additional settings dialog box, expand Application, enter the following values, and
click Next.

Setting Details

Admin Username Accept the default admin user name, admin.

Admin Password/Admin Password The admin password must be a minimum of 15 characters and include at
confirm least one uppercase, one lowercase, one digit, and one special character.
Supported special characters:
@ ! # $ % ? ^

Note A password cannot be based on a dictionary word (for example,

VMware1!)

Root password/Root password The root password must be a minimum of 15 characters and include at
confirm least one uppercase, one lowercase, one digit, and one special character.
Supported special characters:
@ ! # $ % ? ^

Note A password cannot be based on a dictionary word (for example,

VMware1!)

Hostname Enter the hostname for the VMware Cloud Builder appliance.

Network 1 IP Address Enter the IP address for the VMware Cloud Builder appliance.

Network 1 Subnet Mask Enter the subnet mask for the VMware Cloud Builder appliance.

Default Gateway Enter the default gateway for the VMware Cloud Builder appliance.

DNS Servers Enter the IP address of the primary and secondary DNS servers (comma
separated). Do not specify more than two servers.

DNS Domain Name Enter the DNS domain name. For example, vsphere.local.

VMware by Broadcom 9
VMware Cloud Foundation Deployment Guide

Setting Details

DNS Domain Search Paths Enter the DNS domain search path(s). Use a comma if entering multiple
search paths. For example vsphere.local, sfo.vsphere.local.

NTP Servers Enter the NTP server(s). Use a comma if entering multiple NTP servers. NTP
servers can be entered using FQDNs or IP addresses.

9 On the Ready to complete page, review the virtual machine configuration and click Finish.

Note Make sure your passwords meet the requirements specified above before clicking
Finish or your deployment will not succeed.

10 After the VMware Cloud Builder appliance is deployed, SSH in to the VM with the admin
credentials provided in step 8.

11 Ensure that you can ping the ESXi hosts.

12 Verify that the VMware Cloud Builder appliance has access to the required external services,
such as DNS and NTP by performing forward and reverse DNS lookups for each host and the
specified NTP servers.

Prepare ESXi Hosts for VMware Cloud Foundation

Before you can begin the process of deploying VMware Cloud Foundation you must prepare the
ESXi hosts that will form the management domain.

Preparing the ESXi hosts involves installing the correct version of ESXi and performing some
basic configuration tasks.

For the supported ESXi version, see the Bill of Materials (BOM) section of the VMware Cloud
Foundation Release Notes.

Prerequisites

The management domain requires a minimum of four ESXi hosts.

To use vSAN Express Storage Architecture (ESA), your hosts must be ESA-compatible.

Tip See the vSAN ESA VCG for information about compatible hardware.

n Create a Custom ISO Image for ESXi

When your environment requires a custom ISO file for ESXi, you can create one using
VMware PowerCLI or vSphere Lifecycle Manager.

n Install ESXi Interactively and Configure Hosts for VMware Cloud Foundation
You can interactively install ESXi on all the hosts that will form the first cluster in the
management domain, then you configure the management network, DNS, and NTP services.
You can use the same process to add more hosts to the management domain later, or to
install and configure hosts for VI workload domains.

VMware by Broadcom 10
VMware Cloud Foundation Deployment Guide

n Regenerate the Self-Signed Certificate on All Hosts

Once you have configured the ESXi hosts' identity by providing a hostname you must
regenerate the self-signed certificate to ensure the correct common name is defined.

n Configure ESXi Hosts with Signed Certificates

If corporate policy requires that you use external CA-signed certificates instead of VMCA-
signed certificates for ESXi hosts, you can manually add external certificates to the hosts.

Create a Custom ISO Image for ESXi

When your environment requires a custom ISO file for ESXi, you can create one using VMware
PowerCLI or vSphere Lifecycle Manager.

You might need to create a custom ISO image for ESXi in the following situations:

n The ESXi version specified in the VMware Cloud Foundation BOM does not have an
associated ISO file on the Broadcom Support Portal. This can be the case for ESXi patch
releases.

n You need an async patch version of ESXi.

n You need a vendor-specific (OEM) ISO file.

Prerequisites

Download the zip files for the following:

n ESXi patch for the ESXi version specified in the VMware Cloud Foundation BOM or in the
list of supported async patches in KB 88287. You can download patches from the Broadcom
Support Portal.

Note If you are preparing hosts for a VI workload domain where the ESXi hosts have been
async patched to a later version of ESXi than the version listed in the BOM, the new hosts
must use the later version of ESXi.

n OEM add-on for ESXi from the Broadcom Support Portal. If the ESXi version specified in
the BOM is not available in the Select Version drop-down menu, contact your vendor to
determine which OEM add-on version to use.

Create a Custom ESXi ISO Image Using VMware PowerCLI

You can use VMware Power CLI to create a custom ISO.

Prerequisites

VMware PowerCLI 12.0 or later.

VMware by Broadcom 11
VMware Cloud Foundation Deployment Guide

Procedure

1 Gather the required information for the software spec that is used to create the custom ISO.

a In VMware PowerCLI, use the Get-DepotBaseImages cmdlet to get the base image version
from the zip file for the ESXi patch that you downloaded from the patches portal.

For example:

Get-DepotBaseImages “c:\temp\VMware-ESXi-7.0U1d-17551050-depot.zip”

b Use the Get-DepotAddons cmdlet to get the add-on name and version from the zip file
for the OEM add-on for ESXi that you downloaded from the Broadcom Support Portal. (if
applicable)

For example:

Get-DepotAddons “c:\temp\HPE-701.0.0.10.6.5.12-Jan2021-Synergy-Addon-depot.zip”

2 Create the software spec using the information you gathered in step 1.

The software spec is a JSON file that contains information about the ESXi version and vendor
add-on (if applicable). For example:

{
"add_on": {
"name": "HPE-Custom-Syn-AddOn",
"version": "701.0.0.10.6.5-12"
},
"base_image": {
"version": "7.0.1-0.30.17551050"
},
"components": null,
"hardware_support": null,
"solutions": null
}

3 In VMware PowerCLI, use the New-IsoImage cmdlet to generate a custom ISO.

For example:

New-IsoImage -SoftwareSpec “c:\temp\HPE-70U1d-custom.JSON” -Depots “c:\temp\VMware-

ESXi-7.0U1d-17551050-depot.zip” , “c:\temp\HPE-701.0.0.10.6.5.12-Jan2021-Synergy-Addon-
depot.zip” -Destination “c:\temp\HPE-70U1d-custom.iso”

Provide the path to the software spec you created in step 2.

The depot(s) include the path to the zip files for the supported ESXi version and vendor
add-on.
The destination include the path and file name for the custom ISO file.
For more information about the New-IsoImage cmdlet, see https://code.vmware.com/docs/
11794/cmdletreference//doc/New-IsoImage.html.

VMware by Broadcom 12
VMware Cloud Foundation Deployment Guide

Create a Custom ESXi ISO Image Using vSphere Lifecycle Manager

If you have access to a vCenter Server environment, you can use vSphere Lifecycle Manager to
create and export a custom ISO.

Prerequisites

Import the ESXi patch and vendor add-on (if applicable) zip files to the vSphere Lifecycle
Manager depot. See Import Updates to the vSphere Lifecycle Manager Depot.

Procedure

1 Log in to vCenter Server using the vSphere Client.

2 Create a new temporary cluster, selecting the Manage all hosts in the cluster with a single
image check box.

3 Select the ESXi version and vendor add-on (optional) and click OK.

4 Export the vSphere Lifecycle Manager image as an ISO.

See Export an Image.

5 Delete the temporary cluster.

Install ESXi Interactively and Configure Hosts for VMware Cloud

Foundation
You can interactively install ESXi on all the hosts that will form the first cluster in the management
domain, then you configure the management network, DNS, and NTP services. You can use the
same process to add more hosts to the management domain later, or to install and configure
hosts for VI workload domains.

ESXi 8.0 Update 3 and later support installing two data processing units (DPUs) for use with
VMware Cloud Foundation 5.2 or later.

You can utilize the two DPUs in Active/Standby mode to provide high availability. Such
configuration provides redundancy in the event one of the DPUs fails. In the high availability
configuration, both DPUs are assigned to the same NSX-backed vSphere Distributed Switch. For
example, DPU-1 is attached to vmnic0 and vmnic1 of the vSphere Distributed Switch and DPU-2 is
attached to vmnic2 and vmnic3 of the same vSphere Distributed Switch.

VMware by Broadcom 13
VMware Cloud Foundation Deployment Guide

You can also utilize the two DPUs as independent devices to increase offload capacity per ESXi
host. Each DPU is attached to a separate vSphere Distributed Switch and you have no failover
between DPUs in such configuration.

Prerequisites

n Download the ESXi ISO from the Broadcom Support Portal. For the supported ESXi versions,
see the Bill of Materials (BOM) section of the VMware Cloud Foundation Release Notes and
the list of supported async patches in KB 88287. If the required version of ESXi does not have
an ISO available on the Broadcom Support Portal, you can create one. See Create a Custom
ISO Image for ESXi.

Note If you are preparing hosts for a VI workload domain where the ESXi hosts have been
async patched to a later version of ESXi than the version listed in the BOM, the new hosts
must use the later version of ESXi.

n Make sure that you have a host machine for SDDC access. You use this host to connect to
the data center and perform configuration steps.

n Verify that you have the completed Planning and Preparation Workbook.

n Verify the Prerequisite Checklist sheet in the Planning and Preparation Workbook.

Procedure

1 Install ESXi on VMware Cloud Foundation Hosts Using the ISO

Install ESXi on all hosts in the first cluster in the management domain interactively. You can
use the same process to install ESXi on additional hosts for the management domain, or on
hosts for a VI workload domain.

2 Configure the Network on VMware Cloud Foundation Hosts

After the initial boot, use the ESXi Direct Console User Interface (DCUI) for host network
configuration and administrative access.

3 Configure the Virtual Machine Network Port Group on VMware Cloud Foundation Hosts
You perform configuration of the Virtual Machine Network port group for each ESXi host by
using the VMware Host Client.

4 Configure NTP on VMware Cloud Foundation Hosts

Complete the initial configuration of all ESXi hosts by configuring the NTP service to avoid
time synchronization issues in the SDDC.

Install ESXi on VMware Cloud Foundation Hosts Using the ISO

Install ESXi on all hosts in the first cluster in the management domain interactively. You can use
the same process to install ESXi on additional hosts for the management domain, or on hosts for
a VI workload domain.

Repeat this procedure for all hosts in the first cluster in the management domain.

VMware by Broadcom 14
VMware Cloud Foundation Deployment Guide

Procedure

1 Mount the ESXi ISO on the host and restart the machine.

2 Set the BIOS or UEFI to boot from the mounted ISO.

Note If your system has supported data processing units (DPUs), you can only use UEFI to
install and boot ESXi on the DPUs.

See your hardware vendor documentation for information on changing boot order.

3 On the welcome screen, press Enter to continue.

4 Accept the End User License Agreement by pressing Enter.

Starting with ESXi 8.0 Update 3, after the scanning for available devices completes, if your
system has DPUs, you see them automatically listed with their respective PCI slots. You no
longer select a slot. The DPU devices must be identical: same vendor, same hardware version
and same firmware

5 On the Select a Disk to Install or Upgrade screen, select the drive on which to install ESXi on
and press Enter.

6 Select the keyboard type for the host.

You can change the keyboard type after installation in the direct console.

7 Enter the root password for the host.

8 In the Confirm Install screen, if you have DPUs, you see each listed on a separate row. Press
F11 to confirm the start of the installation.

Starting with ESXi 8.0 Update 3, if your systems has DPUs, you see a single progress bar for
the ESXi and DPU installation, with dynamic updates to the label showing what stage of the
installer is being run.

9 On the Installation Complete screen, press Enter to reboot the host.

10 Set the first boot device to be the drive on which you installed ESXi.

11 Repeat this procedure for all remaining hosts.

Configure the Network on VMware Cloud Foundation Hosts

After the initial boot, use the ESXi Direct Console User Interface (DCUI) for host network
configuration and administrative access.

Perform the following tasks to configure the host network settings:

n Configure the network adapter (vmk0) and VLAN ID for the Management Network.

n Configure the IP address, subnet mask, gateway, DNS server, and FQDN for the ESXi host.

Repeat this procedure for all hosts that you are adding to the first cluster of the management
domain. Enter the respective values from the completed Planning and Preparation Workbook.

VMware by Broadcom 15
VMware Cloud Foundation Deployment Guide

Procedure

1 Open the DCUI of the ESXi host.

a Open a console window to the host.

b Press F2 to enter the DCUI.

c Log in by using the esxi_root_user_password.

2 Configure the network.

a Select Configure Management Network and press Enter.

b Select VLAN (Optional) and press Enter.

c Enter the VLAN ID for the Management Network and press Enter.

d Select IPv4 Configuration and press Enter.

e Select Set static IPv4 address and network configuration and press the Space bar.

f Enter the IPv4 Address, Subnet Mask and Default Gateway and press Enter.

g Select DNS Configuration and press Enter.

h Select Use the following DNS Server address and hostname and press the Space bar.

i Enter the Primary DNS Server, Alternate DNS Server and Hostname (FQDN) and press
Enter.

j Select Custom DNS Suffixes and press Enter.

k Ensure that there are no suffixes listed and press Enter.

3 Press Escape to exit and press Y to confirm the changes.

4 Repeat this procedure for all remaining hosts.

Configure the Virtual Machine Network Port Group on VMware Cloud

Foundation Hosts
You perform configuration of the Virtual Machine Network port group for each ESXi host by
using the VMware Host Client.

You configure the VLAN ID of the VM Network port group on the vSphere Standard Switch. This
configuration provides connectivity to the Management network to allow communication to the
vCenter Server Appliance during the automated deployment.

Repeat this procedure for all hosts in the first cluster of the management domain. Enter the
respective values from the completed Planning and Preparation Workbook.

Procedure

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 Click OK to join the Customer Experience Improvement Program.

VMware by Broadcom 16
VMware Cloud Foundation Deployment Guide

3 Configure a VLAN for the VM Network port group.

a In the navigation pane, click Networking.

b Click the Port groups tab, select the VM network port group, and click Edit Settings.

c On the Edit port group - VM network page, enter the Management Network VLAN ID,
and click Save.

4 Repeat this procedure for all remaining hosts.

Configure NTP on VMware Cloud Foundation Hosts

Complete the initial configuration of all ESXi hosts by configuring the NTP service to avoid time
synchronization issues in the SDDC.

Repeat this procedure for all hosts in the first cluster of the management domain. Enter the
respective values from the completed Planning and Preparation Workbook.

Procedure

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 Configure and start the NTP service.

a In the navigation pane, click Manage, and click the System tab.

b Click Time & date and click Edit NTP Settings.

c On the Edit NTP Settings page, select the Use Network Time Protocol (enable NTP
client) radio button, and change the NTP service startup policy to Start and stop with
host.

VMware by Broadcom 17
VMware Cloud Foundation Deployment Guide

d In the NTP servers text box, enter the NTP Server FQDN or IP Address, and click Save.

e To start the service, click Actions, select NTP service, and click Start.

3 Repeat this procedure for all remaining hosts.

Regenerate the Self-Signed Certificate on All Hosts

Once you have configured the ESXi hosts' identity by providing a hostname you must regenerate
the self-signed certificate to ensure the correct common name is defined.

During the installation of ESXi, the installer generates a self-signed certificate for each ESXi
host but the process is performed prior to the ESXi identity being configured. This means all
ESXi hosts have a common name in their self-signed certificate of localhost.localdomain. All
communication between VMware Cloud Builder and the ESXi hosts is performed securely over
HTTPS and as a result it validates the identify when making a connection by comparing the
common name of the certificate against the FQDN provided within the VMware Cloud Builder
configuration file.

To ensure that the connection attempts and validation does not fail, you must manually
regenerate the self-signed certificate after hostname has been configured.

Note VMware Cloud Foundation supports the use of signed certificates. If your organization's
security policy mandates that all ESXi hosts must be configured with a CA-signed certificate, see
Configure ESXi Hosts with Signed Certificates.

Procedure

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 In the Actions menu, click Services > Enable Secure Shell (SSH).

3 Log in to the ESXi host using an SSH client such as Putty.

4 Regenerate the self-signed certificate by executing the following command:

/sbin/generate-certificates

5 Restart the hostd and vpxa services by executing the following command:

/etc/init.d/hostd restart && /etc/init.d/vpxa restart && /etc/init.d/rhttpproxy restart

6 Log back in to the VMware Host Client and click Services > Disable Secure Shell (SSH) from
the Actions menu.

7 Repeat this procedure for all remaining hosts.

Configure ESXi Hosts with Signed Certificates

If corporate policy requires that you use external CA-signed certificates instead of VMCA-signed
certificates for ESXi hosts, you can manually add external certificates to the hosts.

VMware by Broadcom 18
VMware Cloud Foundation Deployment Guide

When you install ESXi software on a server to create an ESXi host, the host initially has an
autogenerated certificate. By default, when the host is added to a vCenter Server system
during bring-up of the management domain or other operations involving hosts (for example,
host commissioning, VI workload domain creation, and so on), the autogenerated certificate is
replaced with a certificate that is signed by the VMware Certificate Authority (VMCA).

When you use external certificates during bring-up, they are not replaced by VMCA-signed
certificates. Once you perform bring-up with external certificates for ESXi hosts, all future hosts
added to VMware Cloud Foundation must also use external certificates.

Prerequisites

External CA-signed certificate and key are available.

Procedure

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 In the navigation pane, click Manage and click the Services tab.

3 Select the TSM-SSH service and click Start if not started.

4 Log in to the ESXi Shell for the first host, either directly from the DCUI or from an SSH client,
as a user with administrator privileges.

5 In the directory /etc/vmware/ssl, rename the existing certificates using the following
commands:

mv rui.crt orig.rui.crt

mv rui.key orig.rui.key

6 Copy the external certificate and key that you want to use to /etc/vmware/ssl.

7 Rename the external certificate and key to rui.crt and rui.key.

VMware by Broadcom 19
VMware Cloud Foundation Deployment Guide

8 Restart the host management agents by running the following commands:

/etc/init.d/hostd restart

/etc/init.d/vpxa restart

9 In the VMware Host Client, select the TSM-SSH service for the ESXi host and click Stop.

10 Repeat for all the ESXi hosts that you are adding to VMware Cloud Foundation.

What to do next

See Deploy the Management Domain Using ESXi Hosts with External Certificates.

Deploy the Management Domain Using VMware Cloud

Builder
The VMware Cloud Foundation deployment process is referred to as bring-up. You specify
deployment information specific to your environment such as networks, hosts, license keys, and
other information in the deployment parameter workbook and upload the file to the VMware
Cloud Builder appliance to initiate bring-up of the management domain.

During bring-up, the management domain is created on the ESXi hosts specified in the
deployment parameter workbook. The VMware Cloud Foundation software components
are automatically deployed, configured, and licensed using the information provided. The
deployment parameter workbook can be reused to deploy multiple VMware Cloud Foundation
instances of the same version.

The following procedure describes how to perform bring-up of the management domain using
the deployment parameter workbook. You can also perform bring-up using a custom JSON
specification. See the VMware Cloud Foundation API Reference Guide for more information.

Note Starting with VMware Cloud Foundation 5.2, you can use the VCF Import Tool to convert
an existing vSphere environment to create the management domain. See Converting or Importing
Existing vSphere Environments into VMware Cloud Foundation.

Some use cases are only available using a custom JSON specification. For example, using custom
CA-signed certificates for ESXi hosts. See Deploy the Management Domain Using ESXi Hosts with
External Certificates.

Procedure

1 In a web browser, log in to the VMware Cloud Builder appliance administration interface:
https://Cloud_Builder_VM_FQDN.

2 Enter the admin credentials you provided when you deployed the VMware Cloud Builder
appliance and then click Log In.

3 On the End-User License Agreement page, select the I Agree to the End User License
Agreement check box and click Next.

VMware by Broadcom 20
VMware Cloud Foundation Deployment Guide

4 Select VMware Cloud Foundation and click Next.

5 Review and acknowledge the prerequisties and click Next.

If there are any gaps, ensure they are fixed before proceeding to avoid issues during the
bring-up process. You can download or print the prerequisite list for reference.

6 Download the deployment parameter workbook from the Broadcom Support portal and fill it
in with the required information.

See About the Deployment Parameter Workbook.

7 Click Next.

8 Click Select File, browse to the completed workbook, and click Open to upload the
workbook.

9 Click Next to begin validation of the uploaded file.

To access the bring-up log file, SSH to the VMware Cloud Builder appliance as admin and
open the /opt/vmware/bringup/logs/vcf-bringup-debug.log file.

If there is an error during the validation and the Next button is grayed out, you can either
make corrections to the environment or edit the deployment parameter workbook and
upload it again. Then click Retry to perform the validation again.

If any warnings are displayed and you want to proceed, click Acknowledge and then click
Next.

10 Click Deploy SDDC.

During the bring-up process, the vCenter Server, NSX, and SDDC Manager appliances are
deployed and the management domain is created. The status of the bring-up tasks is
displayed in the UI.

After bring-up is completed, a green bar is displayed indicating that bring-up was successful.
A link to the SDDC Manager UI is also displayed. If there are errors during bring-up, see
Chapter 3 Troubleshooting VMware Cloud Foundation Deployment.

11 Click Download to download a detailed deployment report. This report includes information
on assigned IP addresses and networks that were configured in your environment.

12 After bring-up is completed, click Finish.

13 Click Launch SDDC Manager.

14 Power off the VMware Cloud Builder appliance.

About the Deployment Parameter Workbook

The deployment parameter workbook contains worksheets categorizing the information required
for deploying VMware Cloud Foundation. The information provided is used to create the
management domain using the VMware Cloud Builder appliance.

VMware by Broadcom 21
VMware Cloud Foundation Deployment Guide

Before you begin filling in the deployment parameter workbook, download the workbook from
the Broadcom Support portal.

The fields in yellow contain sample values that you should replace with the information for your
environment. If a cell turns red, the required information is missing, or validation input has failed.

Important The deployment parameter workbook is not able to fully validate all inputs due
to formula limitations of Microsoft Excel. Some validation issues may not be reported until you
upload the deployment parameter workbook to the VMware Cloud Builder appliance.

Note Do not copy and paste content between cells in the deployment parameter workbook,
since this may cause issues.

The Introduction worksheet in the deployment parameter workbook contains an overview of

the workbook and guidance on how to complete it. For information about the prerequisites for
deploying the management domain, see the Planning and Preparation Workbook.

Credentials Worksheet
The Credentials worksheet details the accounts and initial passwords for the VMware Cloud
Foundation components. You must provide input for each yellow box. A red cell may indicate
that validations on the password length has failed.

Input Required
Update the Default Password field for each user (including the automation user in the last row).
Passwords can be different per user or common across multiple users. The tables below provide
details on password requirements.

Table 2-1. Password Complexity

Password Requirements

ESXi Host root account This is the password which you configured on the hosts during ESXi
installation.

Default Single-Sign on domain 1 Length 8-20 characters

administrator user 2 Must include:
n mix of upper-case and lower-case letters
n a number
n a special character, such as @ ! # $ % ^ or ?
3 Must not include * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

vCenter Server virtual appliance root 1 Length 8-20 characters

account 2 Must include:
n mix of upper-case and lower-case letters
n a number
n a special character, such as @ ! # $ % ^ or ?
3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

VMware by Broadcom 22
VMware Cloud Foundation Deployment Guide

Table 2-1. Password Complexity (continued)

Password Requirements

NSX virtual appliance root account 1 Length 12-127 characters

2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
n at least five different characters
3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

NSX user interface and default CLI admin 1 Length 12-127 characters
account 2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
n at least five different characters
3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

NSX audit CLI account 1 Legnth 12-127 characters

2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
n at least five different characters
3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

SDDC Manager appliance root account 1 Minimum length 15 characters

2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
3 Must not include:
n *{}[]()/\'"`~,;:.<>
n A dictionary word (for example, VMware1!)

VMware by Broadcom 23
VMware Cloud Foundation Deployment Guide

Table 2-1. Password Complexity (continued)

Password Requirements

SDDC Manager super user (vcf) 1 Minimum length 15 characters

2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
3 Must not include:
n *{}[]()/\'"`~,;:.<>
n A dictionary word (for example, VMware1!)

SDDC Manager local account (admin@local) 1 Length 12-127 characters

2 Must include:
n mix of uppercase and lowercase letters
n a number
n a special character, such as @ ! # $ % ^ or ?
3 Must not include: * { } [ ] ( ) / \ ' " ` ~ , ; : . < >

Hosts and Networks Worksheet

The Hosts and Networks worksheet specifies the details for all networks and hosts. This
information is configured on the appropriate VMware Cloud Foundation components.

Management Domain Networks

This section covers the VLANs, gateways, MTU, and expected IP ranges and subnet mask for
each network you have configured on the Top of Rack switches in your environment.

With VMware Cloud Foundation 5.1 and later, you have the ability to create separate distibuted
port groups for management VM (for example, vCenter Server and NSX Manager) traffic and
ESXi host management traffic.

n If you enter information for the VM Management Network, VMware Cloud Foundation creates
a distibuted port group for the VM Management Network using the information you provide.

n If you do not enter information for the VM Management Network, VMware Cloud Foundation
still creates a distibuted port group for VM Management Network, but uses the Management
Network information (gateway, VLAN, MTU).

Network Type VLAN Portgroup Name CIDR Notation Gateway MTU

VM Management Enter the VLAN Enter a Enter the CIDR Enter the Enter MTU for
Network ID. portgroup name. notation for the gateway IP for the network.
network. network. The MTU can
Management
be between 1500
Network
and 9000.

VMware by Broadcom 24
VMware Cloud Foundation Deployment Guide

Network Type VLAN Portgroup Name CIDR Notation Gateway MTU

vMotion Network The VLAN ID can

be between 0
vSAN Network and 4094.

Note The VLAN

ID for Uplink 1
and Uplink 2
Networks must
be unique and
not used by any
other network
type.

Virtual Networking
The deployment parameter workbook provides three vSphere Distributed Switch profiles that
allow you to perform bring-up of hosts with two or four pNICs and to create up to two vSphere
Distributed Switches for isolating VMkernel traffic. The information that you are required to
provide depends on the profile that you select.

Note You can use the VMware Cloud Foundation API to perform bring-up with other
combinations of vSphere Distributed Switches and pNICs that are not available using the vSphere
Distributed Switch profiles.

vSphere Distributed Switch Profile Description

Profile 1 n One vSphere Distributed Switch (vDS): Traffic for

Management, vMotion, vSAN, and Host Overlay
networks using specified pNICs.
n Two or four physical NICs (pNICs)

Profile 2 n Two vSphere Distributed Switches (vDS)

n Four physical NICs (pNICs)
n Primary vDS: Traffic for Management, vMotion, and
Host Overlay networks using specified pNICs.
n Secondary vDS: Traffic for vSAN network using
specified pNICs.

Profile 3 n Two vSphere Distributed Switches (vDS)

n Four physical NICs (pNICs)
n Primary vDS: Traffic for Management, vMotion, and
vSAN networks using specified pNICs.
n Secondary vDS: Traffic for Host Overlay network using
specified pNICs.

After you select a vSphere Distributed Switch Profile, enter the required information for that
profile.

VMware by Broadcom 25
VMware Cloud Foundation Deployment Guide

Primary vSphere Distributed Switch - Name Enter a name for the primary vSphere Distributed Switch
(vDS). You can modify the portgroup names of the
management domain networks to make it clear which vDS
each network uses.

Primary vSphere Distributed Switch - pNICs Select the physical NICs to assign to the primary vDS.

Primary vSphere Distributed Switch - MTU Size Enter the MTU size for the primary vDS. Default value is
9000.

Primary vSphere Distributed Switch - Transport Zone Type Select Overlay or VLAN.

Secondary vSphere Distributed Switch - Name Enter a name for the secondary vSphere Distributed
Switch (vDS). You can modify the portgroup names of the
management domain networks to make it clear which vDS
each network uses.

Note If you are not creating a secondary vDS, enter n/a.

Secondary vSphere Distributed Switch - Transport Zone Select Overlay or VLAN.

Type

Secondary vSphere Distributed Switch - pNICs Select the physical NICs to assign to the secondary vDS.

Secondary vSphere Distributed Switch - MTU Size Enter the MTU size for the secondary vDS. Default value is
9000.

Management Domain ESXi Hosts

Specify the IP addresses of the ESXi hosts for the management domain. In a standard
deployment, only four hosts are required in the management domain. VMware Cloud Foundation
can also be deployed with a consolidated architecture. In a consolidated deployment, all
workloads are deployed in the management domain instead of to separate workload domains. As
such, additional hosts may be required to provide the capacity needed. In this section, only enter
values for the number of hosts desired in the management domain.

Host Name IP Address

Enter host names for each of the four ESXi hosts. Enter IP Address for each of the four ESXi hosts.

Inclusion Ranges
Specify IP inclusion ranges for the vSAN and vMotion networks of the management domain.
IP addresses from the specified range are automatically assigned to hosts. Ensure that the IP
ranges include sufficient IP addresses for the initial deployment. The number of IP addresses
must be at least equal to the number of hosts deployed as part of VMware Cloud Foundation.

As an example, if you specify the range start value as 192.168.1.1 and end as 192.168.1.20, a total of
20 IP addresses would be used.

Do not use special IP addresses, such as the network or broadcast address.

IPs for the vMotion range must be part of the VLAN configured with the vMotion portgroup. IPs
for the vSAN range must be part of the VLAN configured for the vSAN portgroup. All IPs within
the range must be available for use or IP conflicts will occur. It is a good practice to validate this
prior to starting a deployment.

VMware by Broadcom 26
VMware Cloud Foundation Deployment Guide

Table 2-2. Input Required

Network Start IP End IP

vMotion Enter start of IP address range for Enter end of IP address range.
vMotion network.

VSAN Enter start of IP address range for Enter end of IP address range.
vMotion network.

ESXi Host Security Thumbprints

If you want bring-up to validate the SSH fingerprint and SSL thumbprints of the ESXi hosts before
connecting to them to reduce the chance of Man In The Middle (MiTM) attack, select Yes in the
Validate Thumbprints field.

If you set Validate Thumbprints to Yes, follow the steps below.

1 In a web browser, log in to the ESXi host using the VMware Host Client.

2 In the navigation pane, click Manage and click the Services tab.

3 Select the TSM-SSH service and click Start if not started.

4 Connect to the VMware Cloud Builder appliance using an SSH client such as Putty.

5 Enter the admin credentials you provided when you deployed the VMware Cloud Builder
appliance.

6 Retrieve the SSH fingerprint by entering the following command replacing hostname with the
FQDN of your host:

ssh-keygen -lf <(ssh-keyscan hostname 2>/dev/null)

7 Retrieve the SSL thumbprint by entering the following command replacing hostname with the
FQDN of your host:

openssl s_client -connect hostname:443 < /dev/null 2> /dev/null | openssl x509 -sha256
-fingerprint -noout -in /dev/stdin

8 In the VMware Host Client, select the TSM-SSH service for the ESXi host and click Stop.

9 Repeat for each ESXi host and then enter the information in the deployment parameter
workbook.

NSX Host Overlay Network

By default, VMware Cloud Foundation uses DHCP for the management domain Host Overlay
Network TEPs. For this option, a DHCP server must be configured on the NSX host overlay (Host
TEP) VLAN of the management domain. When NSX creates TEPs for the VI workload domain,
they are assigned IP addresses from the DHCP server.

VMware by Broadcom 27
VMware Cloud Foundation Deployment Guide

For the management domain and VI workload domains with uniform L2 clusters, you can choose
to use static IP addresses instead. Make sure the IP range includes enough IP addresses for the
number of hosts that will use the static IP Pool. The number of IP addresses required depends
on the number of pNICs on the ESXi hosts that are used for the vSphere Distributed Switch that
handles host overlay networking. For example, a host with four pNICs that uses two pNICs for
host overlay traffic requires two IP addresses in the static IP pool..

Table 2-3. DHCP Settings

Parameter Value

VLAN ID Enter a VLAN ID for the NSX host overlay network. The
VLAN ID can be between 0 and 4094.

Configure NSX Host Overlay Using a Static IP Pool Select No to use DHCP.

Table 2-4. Static IP Pool Settings

Parameter Value

VLAN ID Enter a VLAN ID for the NSX host overlay network. The
VLAN ID can be between 0 and 4094.

Configure NSX Host Overlay Using a Static IP Pool Select Yes to use a static IP pool.

Pool Description Enter a description for the static IP pool.

Pool Name Enter a name for the static IP pool.

CIDR Notation Enter CIDR notation for the NSX Host Overlay network.

Gateway Enter the gateway IP address for the NSX Host Overlay
network.

NSX Host Overlay Start IP Enter the first IP address to include in the static IP pool.

NSX Host Overlay End IP Enter the last IP address to include in the static IP pool.

Deploy Parameters Worksheet: Existing Infrastructure Details

Your existing DNS infrastructure is used to provide forward and reverse name resolution for all
hosts and VMs in the VMware Cloud Foundation SDDC. External NTP sources are also utilized to
synchronize the time between the software components.

Table 2-5. Infrastructure

Parameter Value

DNS Server #1 Enter IP address of first DNS server.

DNS Server #2 Enter IP address of second DNS server.

Note If you have only one DNS server, enter n/a in this cell.

VMware by Broadcom 28
VMware Cloud Foundation Deployment Guide

Table 2-5. Infrastructure (continued)

Parameter Value

NTP Server #1 Enter IP address or FQDN of first NTP server.

NTP Server #2 Enter IP address or FQDN of second NTP server.

Note If you have only one NTP server, enter n/a in this cell.

Table 2-6. DNS Zone

Parameter Value

DNS Zone Name Enter root domain name for your SDDC management components.

Note VMware Cloud Foundation expects all components to be part of the same DNS zone.

Table 2-7. Customer Experience Improvement Program

Parameter Value

Enable Customer Select an option to activate or deactivate CEIP across vSphere, NSX, and vSAN during bring-
Experience up.
Improvement
Program (“CEIP”)

Table 2-8. Enable FIPS Security Mode on SDDC Manager

Parameter Value

Enable FIPS Security Select an option to activate or deactivate FIPS security mode during bring-up. VMware
Mode on SDDC Cloud Foundation supports Federal Information Processing Standard (FIPS) 140-2. FIPS
Manager 140-2 is a U.S. and Canadian government standard that specifies security requirements
for cryptographic modules. When you enable FIPS compliance, VMware Cloud Foundation
enables FIPS cipher suites and components are deployed with FIPS enabled.
To learn more about support for FIPS 140-2 in VMware products, see https://
www.vmware.com/security/certifications/fips.html.

Note This option is only available for new VMware Cloud Foundation installations and the
setting you apply during bring-up will be used for future upgrades. You cannot change the
FIPS security mode setting after bring-up.

Deployment Parameters Worksheet: License Keys

Provide licensing information for VMware Cloud Foundation.

1 Select Yes or No for License Now.

VMware by Broadcom 29
VMware Cloud Foundation Deployment Guide

2 If you select Yes, in the License Keys section, update the red fields with your license keys.
Ensure the license key matches the product listed in each row and that the license key is valid
for the version of the product listed in the VMware Cloud Foundation BOM. The license key
audit during bring-up validates both the format and validity of the key.

Note When using the per-TiB license for vSAN, be aware that VI workload domain
components like vCenter and NSX Manager will also consume the TiB capacity.

3 If you select No, the VMware Cloud Foundation components are deployed in evaluation
mode.

Important After bring-up, you must switch to licensed mode by adding component license
keys in the SDDC Manager UI or adding and assigning a solution license key in the vSphere
Client. See the VMware Cloud Foundation Administration Guide for information about adding
component license keys in the SDDC Manager UI. See Managing vSphere Licenses for more
information about adding and applying a solution license key for VMware ESXi and vCenter
Server in the vSphere Client. If you are using a solution license key, you must also add a
separate VMware vSAN license key for vSAN clusters. See Configure License Settings for a
vSAN Cluster.

Deploy Parameters Worksheet: vSphere Infrastructure

The vSphere infrastructure section of the Deploy Parameters Worksheet details how you want to
configure the vCenter Server and its related objects.

This section of the deployment parameter workbook contains sample configuration information,
but you can update them with names that meet your naming standards.

Note All host names entries within the deployment parameter workbook expect the short name.
VMware Cloud Builder takes the host name and the DNS zone provided to calculate the FQDN
value and performs validation prior to starting the deployment. The specified host names and IP
addresses must be resolvable using the DNS servers provided, both forward (hostname to IP)
and reverse (IP to hostname), otherwise the bring-up process will fail.

VMware by Broadcom 30
VMware Cloud Foundation Deployment Guide

Table 2-9. vCenter Server

Parameter Host Name IP Address

vCenter Server Enter a host name for the vCenter Enter the IP address for the
Server. vCenter Server that is part of the
management VLAN.

Note This is the same VLAN and

IP address space where the ESXi
management VMKernels reside.

vCenter Server Appliance Size This parameter defines the size of the vCenter Server to be deployed. Default
(Default Small) size is Small. Additional options are: Tiny, Medium, Large, and X-large. See
Hardware Requirements for the vCenter Server Appliance.

vCenter Server Appliance Storage The amount of storage depends on the vCenter Server appliance size. See
Size Storage Requirements for the vCenter Server Appliance.

Table 2-10. vCenter Datacenter and Cluster

Parameter Value

Datacenter Name Enter a name for the management datacenter.

Cluster Name Enter a name for the management cluster.

Enable vLCM Cluster Image Select Yes to use vSphere Lifecycle Manager images for
managing the lifecycle of ESXi hosts in the primary cluster
of management domain. VMware Cloud Builder extracts
a vSphere Lifecycle Manager image from the first ESXi
host and applies that image to all the hosts in the cluster.
The vSphere Lifecycle Manager image is also imported
into SDDC Manager (available at Lifecycle Management >
Image Management.

Note vSAN Express Storage Architecture (ESA) requires

vSphere Lifecycle Manager images.

Select No to use vSphere Lifecycle Manager baselines for

managing the lifecycle of ESXi hosts in the primary cluster
of management domain.

Cluster EVC Setting To enable EVC on the management cluster, select the
CPU chipset that should be applied to enhance vMotion
compatability.

Note If you don't want to enable EVC, enter n/a in this

cell.

Select the architecture model you plan to use. If you choose Consolidated, specify the names for
the vSphere resource pools. You do not need to specify resource pool names if you are using the
standard architecture model. See Introducing VMware Cloud Foundation for more information
about these architecture models.

VMware by Broadcom 31
VMware Cloud Foundation Deployment Guide

Table 2-11. vSphere Resource Pools

Parameter Value

Resource Pool SDDC Management Specify the vSphere resource pool name for management
VMs.

Resource Pool User Edge Specify the vSphere resource pool name for user
deployed NSX VMs in a consolidated architecture.

Resource Pool User VM Specify the vSphere resource pool name for user
deployed workload VMs.

Note Resource pools are created with Normal CPU and memory shares.

Table 2-12. vSphere Datastore

Parameter Value

vSAN Datastore Name Enter vSAN datastore name for your management
components.

Enable vSAN Deduplication and Compression Select Yes to turn on Dedupe and Compression
capabilities of vSAN.

Note This option is only available with vSAN OSA. If

you enable vSAN ESA, deduplication and compression
settings can be specified in the vSAN storage policies
using the vSphere Client.

VMware by Broadcom 32
VMware Cloud Foundation Deployment Guide

Table 2-12. vSphere Datastore (continued)

Parameter Value

Enable vSAN-ESA Select Yes to use vSAN Express Storage Architecture

(ESA) for the first cluster in the management domain.
After bringup, you can create additional clusters (vSAN
ESA or vSAN OSA) in the management domain.

Note vSAN ESA requires the use of vLCM images and is

not supported with vLCM baselines.

vSAN ESA is designed for high-performance NVMe based

TLC flash devices and high performance networks. Each
host that contributes storage contains a single storage
pool of four or more flash devices. Each flash device
provides caching and capacity to the cluster.
Select No to use vSAN Original Storage Architecture
(OSA) for the first cluster in the management domain.
After bringup, you can create additional clusters (vSAN
ESA or vSAN OSA) in the management domain, but you
can create vSAN ESA clusters only if the management
domain is using vLCM images.
For an overview of the differences between vSAN OSA
and vSAN ESA, see Building a vSAN Cluster in the
vSphere documentation..

Path to HCL JSON File vSAN ESA requires a current version of the vSAN
HCL JSON file to ensure that your ESXi hosts are ESA-
compatible.
If the VMware Cloud Builder appliance is not able to
connect to the internet (either directly or through a proxy
server), download the latest vSAN HCL JSON file from
https://partnerweb.vmware.com/service/vsan/all.json and
copy it to the VMware Cloud Builder appliance.
Enter to path to the vSAN HCL JSON file on the VMware
Cloud Builder appliance. For example: /opt/vmware/
bringup/tmp/all.json

If the VMware Cloud Builder appliance does not have direct internet access, you can configure a
proxy server to download the vSAN HCL JSON. A recent version of the HCL JSON file is required
for vSAN ESA.

Table 2-13. Proxy Server Configuration

Parameter Value

Proxy Server Configuration Select Yes to configure a proxy server.

Proxy Server Enter the proxy server FQDN or IP address.

Proxy Port Enter the proxy server port.

Proxy Username

Proxy Password

VMware by Broadcom 33
VMware Cloud Foundation Deployment Guide

Table 2-13. Proxy Server Configuration (continued)

Parameter Value

Proxy Transfer Protocol

HTTPs Proxy Certificate (PEM Encoded)

Deploy Parameters Worksheet: VMware NSX

The NSX section of the Deploy Parameters Worksheet specifies the details you want to use for
deploying VMware NSX components.

Table 2-14. NSX Management Cluster

Parameter Value

NSX Management Cluster VIP Enter the host name and IP address for the NSX Manager
VIP.
The host name can match your naming standards but
must be registered in DNS with both forward and reverse
resolution matching the specified IP.

Note This is the same VLAN and IP address space where

the vCenter and ESXi management VMKernels reside.

NSX Virtual Appliance Node #1 Enter the host name and IP address for the first node in
the NSX Manager cluster.

NSX Virtual Appliance Node #2 Enter the host name and IP address for the second node
in the NSX Manager cluster.

NSX Virtual Appliance Node #3 Enter the host name and IP address for the third node in
the NSX Manager cluster.

NSX Virtual Appliance Size Select the size for the NSX Manager virtual appliances.
The default is medium.

Deploy Parameters Worksheet: SDDC Manager

The SDDC Manager section of the Deploy Parameters Worksheet specifies the details for
deploying SDDC Manager.

Table 2-15. SDDC Manager

Parameter Value

SDDC Manager Hostname Enter a host name for the SDDC Manager VM.

SDDC Manager IP Address Enter an IP address for the SDDC Manager VM.

Network Pool Name Enter the network pool name for the management domain
network pool.

Cloud Foundation Management Domain Name Enter a name for the management domain. This name will
appear in Inventory > Workload Domains in the SDDC
Manager UI.

VMware by Broadcom 34
VMware Cloud Foundation Deployment Guide

Deploy the Management Domain Using ESXi Hosts with External

Certificates
VMware Cloud Foundation supports vCenter Server's Custom Certificate Authority mode during
bring-up using the VMware Cloud Foundation API. Use this mode if you want to use only external
certificates that are signed by a third-party or enterprise CA. In this mode, you are responsible
for managing the certificates. You cannot refresh and renew external certificates from the SDDC
Manager or vSphere Client.

To use external ESXi certificates, you must create a custom JSON file for bring-up. You cannot
use the deployment parameter workbook.

Deploying the management domain with external ESXi certificates enables Custom Certificate
Authority mode, so all future hosts that you add to a workload domain (management or VI) must
also use external ESXi certificates.

Prerequisites

See Configure ESXi Hosts with Signed Certificates.

Procedure

1 Create a JSON file populated with the bring-up information for your environment.

You can see a sample JSON specification in the VMware Cloud Foundation API Reference
Guide.

2 Update the securitySpec section, choosing Custom for the esxiCertsMode and entering your
signing CA chain for certChain.

For example:

"securitySpec" : {
"esxiCertsMode" : "Custom",
"rootCaCerts" : [ {
"alias" : "Rainpole-CA",
"certChain" : [ "-----BEGIN CERTIFICATE-----
MIIDczCCAlugAwIBAgIQI9xwbTkI9J5GhMffcP5CHDANBgkqhkiG9w0BAQsFADBM
MRIwEAYKCZImiZPyLGQBGRYCaW8xGDAWBgoJkiaJk/IsZAEZFghyYWlucG9sZTEc
MBoGA1UEAxMTcmFpbnBvbGUtZGMwMXJwbC1DQTAeFw0yMDAzMzAxNDQ2MTNaFw0y
NTAzMzAxNDU2MTNaMEwxEjAQBgoJkiaJk/IsZAEZFgJpbzEYMBYGCgmSJomT8ixk
ARkWCHJhaW5wb2xlMRwwGgYDVQQDExNyYWlucG9sZS1kYzAxcnBsLUNBMIIBIjAN
BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzpwkz7aPlQcfevcCelHc9DPswHkd
kjY96Vh3GvYlesaVEcy/q/BOvvh3KgLMLy8r7cy2cNPO3FANKOfqVdVx3ghfEUyL
g61W9BskAlwryzJRmjhOJJVqvB8CWjy+eCp7MejHGdEud6WdEvK8CaBcPngEg0KM
eLRNLGe8OCw8yY4GTrjU+H7PYQZtyD0kxxy5f48ueaDXat4ENRGcAuHEfCoMGfaR
bDue1OO4diHd900bCym5ggBNX0jhRudNULXPTayZl2ksImV0+QkaVeptQImXfCgb
kgnHQJ5CxK26up7fB5eAsmGLAsJLBnHuM7P9xvV09EvWjFCgLX/oBBDYTQIDAQAB
o1EwTzALBgNVHQ8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU7oOq
QBK8yg8mHnAfb+u6/GO0ZUcwEAYJKwYBBAGCNxUBBAMCAQAwDQYJKoZIhvcNAQEL
BQADggEBALYxZGj4vWjFDN1atOUsBx2jrmxbExgMAyRpNlSc2aj+7vzxHxUW5VbX
x9nc/BfkTiCK6c7Y9VYb+mgjb8z0kNv58sT4ar1yIl1n63VOCoyyLcaFB8HyEJpD
wUhz4RNPoSijZMpm+M5EuSLfWlhEJo7N8sLqHgvvk1dFpbK8fIHbPS5KJwJibbPe

VMware by Broadcom 35
VMware Cloud Foundation Deployment Guide

w9UuNRdcxN9hFWKBC0SvfgX+1CJxVdvgfi65rSHPuWinJzrXXdH999DfpDESRzwH
0pqE3GtMCt1Nqalp2QJFdahbT+kxj7QWHTjUylSENDHjdln7a8WH8RGxvEy/97YZ
+crXmxvQ/bAgHk9vcRERbRjfyIs7v88=
-----END CERTIFICATE-----" ] } ] }

3 Follow the steps outlined in the VMware Cloud Foundation API Reference Guide to deploy
the management domain.

VMware by Broadcom 36
Troubleshooting VMware Cloud
Foundation Deployment 3
During the deployment stage of VMware Cloud Foundation you can use log files and the
Supportability and Serviceability (SoS) Tool to help with troubleshooting.

Read the following topics next:

n VMware Cloud Builder Log Files

n Using the SoS Utility on VMware Cloud Builder

VMware Cloud Builder Log Files

VMware Cloud Builder contains various log files for different components of the system.

VMware Cloud Builder has a number of components which are used during the bring-up process,
each component generates a log file which can be used for the purpose of troubleshooting. The
components and their purpose are:

n JsonGenerator: Used to convert the deployment parameter workbook into the required
configuration file (JSON) that is used by the Bringup Validation Service and Bringup Service.

n Bringup Service: Used to perform the validation of the configuration file (JSON), the ESXi
hosts and infrastructure where VMware Cloud Foundation will be deployed, and to perform
the deployment and configuration of the management domain components and the first
cluster.

n Supportability and Serviceability (SoS) Utility: A command line utility for troubleshooting
deployment issues.

The following table describes the log file locations:

Component Log Name Location

JsonGenerator jsongenerator-timestamp /var/log/vmware/vcf/sddc-support/

Bringup Service vcf-bringup.log /var/log/vmware/vcf/bringup/

vcf-bringup-debug.log /var/log/vmware/vcf/bringup/

rest-api-debug.log /var/log/vmware/vcf/bringup/

SoS Utility sos.log /var/log/vmware/vcf/sddc-support/

sos-timestamp/

VMware by Broadcom 37
VMware Cloud Foundation Deployment Guide

Using the SoS Utility on VMware Cloud Builder

You can run the Supportability and Serviceability (SoS) Utility on the VMware Cloud Builder
appliance to generate a support bundle, which you can use to help debug a failed bring-up of
VMware Cloud Foundation.

Note After a successful bring-up, you should only run the SoS Utility on the SDDC Manager
appliance. See Supportability and Serviceability (SoS) Tool in the VMware Cloud Foundation
Administration Guide.

The SoS Utility is not a debug tool, but it does provide health check operations that can facilitate
debugging a failed deployment.

To run the SoS Utility in VMware Cloud Builder, SSH in to the VMware Cloud Builder appliance
using the admin administrative account, then enter su to switch to the root user, and navigate to
the /opt/vmware/sddc-support directory and type ./sos followed by the options required for
your desired operation.

./sos --option-1 --option-2 ... --option-n

SoS Utility Help Options

Use these options to see information about the SoS tool itself.

Option Description

--help Provides a summary of the available SoS tool options

-h

--version Provides the SoS tool's version number.

-v

SoS Utility Generic Options

These are generic options for the SoS Utility.

Option Description

--configure-sftp Configures SFTP for logs.

--debug-mode Runs the SoS tool in debug mode.

--force Allows SoS operations from theVMware Cloud Builder appliance after bring-
up.

Note In most cases, you should not use this option. Once bring-up is
complete, you can run the SoS Utility directly from the SDDC Manager
appliance.

--history Displays the last twenty SoS operations performed.

--log-dir LOGDIR Specifies the directory to store the logs.

VMware by Broadcom 38
VMware Cloud Foundation Deployment Guide

Option Description

--log-folder LOGFOLDER Specifies the name of the log directory.

--setup-json SETUP_JSON Custom setup-json file for log collection.

SoS prepares the inventory automatically based on the environment where
it is running. If you want to collect logs for a pre-defined set of components,
you can create a setup.json file and pass the file as input to SoS. A sample
JSON file is available on the VMware Cloud Builder in the /opt/vmware/
sddc-support/ directory.

--skip-known-host-check Skips the specified check for SSL thumbprint for host in the known host.

--zip Creates a zipped tar file for the output.

SoS Utility Log File Options

Option Description

--api-logs Collects output from APIs.

--cloud-builder-logs Collects Cloud Builder logs.

--esx-logs Collects logs from the ESXi hosts only.

Logs are collected from each ESXi host available in the deployment.

--no-clean-old-logs Use this option to prevent the tool from removing any output from a
previous collection run.
By default, before writing the output to the directory, the tool deletes
the prior run's output files that might be present. If you want to retain
the older output files, specify this option.

--no-health-check Skips the health check executed as part of log collection.

--nsx-logs Collects logs from the NSX Manager instances only.

--rvc-logs Collects logs from the Ruby vSphere Console (RVC) only. RVC is an
interface for ESXi and vCenter.

Note If the Bash shell is not enabled in vCenter, RVC log collection will
be skipped .

Note RVC logs are not collected by default with ./sos log collection.

--sddc-manager-logs Collects logs from the SDDC Manager only.

--test Collects test logs by verifying the files.

--vc-logs Collects logs from the vCenter Server instances only.

Logs are collected from each vCenter server available in the
deployment.

--vm-screenshots Collects screen shots from all VMs.

VMware by Broadcom 39
VMware Cloud Foundation Deployment Guide

SoS Utility JSON Generator Options

The JSON generator options within the SoS Utility provide a method to execute the creation of
the JSON file from a completed deployment parameter workbook. To run the JSON generator,
you must provide, as a minimum, a path to the deployment parameter workbook and the design
type using the following syntax:

./sos --jsongenerator --jsongenerator-input JSONGENERATORINPUT --jsongenerator-design

JSONGENERATORDESIGN

Option Description

--jsongenerator Invokes the JSON generator utility.

--jsongenerator-input Specify the path to the input file to be used by the JSON generator utility.
JSONGENERATORINPUT For example: /tmp/vcf-ems-deployment-parameter.xlsx.

--jsongenerator-design Use vcf-ems for VMware Cloud Foundation.

JSONGENERATORDESIGN

--jsongenerator-supress Supress confirmation to force cleanup directory. (optional)

--jsongenerator-logs Set the directory to be used for logs. (optional)

JSONGENERATORLOGS

SoS Utility Health Check Options

The SoS Utility can be used to perform health checks on various components or services,
including connectivity, compute, and storage.

Note The health check options are primarily designed to run on the SDDC Manager appliance.
Running them on the VMware Cloud Builder appliance requires the --force parameter, which
instructs the SoS Utility to identify the SDDC Manager appliance deployed by VMware Cloud
Builder during the bring-up process, and then execute the health check remotely. For example:

./sos --health-check --force

Option Description

--certificate-health Verifies that the component certificates are valid (within the expiry
date).

--connectivity-health Performs a connectivity health check to inspect whether the different

components of the system such as the ESXi hosts, vCenter Servers, NSX
Manager VMs, and SDDC Manager VM can be pinged.

--compute-health Performs a compute health check.

--general-health Verifies ESXi entries across all sources, checks the Postgres DB
operational status for hosts, checks ESXi for error dumps, and gets NSX
Manager and cluster status.

--get-host-ips Returns server information.

VMware by Broadcom 40
VMware Cloud Foundation Deployment Guide

Option Description

--health-check Performs all available health checks.

--ntp-health Verifies whether the time on the components is synchronized with the
NTP server in the VMware Cloud Builder appliance.

--services-health Performs a services health check to confirm whether services are

running

--run-vsan-checks Runs proactive vSAN tests to verify the ability to create VMs within the
vSAN disks.

Sample Output
The following text is a sample output from an --ntp-health operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --ntp-health --skip-known-host --force

Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed
and SDDC Manager is available. Please expe ct failures with SoS operations.
Health Check : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681
Health Check log : /var/log/vmware/vcf/sddc-support/healthcheck-2020-02-11-23-03-53-24681/
sos.log
SDDC Manager : sddc-manager.vrack.vsphere.local
NTP : GREEN
+-----+-----------------------------------------+------------+-------+
| SL# | Area | Title | State |
+-----+-----------------------------------------+------------+-------+
| 1 | ESXi : esxi-1.vrack.vsphere.local | ESX Time | GREEN |
| 2 | ESXi : esxi-2.vrack.vsphere.local | ESX Time | GREEN |
| 3 | ESXi : esxi-3.vrack.vsphere.local | ESX Time | GREEN |
| 4 | ESXi : esxi-4.vrack.vsphere.local | ESX Time | GREEN |
| 5 | vCenter : vcenter-1.vrack.vsphere.local | NTP Status | GREEN |
+-----+-----------------------------------------+------------+-------+

Legend:

GREEN - No attention required, health status is NORMAL

YELLOW - May require attention, health status is WARNING
RED - Requires immediate attention, health status is CRITICAL

Health Check completed successfully for : [NTP-CHECK]

The following text is sample output from a --vm-screenshots log collection operation.

root@cloud-builder [ /opt/vmware/sddc-support ]# ./sos --vm-screenshots

--skip-known-host --force
Welcome to Supportability and Serviceability(SoS) utility!

User passed --force flag, Running SOS from Cloud Builder VM, although Bringup is completed
and SDDC Manager is available. Please expect failures with SoS operations.
Logs : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013

VMware by Broadcom 41
VMware Cloud Foundation Deployment Guide

Log file : /var/log/vmware/vcf/sddc-support/sos-2018-08-24-10-50-20-8013/sos.log

Log Collection completed successfully for : [VMS_SCREENSHOT]

VMware by Broadcom 42
VMware Cloud Foundation
Glossary 4
In VMware Cloud Foundation, you perform specific operations and use unique constructs for
automated SDDC deployment and maintenance.

Term Description

availability zone A collection of infrastructure components. Each availability zone is isolated from the
other availability zones to prevent the propagation of failure or outage across the data
center. In VMware Cloud Foundation, you implement availability of workloads across
availability zones by using vSAN stretched clusters.

Application virtual networks Virtual networks backed by overlay or VLAN NSX segments using the encapsulation
(AVNs) protocol of VMware NSX. An AVN uses a single IP address space to span across data
centers.

bring-up Deployment and initial configuration of a VMware Cloud Foundation system. During
the bring-up process, the management domain is created and the VMware Cloud
Foundation software stack is deployed on the management domain.

commission a host Adding a host to VMware Cloud Foundation inventory. The host becomes unassigned.

dirty host A host that has been removed from a cluster in a workload domain. A dirty host cannot
be assigned to another workload domain until it is decommissioned, re-imaged, and
commissioned again.

decommission a host Removing an unassigned host from the VMware Cloud Foundation inventory. SDDC
Manager does not manage decommissioned hosts.

NSX Edge cluster A logical grouping of NSX Edge nodes. These nodes run on a vSphere cluster, and
provide north-south and east-west routing and network services for the management
or VI workload domain.

free pool Hosts in the VMware Cloud Foundation inventory that are not assigned to a workload
domain.

host A server that is imaged with the ESXi software.

install bundle Contains software for VI workload domains and VMware Aria Suite Lifecycle. You
can use an install bundle to deploy later versions of the software components in a
new VI workload domain than the versions in the Bill of Materials for VMware Cloud
Foundation.

inventory Logical and physical entities managed by VMware Cloud Foundation.

Kubernetes - Workload With Kubernetes - Workload Management, you can deploy and operate the compute,
Management networking, and storage infrastructure for vSphere IaaS Control Plane workloads. A
vSphere IaaS Control Plane workload is an application with containers running inside
vSphere pods, regular VMs, or Tanzu Kubernetes clusters.

VMware by Broadcom 43
VMware Cloud Foundation Deployment Guide

Term Description

Lifecycle Manager (LCM) Automates patching and upgrading of the software stack.

management domain One or more vSphere clusters of physical hosts that contain the management
component VMs, such as vCenter Server, NSX Manager cluster, management NSX
Edge cluster, SDDC Manager, and so on. The management domain supports only vSAN
storage.

network pool Automatically assigns static IP addresses to vSAN and vMotion VMkernel ports so that
you don't need to enter IP addresses manually when creating a VI workload domain or
adding a host or cluster to a workload domain.

update bundle Contains software to update the VMware Cloud Foundation components in your
management or VI workload domain.

principal storage Required for each vSphere cluster, containing the data of the virtual machines in the
cluster. For the management domain, only vSAN principal storage is supported. For a
VI workload domain, you set the principal storage when creating the domain or when
adding a cluster to the domain. You cannot change the principal storage later. See also
supplemental storage.

SDDC Manager A software component that provisions, manages, and monitors the logical and physical
resources of a VMware Cloud Foundation system. SDDC Manager provides the user
interface for managing VMware Cloud Foundation, CLI-based administrator tools, and
an API for further automation.

server A bare-metal server in a physical rack. After imaging, it is referred to as a host.

supplemental storage Extends the capacity of the workload domain for hosting more virtual machines or
storing supporting data, such as backups. You can add or remove supplemental
storage to clusters in the management or VI workload domain at any time.

unassigned host A host in the free pool that does not belong to a workload domain.

vSphere Lifecycle Manager A vCenter Server service, which is integrated with VMware Cloud Foundation, that
(vLCM) enables centralized and simplified life cycle management of ESXi hosts.

virtual infrastructure (VI) One or more vSphere clusters that contain customer workloads. VMware Cloud
workload domain Foundation scales and manages the life cycle of each VI workload domain
independently. The vCenter Server instance and NSX Manager cluster for a VI workload
domain are physically located in the management domain, while the NSX edge nodes -
on the VI workload domain.

vSphere Lifecycle Manager A grouping of multiple bulletins. You can attach a baseline to an ESXi host and check
baseline the compliance of the host against the associated baseline. According to the type
of content, baselines are patch baselines, extension baselines, and upgrade baselines.
SDDC Manager creates the required baseline and baseline group for updating a cluster
in a workload domain.

VMware by Broadcom 44
VMware Cloud Foundation Deployment Guide

Term Description

vSphere Lifecycle Manager A precise description of the software, components, vendor add-ons, and firmware to
image run on an ESXi host. You set up a single image and apply it to all hosts in a cluster, thus
ensuring cluster-wide host image homogeneity.

workload domain A policy-based resource container with specific availability and performance attributes
that combines vSphere, storage (vSAN, NFS, VMFS on FC, or vVols) and networking
(VMware NSX) into a single consumable entity. A workload domain can be created,
expanded, and deleted as part of the SDDC life cycle operations. It can contain clusters
of physical hosts with a corresponding vCenter Server instance to manage them.
VMware Cloud Foundation supports two types of workload domains - the management
domain and one or more VI workload domains.

VMware by Broadcom 45