Configure the First Leaf Switch

Pair
(Dell Networking SmartFabric Services Deployment with VxRail 7.0.400)
Nov 2022

1
 Table Of Contents

Configure the First Leaf Switch Pair

Cabling
Change the default passwords
Configure leaf switch OOB management interfaces
Enable SmartFabric
Connect to SmartFabric Services UI
Update fabric and switch names
Create networks for first VxRail cluster
Create server interface profiles for the first VxRail cluster
Configure L2 uplinks to the external network
Configure L3 routed uplinks to the external network
Additional configuration steps for L3 uplinks
Configure a jump host port

Configure the First Leaf Switch Pair

Configure the First Leaf Switch Pair

Cabling
Change the default passwords
Configure leaf switch OOB management interfaces
Enable SmartFabric
Connect to SmartFabric Services UI
Update fabric and switch names
Create networks for first VxRail cluster
Create server interface profiles for the first VxRail cluster
Configure L2 uplinks to the external network
Configure L3 routed uplinks to the external network
Additional configuration steps for L3 uplinks
Configure a jump host port

Cabling

Cable the switches and VxRail nodes for the first cluster, as shown in the figure below,
and power on all devices.

Figure 11. VxRail node and leaf switch connections

For connection details, see Production network connection details . Also, make OOB
management connections, as shown in OOB management network connection details .

Change the default passwords

For security, change the passwords of the built-in accounts - admin and linuxadmin - on
each switch. The default passwords of these two accounts are admin and linuxadmin
respectively. You can use the admin account to log in to the OS10 CLI, the Linux shell,
and the SFS UI. You can use the linuxadmin account to log in to the Linux shell.

To change the admin password:

1. Connect to the switch console.

2. Log in as admin.
3. Run the following commands:

OS10# configure terminal

Configure the First Leaf Switch Pair

 OS10(config)# username admin password new-password role sysadmin
OS10(config)# exit
OS10# write memory

To change the linuxadmin password:

1. Connect to the switch console.

2. Log in as admin.
3. Run the following commands:

OS10# configure terminal

OS10(config)# system-user linuxadmin password new-password

OS10(config)# exit
OS10# write memory

Repeat the above steps on the remaining switches.

For additional security best practices, see the Dell SmartFabric OS10 and SmartFabric
Services Security Best Practices Guide.

Configure leaf switch OOB management interfaces

An IP address is configured on the OOB management interface of each switch. This

interface is used to access the SFS web UI, and it is also used as the VLT backup link.
Also, it enables console access using SSH as an option to the serial console.

If routing is used on the OOB management network, a management route is also

configured.

Note: Configure a unique OOB management IP address on each switch. The IP

addresses shown are examples only. Use IP addresses suitable for your environment.
The management route should not be 0.0.0.0/0; it may interfere with the data network’s
default route. Use a specific destination prefix, as shown.

Run the following commands on each switch:

OS10# configure terminal

OS10(config)# interface mgmt 1/1/1
OS10(conf-if-ma-1/1/1)# no ip address dhcp
OS10(conf-if-ma-1/1/1)# ip address 100.67.76.30/24
OS10(conf-if-ma-1/1/1)# no shutdown
OS10(conf-if-ma-1/1/1)# exit
OS10(config)# management route 100.67.0.0/16 100.67.76.254

Configure the First Leaf Switch Pair

OS10(config)# end
OS10# write memory

Other global settings may also be configured here, such as ip name-server and ntp
server if used by the switch. These settings may also be configured later in the SFS UI
but are not required for the deployment example in this guide. The hostname of the
switch may be configured at the CLI or in the SFS UI. In this guide, the SFS UI is used.

Enable SmartFabric

Figure 12. First pair of leaf switches in SmartFabric mode

CAUTION: The following commands delete the existing switch configuration. Switch
management settings such as management IP address, management route, hostname,
NTP server, and IP name server are retained.

Ensure the physical VLTi connections are made between leaf pairs before proceeding.

Note: This example uses the two QSFP28 2x 100 Gb DD ports, interfaces 1/1/49-1/1/52,
for the VLTi connections on each S5248F-ON leaf.

To put the first pair of leaf switches in SmartFabric mode and configure them as VLT
peers, run the following commands on each switch:

OS10# configure terminal

OS10(config)# smartfabric l3fabric enable role LEAF vlti ethernet
1/1/49-1/1/52

Reboot to change the personality? [yes/no]:y

The configuration is applied, and the switches reload.

To verify that switches are in SmartFabric mode, run the following command on each
switch:

OS10# show switch-operating-mode

Switch-Operating-Mode : Smart Fabric Mode

Note: For more information, see the OpenManage Network Integration User Guide,
Release 3.2. The guide is available on the Dell OpenManage Network Integration
website. For additional SmartFabric CLI commands, see the SmartFabric Services User
Guide on the SmartFabric OS10 Documentation page.

Configure the First Leaf Switch Pair

Connect to SmartFabric Services UI

To connect to the SFS UI:

1. From a workstation with access to the OOB management network, use a browser to
connect to the management IP address of either leaf switch by going to
https://switch_mgmt_ip_address.
2. Log in as admin. The default password is admin.
Note: After reloading the switches, it takes a few minutes for SFS to come up, and
for the UI to be fully functional.

Note: The SFS UI supports the Google Chrome, Mozilla Firefox, and Microsoft
Edge browsers. Languages other than English are not currently supported.

All SFS UI configuration is performed on the SFS leader switch. If you connect to an
SFS switch that is not the leader, a link to the leader is provided. The figure below
outlines the link in red.

Figure 13. Connected to switch that is not the leader

3. If applicable, click the link that is provided to go to the leader switch, and log in as
admin.
Note: The IPv4 address of the SFS leader may also be determined by running the
show smartfabric cluster command from the CLI of any switch in the SmartFabric.
Only one switch in the SmartFabric has ROLE set to MASTER (or leader). The
remaining switches have ROLE set to BACKUP.

When connected to the SFS leader switch, the Summary > Overview page
displays, as shown in the figure below.

Figure 14. Connected to SFS leader switch

The Device Status box shows that there are two switches online in the
SmartFabric.

4. Click the Topology tab. The current topology displays with the leader identified, as
shown.

Figure 15. SFS topology

Configure the First Leaf Switch Pair

Update fabric and switch names

To change the fabric, rack, and switch names from their default settings, perform the
following steps:

1. In the SFS UI, click Global Settings > Set Fabric & Switch Name > SET FABRIC
& SWITCH NAME.
2. On the Network Fabric page, update the fabric Name (recommended) and
Description (optional), as shown.

Figure 16. Network Fabric name changed to Fabric-01

3. Click NEXT.
4. On the Racks page, update the Name (recommended) and Description (optional)
of the rack. In this example, the rack name is set to Rack 1, as shown.

Figure 17. Rack name changed to Rack 1

5. Click NEXT.
6. On the Switches page, update the Name (recommended, if not previously
configured from the CLI) and Description (optional) of the switches. Hostnames are
set to Leaf1A and Leaf1B in this example, as shown.

Figure 18. Switch name configuration page

7. Click FINISH > OK to apply the settings.

Create networks for first VxRail cluster

The networks and VLAN IDs shown in the table below are used for the first VxRail
cluster in this deployment.

Configure the First Leaf Switch Pair

 Table 8. VxRail networks

Description Network ID Network name VLAN ID

External Management extmgmt-1811 extmgmt 1811

vMotion vmotion-1812 vmotion 1812

vSAN vsan-1813 vsan 1813

VM Network A (optional guest network) vmnet_A-1814 vmnet_A 1814

VM Network B (optional guest network) vmnet_B-1815 vmnet_B 1815

Configure the First Leaf Switch Pair

Note: VLAN 4091 and 3939 are also used by the VxRail nodes. These two networks are
automatically configured on the switches by SFS and no user configuration is necessary.

The External Management, vMotion, and vSAN networks are required. The guest
networks shown are for user VMs and are optional.

The networks are created as as follows:

1. In the SFS UI, select Network > Networks and expand General Purpose
Networks.
2. Click +CREATE.
3. Enter the Network ID, Network Name, and VLAN number for the network, as
shown.

Figure 19. Create external management network

4. Click CREATE.
5. Repeat steps 2 through 4 for the vMotion, vSAN, and guest networks if applicable.

When complete, the screen shows the newly created networks.

Figure 20. Networks created

Create server interface profiles for the first VxRail cluster

Note: Before proceeding, make sure the VxRail nodes are connected to the leaf
switches and powered on.

In SFS, switch ports connected to servers are associated with networks using server
interface profiles. VxRail node-connected ports are automatically recognized by SFS and
identified as such in the Import from Fabric feature.

To add the VxRail node-connected ports to the VxRail networks created in the previous
section, perform the following steps.

1. In the SFS UI, select the Server Interface tab.

2. Under Server Interface Profile, click +IMPORT FROM FABRIC. The Discovered
Server Interface page displays.
3. For VxRail node interfaces, the Server Type column indicates VxRail, as shown.

Figure 21. VxRail interfaces discovered

Configure the First Leaf Switch Pair

 Note: In this example, there are three VxRail nodes connected to the leaf switches,
resulting in six discovered VxRail interfaces. Other connected devices that support
LLDP will also appear in the list. If other devices are connected, it may help to sort
the list by clicking the Server Type column heading.

4. Check the boxes next to all VxRail-connected interfaces and click +ADD TO
SERVER PROFILE.
5. In the Server Profile dialog box:
a. Select New Server Profile.
b. Next to Server Profile ID, enter a profile name, such as vxrail-profile.
c. Leave Bonding Type set to AutoDetect. The page displays as shown:

Figure 22. Sever Profile configuration window

6. Click ASSOCIATE. The new profile, named vxrail-profile in this example, is

associated with the VxRail interfaces, as shown.

Figure 23. New profile associated with VxRail interfaces

7. Check the boxes next to all VxRail interfaces and click +ADD NETWORKS.
8. In the Add Server Interface Networks window, do the following:
a. Next to Tagged Networks, the following networks are selected in this example:
i. extmgmt (VLAN-1811)
ii. vmotion (VLAN-1812)
iii. vsan (VLAN-1813)
iv. vmnet_A (VLAN-1814) – optional guest network
v. vmnet_B (VLAN-1815) – optional guest network
vi. Client_Control_Network (VLAN-3939) – automatically created VxRail
node discovery network
b. Next to Untagged Network, select Client_Management_Network (VLAN-
4091). This network is automatically created and is used to connect to the
VxRail Deployment Wizard.
c. Click ADD to display the Discovered Server Interface page as shown in the
figure below.

Figure 24. Interfaces placed in VxRail networks

9. Click the box next to all VxRail interfaces and click CREATE to apply the
configuration.
10. In the Importing Server Interface Profiles window, a success message is

10

Configure the First Leaf Switch Pair

 displayed for each interface as shown:

Figure 25. Success messages displayed

11. Click OK.

The VxRail interfaces appear in the Server Interface Profile list. When an interface is
selected, the associated leaf switch, port, and networks are displayed to the right of the
interface profile list. The networks for the first VxRail interface selected are outlined in
red in the lower right corner of the following figure.

Figure 26. VxRail interface selected

Ensure that the VxRail interfaces are in the correct networks.

Optionally, you may also verify the network to interface configuration at the CLI of the
leaf switches by running the show virtual-network command.

Configure L2 uplinks to the external network

Uplinks to the existing network may be configured as L2, L3 routed, or L3 VLAN. This
section covers L2 uplinks.

Note: If L3 uplinks are used, go to the Configure L3 routed uplinks to the external
network. If you use NSX-T in the SmartFabric, Dell Technologies recommends using L3
uplinks.

The switches are cabled as shown in L2 uplinks to the external network . When L2 uplink
configuration is complete, Leaf1A and Leaf1B connect with a VLT port channel to a
switch pair named External-A and External-B. In this example, an existing DNS/NTP
server also connects to the external switches using a VLT port channel. All VLT port
channels use LACP in this guide.

Note: DNS and NTP servers do not have to connect in this manner if they are reachable
on the network.

All ports on the four switches that are shown in L2 uplinks to the external network are in
the External Management VLAN, 1811.

Figure 27. L2 uplinks to the external network

11

Configure the First Leaf Switch Pair

Configure L2 uplinks in SFS
Note: Any ports available on the leaf switches may be used as uplinks, provided they are
compatible with the corresponding ports on the external switches. If leaf switch uplink
ports will not use their native speeds, the interfaces must be first broken out to the
correct speed before the uplinks are created. A breakout example is shown in the
Change the port-group speed in the SFS web UI .

L2 uplinks to the external network are configured as follows:

1. In the SFS UI, select the Uplink tab.

2. Under Uplink, click +CREATE

Figure 28. SFS UI Uplink tab

3. On the Uplink Details page:

a. Set Uplink Port Type to L2.
b. Enter a Name (required) and Description (optional).

Figure 29. Uplink Details page

4. Click NEXT.
5. On the Port Configuration page:
a. Next to Select Rack to Create Uplink on , select the rack that contains the
switches with the uplinks. In this example, Rack 1 is selected.
b. Under each leaf, select the interface or interfaces that you will use for the
uplink. In this example, 100 GbE interfaces 1/1/53 and 1/1/54 are selected on
each leaf. The Port Channel mode is kept at its default setting, LACP.
Note: Be sure to configure the corresponding ports on the external switches
with the same port channel mode. External switch configuration examples using
LACP are provided in the Configure external switches for L2 connections .

Figure 30. Port Configuration page

6. Click NEXT.
7. In the Network Configuration > Tagged Networks dropdown box, select the
External Management VLAN created earlier; extmgmt (VLAN-1811). This makes
the uplinks the tagged members of the External Management VLAN.
Note: In this example, an untagged network is not selected.

12

Configure the First Leaf Switch Pair

 8. If networks automatically created through vCenter integration are to be extended on
this uplink, select Yes. Otherwise, select No (default). No is used in this example.
Note: Networks created through vCenter integration refers to networks that are
created in vCenter that are automatically configured on the SmartFabric switches by
OMNI.

Figure 31. Network configuration

9. Click FINISH to apply the settings. After configuration, the uplink is listed on the
Uplink page of the SFS UI, along with a series of Success messages.
Note: You may need to click the blue Refresh icon for the uplink to display.

Figure 32. Uplink page after L2 uplink is configured

Note: Creating the first uplink also sets the preferred leader flag on certain switches
in the fabric, if not already configured. Other Success messages indicating the
preferred leader is set may be displayed. The preferred leader setting is covered
later in this guide.

10. Click the Topology tab. The uplink connections are added to the topology as shown
in the following figure.

Figure 33. Topology with L2 uplink created

Optionally, enter the show smartfabric uplinks command at the CLI of either leaf switch
to view configured interfaces and networks on the uplink.

Note: The command output that is shown in the following command is for Leaf1A. The
output for Leaf1B is the same.

Leaf1A# show smartfabric uplinks

----------------------------------------------------------
Name : L2-to-external-network
Description :
ID : b2662568-1e61-4aaf-aa5d-a7c5fcf2c3e1
Media Type : Ethernet - No Spanning Tree
Native Vlan :0
Untagged-network :
Networks : extmgmt-1811

13

Configure the First Leaf Switch Pair

Configured-Interfaces : JX600Q2:ethernet1/1/53, JVSZZP2:ethernet1/1/53, JX600Q2:eth
ernet1/1/54, JVSZZP2:ethernet1/1/54
----------------------------------------------------------

Configure external switches for L2 connections

This section shows example configurations for both external switches for L2 connections
to the SmartFabric.

Note: The external switches that are used in this example are Dell PowerSwitch
systems. If the external switches are Cisco Nexus, see Cisco Nexus External Switch
Configuration Example.

Note: This is only an example. Modify your external switch configuration as needed for
your network.

General settings

Configure the hostname, OOB management IP address, and OOB management route
as shown.

External-A External-B

configure terminal configure terminal

hostname External-A hostname External-B

interface mgmt1/1/1 interface mgmt1/1/1

no ip address dhcp no ip address dhcp
ip address 100.67.76.41/24 ip address 100.67.76.40/24
no shutdown no shutdown

management route 100.67.0.0/16 100.67.76.254 management route 100.67.0.0/16 100.67.76.254

14

Configure the First Leaf Switch Pair

Configure VLANs

Create the External Management VLAN. If traffic will be routed from the external
switches to other external networks, assign a unique IP address on each switch and
configure VRRP to provide gateway redundancy. Set the VRRP priority. The switch with
the highest priority value becomes the master VRRP router. Assign the same virtual
address to both switches.

External-A External-B

interface vlan1811 interface vlan1811

description External_Mgmt description External_Mgmt
ip address 172.18.11.252/24 ip address 172.18.11.253/24
no shutdown no shutdown
vrrp-group 11 vrrp-group 11
priority 150 priority 100
virtual-address 172.18.11.254 virtual-address 172.18.11.254

15

Configure the First Leaf Switch Pair

Configure interfaces

Configure the interfaces for connections to the SFS leaf switches. Interfaces 1/1/13 and
1/1/14 are configured in VLT port channel 100 in this example. Port-channel 100 is set
as an LACP port channel with the channel-group 100 mode active command.

Use the switchport mode trunk command to enable the port channel to carry traffic for
multiple VLANs. Configure the port channel as tagged on VLAN 1811 (the External
Management VLAN).

Optionally, allow the forwarding of jumbo frames with the MTU 9216 command.

In this example, interface 1/1/1 on each external switch is configured in VLT port
channel 1 for connections to the DNS/NTP server. Port-channel 1 is set as an LACP port
channel with the channel-group 1 mode active command.

Configure ports that are directly connected to nodes, servers, or other endpoints as STP
edge ports. As a best practice, flow control settings remain at their factory defaults as
shown.

16

Configure the First Leaf Switch Pair

External-A External-B

interface port-channel100 interface port-channel100

description "To Leaf1A/1B" description "To Leaf1A/1B"
no shutdown no shutdown
switchport mode trunk switchport mode trunk
switchport trunk allowed vlan 1811 switchport trunk allowed vlan 1811
mtu 9216 mtu 9216
vlt-port-channel 100 vlt-port-channel 100

interface range ethernet1/1/13-1/1/14 interface range ethernet1/1/13-1/1/14

description "To Leaf1A/1B" description "To Leaf1A/1B"
no switchport no switchport
channel-group 100 mode active channel-group 100 mode active
mtu 9216 mtu 9216
no shutdown no shutdown

interface port-channel1 interface port-channel1

description "To DNS/NTP" description "To DNS/NTP"
no shutdown no shutdown
switchport access vlan 1811 switchport access vlan 1811
vlt-port-channel 1 vlt-port-channel 1
spanning-tree port type edge spanning-tree port type edge

interface ethernet1/1/1 interface ethernet1/1/1

description "To DNS/NTP" description "To DNS/NTP"
no switchport no switchport
channel-group 1 mode active channel-group 1 mode active
no shutdown no shutdown
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

17

Configure the First Leaf Switch Pair

Configure VLT

This example uses interfaces 1/1/11 and 1/1/12 for the VLTi. Remove each interface
from L2 mode with the no switchport command.

Create the VLT domain. The backup destination is the OOB management IP address of
the VLT peer switch. Configure the interfaces that are used as the VLTi with the
discovery-interface command.

As a best practice, use the vlt-mac command to manually configure the same VLT MAC
address on both the VLT peer switches. This improves VLT convergence time when a
switch is reloaded.

CAUTION: Be sure that the VLT MAC address is the same on both switches to avoid
any unpredictable behavior.

If you do not configure a VLT MAC address, the MAC address of the primary peer is
used as the VLT MAC address on both switches.

Note: For more information about VLT, see the Dell SmartFabric OS10 User Guide on
the Dell Networking OS10 Info Hub.

When the configuration is complete, exit configuration mode and save the configuration
with the end and write memory commands.

External-A External-B

interface range ethernet1/1/11-1/1/12 interface range ethernet1/1/11-1/1/12

description VLTi description VLTi
no switchport no switchport
no shutdown no shutdown
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

vlt-domain 255 vlt-domain 255

backup destination 100.67.76.40 backup destination 100.67.76.41
discovery-interface ethernet1/1/11-1/1/12 discovery-interface ethernet1/1/11-1/1/12
vlt-mac 00:00:01:02:03:20 vlt-mac 00:00:01:02:03:20

end end
write memory write memory

18

Configure the First Leaf Switch Pair

Validation
Once the uplink interfaces have been configured on the external switches and in the
SFS UI, additional validation is done using the switch CLI.

Show command output on External-A

Note: The command output that is shown in the following commands is for the External-
A switch. The output for External-B is similar.

Run the show vlan command to verify that the ports are correctly assigned to the
External Management VLAN. Port channel 100 connects to the SFS leaf switches and is
a tagged member of the same VLAN configured on the SmartFabric uplinks (VLAN
1811). It is tagged because it is also tagged on the SmartFabric leaf switches. The
DNS/NTP server is connected on port channel 1, which is an access member of VLAN
1811 in this example.

External-A# show vlan

Codes: * - Default VLAN, M - Management VLAN, R - Remote Port Mirroring VLANs,
@ – Attached to Virtual Network, P - Primary, C - Community, I - Isolated
Q: A - Access (Untagged), T - Tagged
NUM Status Description Q Ports
* 1 Active A Eth1/1/2-1/1/10,1/1/15
A Po100,1000
1811 Active External_Mgmt T Po100,1000
A Po1
4094 Active T Po1000

The show port channel summary command confirms port channel 100 connected to the
leaf switches is up and active. Port channel 1000 is the VLTi, and port channel 1 is
connected to the DNS/NTP server.

External-A# show port-channel summary

Flags: D - Down I - member up but inactive P - member up and active
U - Up (port-channel) F - Fallback Activated
--------------------------------------------------------------------------------
Group Port-Channel Type Protocol Member Ports
--------------------------------------------------------------------------------
1 port-channel1 (U) Eth DYNAMIC 1/1/1(P)
100 port-channel100 (U) Eth DYNAMIC 1/1/13(P) 1/1/14(P)
1000 port-channel1000 (U) Eth STATIC 1/1/11(P) 1/1/12(P)

Show command output on Leaf1A

19

Configure the First Leaf Switch Pair

Note: The command output that is shown in the following commands is for Leaf1A. The
output for Leaf1B is similar.

With SFS, port channel numbers are automatically assigned as they are created. Port
channel 1 is the uplink that is connected to the external switches and is up and active.
Port channel 1000 is reserved for the VLTi.

Leaf1A# show port-channel summary

Flags: D - Down I - member up but inactive P - member up and active

U - Up (port-channel) F - Fallback Activated
--------------------------------------------------------------------------------
Group Port-Channel Type Protocol Member Ports
--------------------------------------------------------------------------------
1 port-channel1 (U) Eth DYNAMIC 1/1/53(P) 1/1/54(P)
1000 port-channel1000 (U) Eth STATIC 1/1/49(P) 1/1/50(P) 1/1/51(P) 1/1/52(P)

The L2 uplink, port channel 1 in this example, is added as a tagged member of VLAN
1811. This is verified at the CLI using the show virtual-network 1811 command as
follows:

Leaf1A# show virtual-network 1811

Codes: DP - MAC-learn Dataplane, CP - MAC-learn Controlplane, UUD - Unknown-Unic
ast-Drop
Virtual Network: 1811
VLTi-VLAN: 1811
Members:
VLAN 1811: port-channel1, port-channel1000, ethernet1/1/1, ethernet1/1/2, ethernet
1/1/3
VxLAN Virtual Network Identifier: 1811
Source Interface: loopback2(172.30.0.0)
Remote-VTEPs (flood-list):

Configure L3 routed uplinks to the external network

Uplinks to the existing network may be configured as L2, L3 routed, or L3 VLAN. This
section covers L3 routed uplinks.

Note: If L2 uplinks were configured in the preceding section, skip this section and go to
Configure a jump host port.

Note: L3 VLAN uplink configuration is beyond the scope of this guide.

20

Configure the First Leaf Switch Pair

Connections, port numbers, and networks used for external management are shown in
the figure below. The External Management VLAN is VLAN 1911 on the external
switches and is VLAN 1811 on the SmartFabric switches.

Figure 34. L3 routed uplinks to the external network. L3 routed uplinks to the
external network

Point-to-point IP networks
The point-to-point links used in this deployment are labeled A-E in Point-to-point
connections.

Figure 35. Point-to-point connections

Each L3 uplink is a separate, point-to-point IP network. L3 routed uplink IP addresses

details the links labeled in Point-to-point connections. The IP addresses in the table
below are used in the switch configuration examples.

Table 9. L3 routed uplink IP addresses

Link Source Source IP Destination Destination IP

Network
label switch address switch address

External- 192.168.1.0
A 192.168.1.0 Leaf1A 192.168.1.1
A /31

External- 192.168.1.2
B 192.168.1.2 Leaf1B 192.168.1.3
A /31

External- 192.168.2.0
C 192.168.2.0 Leaf1A 192.168.2.1
B /31

External- 192.168.2.2
D 192.168.2.2 Leaf1B 192.168.2.3
B /31

External- 192.168.3.2
E 192.168.3.20 External-B 192.168.3.21
A 0/31

21

Configure the First Leaf Switch Pair

BGP example
This section covers the L3 routed uplink configuration with BGP.

Note: If BGP is not used, go to Static route example.

BGP ASNs and router IDs

BGP ASNs and router IDs shows the autonomous system numbers (ASNs) and router
IDs used for the external switches and SFS leaf switches in this example. External
switches share a common ASN, and all SFS leaf switches share a common ASN.

Figure 36. BGP ASNs and router IDs

Note: Using private ASNs in the data center is a best practice. Private, 2-byte ASNs
range from 64512 through 65534.

In this example, ASN 65101 is used on both external switches. SFS leaf switches use
ASN 65011 by default for all leafs in the fabric.

Note: If L3 uplinks are connected from SFS spine switches, the spine switches use ASN
65012 by default.

The IP addresses shown on the external network switches in BGP ASNs and router IDs
are loopback addresses used as BGP router IDs. On the SmartFabric switches, BGP
router IDs are automatically configured from the SFS default private subnet address
block, 172.16.0.0/16.

Note: SFS default ASNs and IP address blocks may be changed by going to Global
Settings > Fabric Settings in the SFS web UI.

Configure L3 routed uplinks with BGP in SFS

The following table shows the values entered in the SFS web UI to configure the L3
uplinks for this example. The steps below the table are run once for each uplink using
the values in the table.

22

Configure the First Leaf Switch Pair

 Table 10. L3 uplink configuration details with BGP

Leaf1A-to- Leaf1A-to- Leaf1B-to- Leaf1B-to-

Field name
External-A External-B External-A External-B

Uplink Port Type L3 L3 L3 L3

L3 Routed L3 Routed L3 Routed L3 Routed

L3 Type
Interface Interface Interface Interface

Uplink Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Switch Group Leaf Leaf Leaf Leaf

Rack Rack 1 Rack 1 Rack 1 Rack 1

Node Leaf1A Leaf1A Leaf1B Leaf1B

Ethernet Ethernet Ethernet Ethernet

Interface
1/1/53 1/1/54 1/1/53 1/1/54

Network Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Prefix Length 31 31 31 31

IP Address 192.168.1.1 192.168.2.1 192.168.1.3 192.168.2.3

Routing Protocol eBGP eBGP eBGP eBGP

Policy Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Peer Interface IP
192.168.1.0 192.168.2.0 192.168.1.2 192.168.2.2
Address

Peer ASN 65101 65101 65101 65101

BFD Neighbor Enable Enable Enable Enable

Fabric
unchecked unchecked unchecked unchecked
Interconnect

23

Configure the First Leaf Switch Pair

 Leaf1A-to- Leaf1A-to- Leaf1B-to- Leaf1B-to-
Field name
External-A External-B External-A External-B

24

Configure the First Leaf Switch Pair

Note: Any ports available on the leaf switches may be used as uplinks, provided they are
compatible with the corresponding ports on the external switches. If leaf switch uplink
ports will not use their native speeds, the interfaces must be first broken out to the
correct speed before the uplinks are created. A breakout example is shown in Change
the port-group speed in the SFS web UI.

To configure L3 routed uplinks with BGP, do the following using the data from the L3
uplink configuration details with BGP table above.

1. In the SFS UI, select the Uplink tab.

2. Under Uplink, click CREATE.

Figure 37. SFS UI Uplink tab

3. On the Uplink Details page:

a. Set Uplink Port Type to L3.
b. Next to L3 Type, select L3 Routed Interface.
c. Enter a unique uplink Name.

Figure 38. Uplink Details page

4. Click NEXT.
5. On the Port Configuration page:
a. Leave the Switch Group set to Leaf.
b. Next to Select Rack to Create Uplink on , select the rack that contains the
switches with the uplinks. In this example, Rack 1 is selected.
c. Next to Node, select the first leaf, Leaf1A in this example.
d. In the Interface Selection list, select the interface that will be used for the
uplink. In this example, interface 1/1/53 is selected.

Figure 39. Port Configuration page

6. Click NEXT.
7. On the Network Configuration page:
a. Enter a unique network profile Name.
b. Enter the Prefix Length and IP Address of the interface.
c. Select the Routing Protocol, eBGP.
d. Enter a unique Policy Name.
e. Enter the Peer Interface IP Address and Peer ASN.
f. Next to BFD Neighbor, select the Enable or Disable button. In this example, it
is set to Enable.

25

Configure the First Leaf Switch Pair

 Note: Dell Technologies recommends that you enable Bidirectional Forwarding
Detection (BFD) if the external switches support it. BFD rapidly detects
communication failures between two adjacent routers.

g. Leave the Fabric Interconnect box unchecked.

Figure 40. Network Configuration page with BGP

8. Click FINISH. The Success message displays.

Note: Creating the first uplink also sets the preferred leader flag on certain switches
in the fabric, if not already configured. Additional Success messages indicating the
preferred leader is set may display. The preferred leader setting is covered in Verify
preferred leader setting before fabric expansion.

9. Repeat the steps in this section for the remaining three uplinks using the data from
the preceding table. After configuration, the uplinks are listed on the Uplink page of
the SFS UI, as shown in the following figure.

Figure 41. Uplink page after uplinks are configured

The Topology tab displays as shown in the following figure:

Figure 42. Topology with L3 uplinks created

Static route example

This section shows L3 routed uplink configuration with a static route.

Note: If BGP is used instead of a static route, continue to Configure external switches
for L3 connections.

Note: Currently, only one static route per L3 uplink is allowed. If multiple routes are
needed, use a default route, 0.0.0.0/0, as the destination network, or add additional
uplinks for specific networks. Support for multiple static routes per L3 uplink is planned
for a future release.

Configure L3 uplinks with a static route in SFS

The following table shows the values entered in the SFS UI to configure the L3 uplinks
for this example. The steps below the table are run once for each uplink using the values

26

Configure the First Leaf Switch Pair

from the table.

Table 11. L3 uplink configuration details with a static route

Leaf1A-to- Leaf1A-to- Leaf1B-to- Leaf1B-to-

Field name
External-A External-B External-A External-B

Uplink Port
L3 L3 L3 L3
Type

L3 Routed L3 Routed L3 Routed L3 Routed

L3 Type
Interface Interface Interface Interface

Uplink Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Switch Group Leaf Leaf Leaf Leaf

Rack Rack 1 Rack 1 Rack 1 Rack 1

Node Leaf1A Leaf1A Leaf1B Leaf1B

Ethernet Ethernet Ethernet Ethernet

Interface
1/1/53 1/1/54 1/1/53 1/1/54

Network Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Prefix Length 31 31 31 31

IP Address 192.168.1.1 192.168.2.1 192.168.1.3 192.168.2.3

Routing
Static Route Static Route Static Route Static Route
Protocol

Policy Name Leaf1A-ExtA Leaf1A-ExtB Leaf1B-ExtA Leaf1B-ExtB

Network
172.19.11.0 172.19.11.0 172.19.11.0 172.19.11.0
Address

Prefix Length 24 24 24 24

27

Configure the First Leaf Switch Pair

 Leaf1A-to- Leaf1A-to- Leaf1B-to- Leaf1B-to-
Field name
External-A External-B External-A External-B

Next Hop IP
192.168.1.0 192.168.2.0 192.168.1.2 192.168.2.2
Address

Fabric
unchecked unchecked unchecked unchecked
Interconnect

28

Configure the First Leaf Switch Pair

Note: Any ports available on the leaf switches may be used as uplinks, provided they are
compatible with the corresponding ports on the external switches. If leaf switch uplink
ports will not use their native speeds, the interfaces must be first broken out to the
correct speed before the uplinks are created. A breakout example is shown in Change
the port-group speed in the SFS web UI.

To configure L3 routed uplinks with a static route, perform the following steps:

1. In the SFS UI, select the Uplink tab.

2. Under Uplink, click CREATE.

Figure 43. SFS UI Uplink tab

3. On the Uplink Details page:

a. Set Uplink Port Type to L3.
b. Next to L3 Type, select L3 Routed Interface.
c. Enter a unique Name.

Figure 44. Uplink Details page

4. Click NEXT.
5. On the Port Configuration page:
a. Leave the Switch Group set to Leaf.
b. Next to Select Rack to Create Uplink on , select the rack that contains the
switches with the uplinks. In this example, Rack 1 is selected.
c. Next to Node, select the first leaf, Leaf1A in this example.
d. In the Interface Selection list, select the interface that will be used for the
uplink. In this example, interface 1/1/53 is selected.

Figure 45. Port Configuration page

6. Click NEXT.
7. On the Network Configuration page:
a. Enter a unique Name.
b. Enter the Prefix Length and IP Address of the interface.
c. Select the Routing Protocol, Static Route.
d. Enter a unique Policy Name.
e. Enter the destination Network Address and Prefix Length. This is the
external network that contains DNS and NTP services, 172.19.11.0/24, in this
example.
f. Enter the Next Hop IP Address. This is the IP address of the connected

29

Configure the First Leaf Switch Pair

 interface on the external switch.
g. Leave the Fabric Interconnect box unchecked.

Figure 46. Network Configuration page with static route

8. Click Finish. A Success message displays.

Note: Creating the first uplink also sets the preferred leader flag on certain switches
in the fabric, if not already configured. Other Success messages indicating the
preferred leader is set may show. The preferred leader setting is covered later in
this guide.

9. Repeat the steps in this section for the remaining three uplinks using the data from
the preceding table. After configuration, the uplinks are listed on the Uplink page of
the SFS UI, as shown.

Figure 47. Uplink page after uplinks are configured

The Topology tab displays as shown in the figure below.

Figure 48. Topology with L3 uplinks created

Configure external switches for L3 connections

This section shows example configurations for both external switches for L3 routed
connections to the SmartFabric.

Note: The external switches used in this example are Dell PowerSwitch systems. If the
external switches are Cisco Nexus, see Cisco Nexus External Switch Configuration
Example.

Note: This is only an example. Modify your external switch configuration as needed for
your network.

General settings

Configure the hostname, OOB management IP address, and management route.

30

Configure the First Leaf Switch Pair

External-A External-B

configure terminal configure terminal

hostname External-A hostname External-B

interface mgmt1/1/1 interface mgmt1/1/1

no ip address dhcp no ip address dhcp
ip address 100.67.76.41/24 ip address 100.67.76.40/24
no shutdown no shutdown

management route 100.67.0.0/16 100.67.76.254 management route 100.67.0.0/16 100.67.76.254

31

Configure the First Leaf Switch Pair

Configure VLANs

VLAN 1911 represents a preexisting management VLAN on the external network. DNS
and NTP services are on this VLAN. Assign a unique IP address to the VLAN on each
switch.

Configure VRRP to provide gateway redundancy. Set the VRRP priority. The switch with
the highest priority value becomes the master VRRP router. Assign the same virtual
address to both switches.

External-A External-B

interface vlan1911 interface vlan1911

no shutdown no shutdown
ip address 172.19.11.252/24 ip address 172.19.11.253/24

vrrp-group 19 vrrp-group 19
priority 150 priority 100
virtual-address 172.19.11.254 virtual-address 172.19.11.254

32

Configure the First Leaf Switch Pair

Configure interfaces

Configure the interfaces for connections to the SFS switches. Ports 1/1/13 and 1/1/14
are configured as L3 interfaces. The IP addresses used are from L3 routed uplink IP
addresses. Optionally, allow the forwarding of jumbo frames with the MTU 9216
command. As a best practice, flow control settings remain at their factory defaults as
shown.

In this example, VLT port channel 1 connects to the DNS/NTP server. It is on VLAN
1911, which represents the preexisting management VLAN, and the port channel is
configured as a spanning tree edge port.

Interface 1/1/1 on each external switch is configured in VLT port channel 1 for
connections to the DNS/NTP server. Port-channel 1 is set as an LACP port channel with
the channel-group 1 mode active command.

External-A External-B

33

Configure the First Leaf Switch Pair

External-A External-B

interface ethernet1/1/13 interface ethernet1/1/13

description Leaf1A description Leaf1A
no shutdown no shutdown
no switchport no switchport
mtu 9216 mtu 9216
ip address 192.168.1.0/31 ip address 192.168.2.0/31
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

interface ethernet1/1/14 interface ethernet1/1/14

description Leaf1B description Leaf1B
no shutdown no shutdown
no switchport no switchport
mtu 9216 mtu 9216
ip address 192.168.1.2/31 ip address 192.168.2.2/31
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

interface port-channel1 interface port-channel1

description "To DNS/NTP" description "To DNS/NTP"
no shutdown no shutdown
switchport access vlan 1911 switchport access vlan 1911
vlt-port-channel 1 vlt-port-channel 1
spanning-tree port type edge spanning-tree port type edge

interface ethernet1/1/1 interface ethernet1/1/1

description "To DNS/NTP" description "To DNS/NTP"
no switchport no switchport
channel-group 1 mode active channel-group 1 mode active
no shutdown no shutdown
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

34

Configure the First Leaf Switch Pair

Configure VLT

This example uses interfaces 1/1/11 and 1/1/12 for the VLTi. Remove each interface
from L2 mode with the no switchport command. As a best practice, flow control settings
remain at their factory defaults, as shown.

Create the VLT domain. The backup destination is the OOB management IP address of
the VLT peer switch. Configure the interfaces used as the VLTi with the discovery-
interface command.

As a best practice, use the vlt-mac command to manually configure the same VLT MAC
address on both the VLT peer switches. This improves VLT convergence time when a
switch is reloaded.

CAUTION: Be sure the VLT MAC address is the same on both switches to avoid any
unpredictable behavior.

If you do not configure a VLT MAC address, the MAC address of the primary peer is
used as the VLT MAC address on both switches.

Note: For more information about VLT, see the Dell SmartFabric OS10 User Guide on
the Dell Networking OS10 Info Hub.

External-A External-B

interface range ethernet1/1/11-1/1/12 interface range ethernet1/1/11-1/1/12

description VLTi description VLTi
no shutdown no shutdown
no switchport no switchport
flowcontrol receive on flowcontrol receive on
flowcontrol transmit off flowcontrol transmit off

vlt-domain 255 vlt-domain 255

backup destination 100.67.76.40 backup destination 100.67.76.41
discovery-interface ethernet1/1/11-1/1/12 discovery-interface ethernet1/1/11-1/1/12
vlt-mac 00:00:01:02:03:20 vlt-mac 00:00:01:02:03:20

35

Configure the First Leaf Switch Pair

Configure BGP
Note: If BGP is not used, go to Configure static routes .

Configure a loopback interface to use for the BGP router ID.

Enable BFD.

Configure the BGP ASN with the router bgp command. The external switches share the
same ASN. Use the address that was set for interface loopback0 as the router ID.

Use the address-family ipv4 unicast and redistribute connected commands to

redistribute IPv4 routes from physically connected interfaces.

Configure the neighbor IP addresses, ASNs and enable BFD.

VLAN 4000 is used for the iBGP connection between the external switches. VLAN4000
IP addresses are configured as in L3 routed uplink IP addresses.

When the configuration is complete, exit configuration mode and save the configuration
with the end and write memory commands.

36

Configure the First Leaf Switch Pair

External-A External-B

interface loopback0 interface loopback0

description router_ID description router_ID
no shutdown no shutdown
ip address 10.0.2.1/32 ip address 10.0.2.2/32

bfd enable bfd enable

router bgp 65101 router bgp 65101

router-id 10.0.2.1 router-id 10.0.2.2

address-family ipv4 unicast address-family ipv4 unicast

redistribute connected redistribute connected

neighbor 192.168.1.1 neighbor 192.168.2.1

remote-as 65011 remote-as 65011
bfd bfd
no shutdown no shutdown

neighbor 192.168.1.3 neighbor 192.168.2.3

remote-as 65011 remote-as 65011
bfd bfd
no shutdown no shutdown

neighbor 192.168.3.21 neighbor 192.168.3.20

remote-as 65101 remote-as 65101
bfd bfd
no shutdown no shutdown

interface vlan4000 interface vlan4000

description iBGP description iBGP
no shutdown no shutdown
ip address 192.168.3.20/31 ip address 192.168.3.21/31

end end
write memory write memory

37

Configure the First Leaf Switch Pair

Configure static routes
Note: If BGP is used, skip this section and go to Validate BGP example.

Configure two routes to the external management network. This is 172.18.11.0/24, one
to the connected IP address of Leaf1A, and one to Leaf1B.

When the configuration is complete, exit configuration mode and save the configuration
with the end and write memory commands.

External-A External-B

ip route 172.18.11.0/24 192.168.1.1 ip route 172.18.11.0/24 192.168.2.1

ip route 172.18.11.0/24 192.168.1.3 ip route 172.18.11.0/24 192.168.2.3

end end
write memory write memory

38

Configure the First Leaf Switch Pair

Validate BGP example
Note: This section shows validation commands for the BGP example. If static routes are
used, skip this section and go to Validate static route example .

Now that the uplink interfaces are configured on the external switches and on the SFS
leaf switches, connectivity can be verified using the switch CLI.

Show command output on External-A (BGP example)

Note: The command output shown in the following commands is for the External-A
switch. The output for External-B is similar.

Run the show ip bgp summary command to verify that BGP is up for each neighbor.
When BGP is up, uptime is shown in the Up/Down column. The neighbors for External-A
shown in the output below are Leaf1A, Leaf1B, and External-B.

External-A# show ip bgp summary

BGP router identifier 10.0.2.1 local AS number 65101
Neighbor AS MsgRcvd MsgSent Up/Down State/Pfx
192.168.1.1 65011 1327 1316 19:09:00 4
192.168.1.3 65011 1325 1324 19:09:00 4
192.168.3.21 65101 1319 1315 19:01:18 5

Run the show ip interface brief command to verify connected interfaces are up, and IP
addresses are configured correctly. In the output below, interface 1/1/1 and port channel
1 connect to the DNS/NTP server. 1/1/13-1/1/14 are the links to the SFS leaf switches,
and 1/1/11-1/1/12 are the VLTi links. VLAN 4094 and port channel 1000 are
automatically configured for the VLTi.

VLAN 1911 is the external management VLAN that contains the DNS/NTP server. VLAN
4094 and port channel 1000 are automatically configured for the VLTi.

Note: Unused interfaces have been removed from the output for brevity.

External-A# show ip interface brief

Interface Name IP-Address OK Method Status Protocol
===================================================================
=============
Ethernet 1/1/1 unassigned YES unset up up
Ethernet 1/1/11 unassigned YES unset up up
Ethernet 1/1/12 unassigned YES unset up up
Ethernet 1/1/13 192.168.1.0/31 YES manual up up

39

Configure the First Leaf Switch Pair

Ethernet 1/1/14 192.168.1.2/31 YES manual up up
Management 1/1/1 100.67.76.41/24 YES manual up up
Vlan 1 unassigned YES unset up up
Vlan 1911 172.19.11.252/24 YES manual up up
Vlan 4000 192.168.3.20/31 YES manual up up
Vlan 4094 unassigned YES unset up up
Port-channel 1 unassigned YES unset up up
Port-channel 1000 unassigned YES unset up up
Loopback 0 10.0.2.1/32 YES manual up up

Theshow ip route command output for the External-A switch appears as shown. No BGP
routes from the SFS fabric are learned at this stage of deployment. Interfaces 1/1/13 and
1/1/14 are connected to the SFS leaf switches.

External-A# show ip route

Codes: C - connected
S - static
B - BGP, IN - internal BGP, EX - external BGP
O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,
E2 - OSPF external type 2, * - candidate default,
+ - summary route, > - non-active route
Gateway of last resort is not set
Destination Gateway Dist Last Change
----------------------------------------------------------------------------------
C 10.0.2.1/32 via 10.0.2.1 loopback0 0/0 00:39:19
B IN 10.0.2.2/32 via 192.168.3.21 200/0 00:31:38
C 172.19.11.0/24 via 172.19.11.252 vlan1911 0/0 00:44:00
C 192.168.1.0/31 via 192.168.1.0 ethernet1/1/13 0/0 01:44:44
C 192.168.1.2/31 via 192.168.1.2 ethernet1/1/14 0/0 01:40:50
B IN 192.168.2.0/31 via 192.168.3.21 200/0 00:31:38
B IN 192.168.2.2/31 via 192.168.3.21 200/0 00:31:38
C 192.168.3.20/31 via 192.168.3.20 vlan4000 0/0 00:31:51

Show command output on Leaf1A (BGP example)

Note: The command output shown in the following commands is for Leaf1A. The output
for Leaf1B is similar.

Run the show ip bgp summary command to verify that BGP is up for each neighbor.
When BGP is up, uptime is shown in the Up/Down column. The neighbors for Leaf1A
shown in the output below are Leaf1B, External-A, and External-B.

Leaf1A# show ip bgp summary

BGP router identifier 172.16.128.0 local AS number 65011

40

Configure the First Leaf Switch Pair

Neighbor AS MsgRcvd MsgSent Up/Down State/Pfx
172.16.0.0 65011 13 16 00:06:59 8
192.168.1.0 65101 12 14 00:07:30 8
192.168.2.0 65101 8 9 00:04:14 8

Run the show ip interface brief command to verify connected interfaces are up, and IP
addresses are configured correctly.

In the output below, interfaces 1/1/1-1/1/3 are connected to the VxRail nodes, 1/1/49-
1/1/52 are the VLTi, and 1/1/53-1/1/54 are the uplinks to the external switches. SFS uses
VLAN 4000-4090, Loopback 1, and Loopback 2 internally. VLAN 4094 and port channel
1000 are automatically configured for the VLTi.

Note: Unused interfaces have been removed from the output for brevity.

Leaf1A# show ip interface brief

Interface Name IP-Address OK Method Status Protocol
===================================================================
=============
Ethernet 1/1/1 unassigned YES unset up up
Ethernet 1/1/2 unassigned YES unset up up
Ethernet 1/1/3 unassigned YES unset up up
Ethernet 1/1/49 unassigned YES unset up up
Ethernet 1/1/50 unassigned YES unset up up
Ethernet 1/1/51 unassigned YES unset up up
Ethernet 1/1/52 unassigned YES unset up up
Ethernet 1/1/53 192.168.1.1/31 YES manual up up
Ethernet 1/1/54 192.168.2.1/31 YES manual up up
Management 1/1/1 100.67.76.30/24 YES manual up up
Vlan 4000 unassigned YES unset up up
Vlan 4089 unassigned YES unset up up
Vlan 4090 172.16.0.1/31 YES manual up up
Vlan 4094 unassigned YES unset up up
Port-channel 1000 unassigned YES unset up up
Loopback 1 172.16.128.0/32 YES manual up up
Loopback 2 172.30.0.0/32 YES manual up up
Virtual-network 3939 unassigned YES unset up up

Run the show ip route command to verify routes to the external management VLAN,
172.19.11.0/24, have been learned using BGP from the external switches. In this
example, two routes to 172.19.11.0/24 are learned, one using each external switch and
are underlined in the example below.

Leaf1A# show ip route

41

Configure the First Leaf Switch Pair

Codes: C - connected
S - static
B - BGP, IN - internal BGP, EX - external BGP
O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,
E2 - OSPF external type 2, * - candidate default,
+ - summary route, > - non-active route
Gateway of last resort is not set
Destination Gateway Dist Last Change
----------------------------------------------------------------------------------
B EX 10.0.2.1/32 via 192.168.1.0 20/0 00:43:16
via 192.168.2.0
B EX 10.0.2.2/32 via 192.168.1.0 20/0 00:43:16
via 192.168.2.0
C 172.16.0.0/31 via 172.16.0.1 vlan4090 0/0 02:19:46
C 172.16.128.0/32 via 172.16.128.0 loopback1 0/0 02:20:07
B IN 172.16.128.1/32 via 172.16.0.0 200/0 02:19:44
B EX 172.19.11.0/24 via 192.168.1.0 20/0 00:43:32
via 192.168.2.0
C 172.30.0.0/32 via 172.30.0.0 loopback2 0/0 02:20:07
C 192.168.1.0/31 via 192.168.1.1 ethernet1/1/53 0/0 01:12:49
B IN 192.168.1.2/31 via 172.16.0.0 200/0 01:09:12
C 192.168.2.0/31 via 192.168.2.1 ethernet1/1/54 0/0 01:10:18
B IN 192.168.2.2/31 via 172.16.0.0 200/0 01:07:51
B EX 192.168.3.20/31 via 192.168.1.0 20/0 00:43:21
via 192.168.2.0

Validate static route example

Note: This section shows validation commands for the static route example. If BGP was
used, skip this section and go to Additional configuration steps for L3 uplinks.

Once the uplink interfaces have been configured on the external switches and in the
SFS web UI, connectivity can be verified using the switch CLI.

Show command output on External-A (static route example)

Note: The command output shown in the following commands is for the External-A
switch. The output for External-B is similar.

Run the show ip interface brief command to verify connected interfaces are up, and IP
addresses are configured correctly. In the output below, interface 1/1/1 and port channel
1 connect to the DNS/NTP server. 1/1/13-1/1/14 are the links to the SFS leaf switches,

42

Configure the First Leaf Switch Pair

and 1/1/11-1/1/12 are the VLTi links.

VLAN 1911 is the external management VLAN that contains the DNS/NTP server. VLAN
4094 and port channel 1000 are automatically configured for the VLTi.

Note: Unused interfaces have been removed from the output for brevity.

External-A# show ip interface brief

Interface Name IP-Address OK Method Status Protocol
===================================================================
=============
Ethernet 1/1/1 unassigned YES unset up up
Ethernet 1/1/11 unassigned YES unset up up
Ethernet 1/1/12 unassigned YES unset up up
Ethernet 1/1/13 192.168.1.0/31 YES manual up up
Ethernet 1/1/14 192.168.1.2/31 YES manual up up
Management 1/1/1 100.67.76.41/24 YES manual up up
Vlan 1 unassigned YES unset up up
Vlan 1911 172.19.11.252/24 YES manual up up
Vlan 4094 unassigned YES unset up up
Port-channel 1 unassigned YES unset up up
Port-channel 1000 unassigned YES unset up up

Run the show ip route command to verify static routes to the external management
VLAN, 172.18.11.0/24, are properly configured.

External-A# show ip route

Codes: C - connected
S - static
B - BGP, IN - internal BGP, EX - external BGP
O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,
E2 - OSPF external type 2, * - candidate default,
+ - summary route, > - non-active route
Gateway of last resort is not set
Destination Gateway Dist Last Change
----------------------------------------------------------------------------------
S 172.18.11.0/24 via 192.168.1.1 ethernet1/1/13 1/0 3 days 23:35:18
via 192.168.1.3 ethernet1/1/14
C 172.19.11.0/24 via 172.19.11.252 vlan1911 0/0 3 days 23:26:55
C 192.168.1.0/31 via 192.168.1.0 ethernet1/1/13 0/0 21:58:31
C 192.168.1.2/31 via 192.168.1.2 ethernet1/1/14 0/0 21:58:33

Show command output on Leaf1A (static route example)

43

Configure the First Leaf Switch Pair

Note: The command output shown in the following commands is for Leaf1A. The output
for Leaf1B is similar.

Run the show ip interface brief command to verify connected interfaces are up, and IP
addresses are configured correctly.

In the output below, interfaces 1/1/1-1/1/3 are connected to the VxRail nodes, 1/1/49-
1/1/52 are the VLTi links, and 1/1/53-1/1/54 are the uplinks to the external switches.

Note: Unused interfaces have been removed from the output for brevity.

Leaf1A# show ip interface brief

Ethernet 1/1/1 unassigned YES unset up up
Ethernet 1/1/2 unassigned YES unset up up
Ethernet 1/1/3 unassigned YES unset up up
Ethernet 1/1/49 unassigned YES unset up up
Ethernet 1/1/50 unassigned YES unset up up
Ethernet 1/1/51 unassigned YES unset up up
Ethernet 1/1/52 unassigned YES unset up up
Ethernet 1/1/53 192.168.1.1/31 YES manual up up
Ethernet 1/1/54 192.168.2.1/31 YES manual up up
Management 1/1/1 100.67.76.30/24 YES manual up up
Vlan 4000 unassigned YES unset up up
Vlan 4090 172.16.0.1/31 YES manual up up
Vlan 4094 unassigned YES unset up up
Port-channel 1000 unassigned YES unset up up
Loopback 1 172.16.128.0/32 YES manual up up
Loopback 2 172.30.0.0/32 YES manual up up
Virtual-network 3939 unassigned YES unset up up

Run the show ip route command to verify static routes to the external management
VLAN, 172.19.11.0/24, are correctly configured. The static routes are underlined in the
command output below.

Note: Since SFS uses BGP to exchange routes within the fabric, some BGP routes
appear in the output.

Leaf1A# show ip route

Codes: C - connected
S - static
B - BGP, IN - internal BGP, EX - external BGP
O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,

44

Configure the First Leaf Switch Pair

 E2 - OSPF external type 2, * - candidate default,
+ - summary route, > - non-active route
Gateway of last resort is not set
Destination Gateway Dist Last Change
----------------------------------------------------------------------------------
C 172.16.0.0/31 via 172.16.0.1 vlan4090 0/0 02:09:34

C 172.16.0.0/31 via 172.16.0.1 vlan4090 0/0 00:40:43

B IN 172.16.128.0/32 via 172.16.0.0 200/0 00:40:42
C 172.16.128.1/32 via 172.16.128.1 loopback1 0/0 00:40:50
S 172.19.11.0/24 via 192.168.1.0 ethernet1/1/53 1/0 00:37:51
via 192.168.2.0 ethernet1/1/54
C 172.30.0.0/32 via 172.30.0.0 loopback2 0/0 00:40:50
C 192.168.1.0/31 via 192.168.1.1 ethernet1/1/53 0/0 00:37:56
B IN 192.168.1.2/31 via 172.16.0.0 200/0 00:35:52
C 192.168.2.0/31 via 192.168.2.1 ethernet1/1/54 0/0 00:36:57
B IN 192.168.2.2/31 via 172.16.0.0 200/0 00:34:51

Additional configuration steps for L3 uplinks

Note: If using L2 uplinks, skip this section and go to Configure a jump host port.

Traffic on the External Management network, VLAN 1811, must be able to reach the
DNS server on the external network during VxRail deployment. To accomplish this with
L3 uplinks, an IP address is assigned to virtual network 1811 on each leaf switch. An
anycast gateway (or virtual router) address shared by all leafs is also configured on the
same network.

Since this is on virtual network 1811, available IP addresses in the 172.18.11.0/24

address block are used per the planning data in VLANs used by VxRail.

45

Configure the First Leaf Switch Pair

 Table 12. Leaf switch External
Management network IP addresses and
anycast gateway

Item IP address/prefix

Leaf1A IP address 172.18.11.253/24

Leaf1B IP address 172.18.11.252/24

Gateway IP address 172.18.11.254/24

46

Configure the First Leaf Switch Pair

Note: If present, additional leaf switches in the fabric will also need one IP address per
leaf on this network. This is done later in this guide after adding leaf switches to the
SmartFabric.

The IP addresses and gateway are configured as follows:

1. In the SFS UI, select Network > Networks and expand General Purpose
Networks.
2. Select the radio button next to the external management VLAN, 1811 in this
example.
3. Click EDIT.
4. Change the Network Type to Layer 3 General Purpose Network.
5. Next to IP Addresses, enter the range of IP addresses used for the leaf switches.
One address is assigned to each leaf switch. For this example, 172.18.11.252-253
is entered.
6. Enter the Prefix Length and Gateway IP Address from the previous table. When
complete, the window displays as shown in the figure below.

Figure 49. Edit Layer 3 General Purpose Network window

7. Click EDIT to apply the settings.

Optionally, the addresses can be verified by running the show interface virtual network
1811 command at the CLI of the leaf switches. The IP address assigned on Leaf1A is
underlined in the command output below.

Leaf1A# show interface virtual-network 1811

Virtual-network 1811 is up, line protocol is up
Description: virtual-network1811
Address is 3c:2c:30:89:1c:22, Current address is 3c:2c:30:89:1c:22
Interface index is 105
Internet address is 172.18.11.252/24
(output truncated)

BGP Validation
Note: If static routes are used, go to the Configure a jump host port section. (Static route
validation was done earlier in the Validate static route example section of this guide).

If BGP is used on the uplinks, ensure the external switches have learned the routes to
the VxRail External Management network, 172.18.11.0/24 in this example, to reach the
VxRail nodes and VxRail Manager. This is done with the show ip route command. The

47

Configure the First Leaf Switch Pair

BGP-discovered route to 172.18.11.0/24 is underlined in the output below.

Note: The command output shown is for the External-A switch. The output for External-B
is similar. BGP verification from the leaf switches was done in Show command output on
Leaf1A (BGP example).

Note: Command output from a Cisco Nexus switch is shown in the BGP validation on
N9K-External-A during VxRail deployment subsection of the Validate L3 connections to
Cisco Nexus switches section.

External-A# show ip route

Codes: C - connected
S - static
B - BGP, IN - internal BGP, EX - external BGP
O - OSPF, IA - OSPF inter area, N1 - OSPF NSSA external type 1,
N2 - OSPF NSSA external type 2, E1 - OSPF external type 1,
E2 - OSPF external type 2, * - candidate default,
+ - summary route, > - non-active route
Gateway of last resort is not set
Destination Gateway Dist/Metric Change
----------------------------------------------------------------------------------
C 10.0.2.1/32 via 10.0.2.1 loopback0 0/0 18:36:55
B IN 10.0.2.2/32 via 192.168.3.21 200/0 18:16:18
C 172.19.11.0/24 via 172.19.11.252 vlan1911 0/0 18:29:16
B EX 172.18.11.0/24 via 192.168.1.1 20/0 16:02:33
via 192.168.1.3
C 192.168.1.0/31 via 192.168.1.0 ethernet1/1/13 0/0 21:10:53
C 192.168.1.2/31 via 192.168.1.2 ethernet1/1/14 0/0 18:36:56
B IN 192.168.2.0/31 via 192.168.3.21 200/0 21:10:51
B IN 192.168.2.2/31 via 192.168.3.21 200/0 18:16:18
C 192.168.3.20/31 via 192.168.3.20 vlan4000 0/0 18:29:12

Configure a jump host port

Note: The deployment examples in this guide use a jump host connected directly to a
leaf switch. If you are doing a remote deployment without a jump host, skip this section
and see the Remote Deployment of VxRail with SmartFabric Services Deployment
Guide.

VxRail Manager is used for VxRail deployments. The VxRail Manager VM automatically
runs on the primary VxRail node, which is the node with the lowest VxRail serial number.

Note: Before VxRail deployment, VxRail Manager is accessible on an untagged port on

48

Configure the First Leaf Switch Pair

the SFS Client Management VLAN (VLAN 4091 by default). The default IP address is
192.168.10.200. Dell Technologies recommends not changing the VxRail Manager
default address unless it conflicts with your environment, or you are deploying remotely
without a jump host.

VxRail Manager is accessed by connecting a laptop computer or a jump host directly to

any available leaf switch port, as shown in the following figure.

Figure 50. Jump host connected leaf switch for VxRail deployment

This section covers the configuration of a leaf switch port for connection to a jump host
or laptop computer (referred to only as a jump host for the remainder of this guide).

Change native port speed on S5200 series switches

If the jump host has a 1 GbE or 10 GbE NIC, and it is connected to a 25 GbE port on an
S5200 series switch, the switch port used must be changed from its native 25 GbE
speed to 10 GbE for the port to come up.

If the jump host has a 1 GbE or 10 GbE NIC, and is connected to a 10 GbE port on an
S4100 series leaf switch, or has a 25 GbE NIC and connects to an S5200 series leaf
switch, leave the port at its native speed, skip this section, and go to the Configure the
jump host interface section.

Note: When in 10 GbE mode, an S5200 series switch port will autonegotiate to 1 GbE
when connected to a 1 GbE NIC. To connect a jump host with a 1 GbE BASE-T or 10
GbE BASE-T port to an SFP port on the leaf switch, use a Dell supported SFP-1G-T or
SFP-10G-T adapter.

Determine the port group

Determine the port group containing the port that the jump host will use. This is done by
referencing the table below for S5200 series switches, or by running the show port-group
command from the leaf switch CLI. For example, if the jump host is connected to port
1/1/9, it is in port group 1/1/3.

Nativ
Port Non- Non-native
Native physical e Applicable
group native logical interface
interface name spee switches
number speed name
d

49

Configure the First Leaf Switch Pair

 Nativ
25g-
Port Non- Non-native
Native physical e
4x Applicable
group native logical interface
interface name
Eth 1/1/1-1/1/4 spee
25g- switches
number speed name
Eth 1/1/5-1/1/8 d
4x
Eth 1/1/9-1/1/12 25g-
Eth 1/1/13- 4x
1/1/16 25g-
Eth 1/1/17- 4x
1/1/20 25g- S5212/S5224
Eth 1/1/21- 4x /S5248/S529
1/1/24 25g- 6
Eth 1/1/25- 4x S5212/S5224
1/1/28 25g- /S5248/S529
1/1/1 10g-4x Eth 1/1/x:1
Eth 1/1/29- 4x 6
1/1/2 10g-4x Eth 1/1/x:1
1/1/32 25g- S5212/S5224
1/1/3 10g-4x Eth 1/1/x:1
Eth 1/1/33- 4x /S5248/S529
1/1/4 10g-4x Eth 1/1/x:1
1/1/36 25g- 6
1/1/5 10g-4x Eth 1/1/x:1
Eth 1/1/37- 4x S5224/S5248
1/1/6 10g-4x Eth 1/1/x:1
1/1/40 25g- /S5296
1/1/7 10g-4x Eth 1/1/x:1
Eth 1/1/41- 4x S5224/S5248
1/1/8 10g-4x Eth 1/1/x:1
1/1/44 25g- /S5296
1/1/9 10g-4x Eth 1/1/x:1
Eth 1/1/45- 4x S5224/S5248
1/1/10 10g-4x Eth 1/1/x:1
1/1/48 25g- /S5296
1/1/11 10g-4x Eth 1/1/x:1
Eth 1/1/49- 4x S5248/S5296
1/1/12 10g-4x Eth 1/1/x:1
1/1/52 25g- S5248/S5296
1/1/13 10g-4x Eth 1/1/x:1
Eth 1/1/53- 4x S5248/S5296
1/1/14 10g-4x Eth 1/1/x:1
1/1/56 25g- S5248/S5296
1/1/15 10g-4x Eth 1/1/x:1
Eth 1/1/57- 4x S5248/S5296
1/1/16 10g-4x Eth 1/1/x:1
1/1/60 25g- S5248/S5296
1/1/17 10g-4x Eth 1/1/x:1
Eth 1/1/61- 4x S5296
1/1/18 10g-4x Eth 1/1/x:1
1/1/64 25g- S5296
1/1/19 10g-4x Eth 1/1/x:1
Eth 1/1/65- 4x S5296
1/1/20 10g-4x Eth 1/1/x:1
1/1/68 25g- S5296
1/1/21 10g-4x Eth 1/1/x:1
Eth 1/1/69- 4x S5296
1/1/22 10g-4x Eth 1/1/x:1
1/1/72 25g- S5296
1/1/23 10g-4x Eth 1/1/x:1
Eth 1/1/73- 4x S5296
1/1/24 10g-4x Eth 1/1/x:1
1/1/76 25g- S5296
Eth 1/1/77- 4x S5296
1/1/80 25g- S5296
Eth 1/1/81- 4x S5296
1/1/84 25g- S5296
Eth 1/1/85- 4x
1/1/88 25g-
Eth 1/1/89-

50

Configure the First Leaf Switch Pair

1/1/92 4x
Eth 1/1/93- Nativ
25g-
Port Non- Non-native
Native
1/1/96 physical e
4x Applicable
group native logical interface
interface name spee
25g- switches
number speed name
d
4x

51

Configure the First Leaf Switch Pair

Note: Changing the speed is done for all ports in the port group. In this example, setting
port group 1/1/3 to 10g-4x changes ports 1/1/9-1/1/12 to 10 GbE, and the ports are
renamed 1/1/9:1-1/1/12:1.

Change the port-group speed in the SFS web UI

In this section, port group 1/1/3 is changed from its 4x25 G native speed to 4x10 G to
accommodate a jump host connected to port 1/1/9 on Leaf1A.

1. In the SFS UI, select the Switches tab.

2. Under Fabric Switches, expand the switch the jump host is connected to. Leaf1A
is used in this example.
3. Expand Breakout Ports & Jump Port.
4. Select the radio button next to the port group. In this example, port-group1/1/3.

Figure 51. Port-group 1/1/3 selected

5. Click BREAKOUT PORT.

6. Next to Mode, select 4X10GEFixedFormFactor, as shown in the following figure.

Figure 52. Breakout switch ports

7. Click SUBMIT to apply the setting.

8. A green Success message displays as shown.

Figure 53. Port group 1/1/3 configured

Configure the jump host interface

The jump host interface is configured as follows:

1. In the SFS UI, select the Switches tab.

2. Under Fabric Switches, expand the switch the jump host is connected to. Leaf1A
is used in this example.
3. Expand Breakout Ports & Jump Port.
4. Click +JUMP PORT.
5. In the Add Jump Port window:
a. Enter a Name, such as Jump Host 1.
b. Next to Interface Name, select the interface that the jump host is connected to.
Note: Port 1/1/9 was automatically renamed to 1/1/9:1 when its port group was

52

Configure the First Leaf Switch Pair

 changed from its native setting of 4x 25 GbE to 4x 10 GbE.

c. Next to Untagged Network, select Client_Management_Network (VLAN-

4091), as shown in the following figure.

Figure 54. Configure jump port

6. Click ADD to apply the settings. The jump host interface is added as an untagged
member of the Client_Management_Network (VLAN 4091 by default). Optionally,
from the CLI of the leaf switch the jump host is connected to, run the show virtual-
network 4091 command to validate the settings. The jump host port, ethernet
1/1/9:1 in this example, is underlined in the command output below.

Leaf1A# show virtual-network 4091

Codes: DP - MAC-learn Dataplane, CP - MAC-learn Controlplane, UUD - Unknown-Unic
ast-Drop
Virtual Network: 4091
Description: Default untagged network for client onboarding
VLTi-VLAN: 4091
Members:
Untagged: ethernet1/1/1, ethernet1/1/2, ethernet1/1/3, ethernet1/1/9:1
VLAN 4091: port-channel1000
VxLAN Virtual Network Identifier: 4091
Source Interface: loopback2(172.30.0.0)
Remote-VTEPs (flood-list):

Configure the jump host IP addresses

By default, the initial (temporary) VxRail Manager IP address is 192.168.10.200/24, and
it is in VLAN 4091. During VxRail deployment, the VxRail Manager address changes to
its new (permanent) address on the External Management VLAN (VLAN 1811 in this
example). The new VxRail Manager address used in this guide is 172.18.11.72/24 per
the planning data in VxRail deployment settings - first cluster .

During VxRail deployment the jump host must be able to reach both the initial and new
VxRail Manager addresses. So, two addresses are configured on its network adapter -
one for each network.

The IP addresses are configured on the jump host NIC in this example as follows:

192.168.10.201/24, to communicate with the initial VxRail Manager address,

192.168.10.200/24
172.18.11.201/24, to communicate with the new VxRail Manager address,
172.18.11.72/24

53

Configure the First Leaf Switch Pair

Note: Both addresses may be configured simultaneously if the network adapter supports
it, or in sequence if required. During VxRail deployment, the jump host port on the switch
is moved from VLAN 4091 to the External Management VLAN, along with VxRail
Manager. Since the jump host port on the switch is untagged in VLAN 4091, and will also
be untagged in the External Management VLAN, no VLAN information is configured on
the jump host NIC.

Once the jump host has been configured with an IP address on the 192.168.10.0/24
network, verify the jump host can communicate with VxRail Manager by pinging
192.168.10.200 from the jump host.

Figure 55. Jump host successfully pings VxRail Manager

54

Configure the First Leaf Switch Pair