ACI
MIGRATION
AND
IMPLEMENTATION
Rene
Raeber
Dis+nguished
Engineer,
Datacenter
EMEAR
IEEE-802.1
Architect
Mercator
The
ques:on
is:
Migrating from Current to New!
Nexus 9500
Spine
10G
40G BiDi
10G
VTEP
1G
Server
Nexus 9300
TOR
VTEP
VTEP
VTEP
1G &
10G
Server
There must be a middle road!
5
Things we would like to understand how to do
Extend ACI to
WAN/DCI
AVS
vSwitch
Let me just run
my network (but
fix my Flooding,
Mobility,
Configuration,
Troubleshooting
challenges)
AVS
vSwitch
Extend ACI to local
hypervisors
Interconnect to existing
DC Networks
Extend ACI to to existing Nexus
installations via a full ACI VXLAN
Switching Enabled Hypervisor and
remote ACI Physical Leaf
6
The Power of Datacenter Networks
The Power of Datacenter Networks
Guiding Principles
Allow for gradual migration of existing classic topologies they will not
go away overnight
Facilitate the 40Gig market transition
Adopt and allow for integration of overlay technologies such as VXLAN
Consider ACI for Green-Field environments or environments looking for
increased operational flexibility
Hypervisors come in different flavors and encapsulation styles
Still need WAN services!
9
A World of many options
1
2
Border Leafs
VXLAN Based Fabric
AVS
4
VTEP
3
5
VXLAN Enabled
Hypervisor
VTEP
VXLAN Enabled
Hypervisor
Classic POD (Mix of N9K and classic platforms in StandAlone)
ACI based network
VXLAN based hypervisor
Remote Leaf (2H2015)
DCI
Service Interconnect to
ASR9K/N7K WAN/DCI
VTEP
10
Add Nexus 9000 to Existing Nexus 2000-7000 Fabric
Deploy standalone Nexus 9000 into existing Nexus Fabric to add network capacity.
Existing
Nexus 2K-7K
Fabric
What you get:
Nexus 9000 Switches
40 Gig capability with QSA for backward capability
Programmability through various APIs (python/puppet/chef)
Nexus 9000
N1Kv
Leverage existing APIs, cloud orchestration/automation tools
Power savings and lower TCO, specifically in N9500 chassis
11
Add an ACI POD
Deploy ACI Fabric in parallel with existing Nexus Fabric. Connect via L2/L3.
ACI
Fabric
Existing
Nexus 2K-7K
Fabric
APIC
L2 or L3 Connection
Nexus 9000
N1Kv
12
Extending ACI Policy to Servers on Existing Fabric
Deploy/upgrade AVS & Remote Leaf N9300 in existing Nexus Fabric.
Extend ACI Policy model over existing Nexus Fabric, allowing apps on existing Nexus Fabric to realize benefits of ACI.
ACI
Fabric
Existing
Nexus 2K-7K
Fabric
APIC
L2 or L3 Connection
PROFILE
*Nexus
9300
Nexus
9000
PROFILE
PROFILE
AVS
*Remote Leaf s/w 2H CY15
ACI POLICY
AVS
13
INTEGRATION / MIGRATION
REMOTE VTEP (PHYSICAL) VIA NEXUS 9300
Classical L2
ACI Infra / L3
Why is this extra box in the middle ?
One could connect the ACI spines to
the pair of N7K Aggregation switches
as full mesh. Still the same results,
but harder to scale when adding more
ACI spines;
Cabling mismatch (40GE on the ACI
side and 10GE on the Nexus side);
Route within the fabric for full remote
VTEP switching (versus route via
outside from the border leaf);
VTEPExternal
AVS
OVS
Why are these links called ACI Infra ?
By the time of vLeaf full switching
\support, the ACI Infra links will be
used to bootstrap the remote VTEP
(physical or virtual);
The APIC VTEP address is then only
reachable through the link at the
spines (and not via Border Leaf)
14
INTEGRATION / MIGRATION
REMOTE VTEP (PHYSICAL) VIA NEXUS 9300
ACI Spines primary forwarding related features are:
- Directory/Proxy Service;
- Multicast Root;
IP Forwarding
- IP Forwarder;
Classical L2
ACI Infra / L3
Directory/Proxy & Multicast root
Services Located in ACI Spine
VTEPExternal
AVS
OVS
15
INTEGRATION / MIGRATION
REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING
What changes ?
- VTEP internal;
- ACI Infra at remote links;
Classical L2
ACI Infra / L3
VTEPInternal
AVS
OVS
16
INTEGRATION / MIGRATION
REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING
Classical L2
ACI Infra / L3
Grow the ACI Fabric as needed
AVS
OVS
17
INTEGRATION / MIGRATION
REMOTE VTEP (PHYSICAL AND VIRTUAL) FULL ACI SWITCHING H1CY15
Classical L2
ACI Infra / L3
and add further services and nodes at ACI
AVS
OVS
AVS
OVS
OpenStack KVM
Juno (basic)
K release (full)
FCS
FCS
18
19
Integration Scenario
Customer Selected ACI for his existing workloads.
Need to interconnect ACI to the existing infrastructure
Need to move (migrate) workloads
Very likely scenario & needs to be easy.
20
The Migration Steps
1.
Extend L2 into ACI
2.
Configure ACI for this L2 extension
3.
Create new EPG and contracts for the workloads to move into
4.
Move Workloads
5.
Move HSRP Default Gateway over to ACI
6.
Turn off the Existing Network
Easy.
21
Step 1: Connect Fabric to Existing Network
Functionally we are expanding the VLANs into ACI.
Existing Design
ACI Fabric
HSRP
Default GW
VLAN 10 / Subnet 10
EPG-10 = VLAN 10
VM
VM
VM
22
Step 2: Connect Fabric to Existing Network
The ACI Infra Admin creates the Leaf interface policy (speed,
CDP, LLDP etc) for the port.
The ACI Tenant Admin uses that port for the migration (see later).
APIC
Existing Design
Lets call this Tenant Red
HSRP
Default GW
VLAN 10 / Subnet 10
VM
VM
VM
23
Now its virtual !
vvvvvvvvvvv
24
Step 3: Configure ACI in preparation for the migration
(EPG equals VLAN)
Tenant Red
Context Red
Always need a Tenant & Context
For the migration:
Bridge Domain 10
Subnet 10
EPG-10
Bridge Domain 20
Subnet 20
EPG-20
create a Bridge-Domain for each
VLAN & define the subnet.
Create EPG and assign it the correct
subnet and VLAN.
Per Bridge-Domain:
We dont want ACI to route this subnet
yet, the existing HSRP gateways
remain the default gateway for now.
Disable Unicast Routing and Enable
flooding
25
Step 3 (continued): Configure ACI Bridge Domain settings
Tenant Red
Temporary Bridge Domain specific
settings while we are using the
HSRP gateways in the existing
network.
Select Forwarding to be Custom
which allow
Context Red
Bridge Domain 10
Subnet 10
EPG-10
Enable Flooding of L2 unknown unicast
Enable ARP flooding
Disable Unicast routing
26
Step 3 (cont.): Create EPG
EPG=VLAN model
Create EPG
Link it to the right vCenter (VMM)
This allows APIC to create DVS
switches on ESXi and ensures
correct signaling between APIC/
vCenter
Connect EPG to the port connected
to existing network. Specify VLAN.
Interface Policy was already set by
ACI Infra Admin in Step 2.
Bridge Domain 10
Subnet 10
EPG-10
VMM Domain - vCenter
Static Binding to port (vlan-10)
Interface Policy
Leaf 2
Port 3
27
Step 3 (cont.): Create EPG (expanding to multiple EPGs)
EPG=VLAN model
Bridge Domain 20
Bridge Domain 10
Subnet 10
EPG-10
Subnet 20
EPG-20
VMM Domain - vCenter
VMM Domain - vCenter
Static Binding to port (vlan-10)
Static Binding to port (vlan-20)
Interface Policy
Leaf 2
Port 3 (trunk)
28
Step 4: Migrate Workloads
APIC point of view, the policy model
EPG 10
P
VM
VM
VM
APIC
VMs will need to be connected to new Port
Group under APIC control (AVS or DVS).
Existing Design
HSRP
Default GW
VLAN 10 / Subnet A
VM
VM
VM
29
Step 5: Complete the Migration
Change BD settings back to normal for ACI mode
Change BD settings back to default.
No Flooding
Unicast Routing enabled.
30
FEX Topology Support Roadmap
Standalone and ACI
Active/Standby
Teaming
Straight Through
(Single Homed)
vPC (Dual Homed)
EvPC
Nexus 9300
Standalone
6.1(2)I2(3)
6.1(2)I2(3)
Target 1HCY15
Future
Nexus 9300
ACI Leaf
Supported at FCS
Brahmaputra 1HCY15
Target 2HCY15
Future
31
Organization Implications
Cisco Infrastructure Team Journey
STORAGE
SECURITY
NETWORK
ARCHITECTURE
DESIGN
IMPLEMENTATION
OPERATIONS
Network
Virtual Teams
COMPUTE
UC/Video
Infrastructure as a Service
32
32
APIC Screen shots
33
Normative
ACI Application Centric Infrastructure
APIC Application Policy Infrastructure Controller
DFA Distributed Fabric Automation
VDP Virtual Station Interface Discovery Protocol
VXLAN - Virtual eXtensible Local Area Network
VXLAN Segment - VXLAN Layer 2 overlay network over which VMs communicate
VXLAN Overlay Network - another term for VXLAN Segment
VXLAN Gateway - an entity which forwards traffic between VXLAN and non-VXLAN environments
VTEP - VXLAN Tunnel End Point - an entity which originates and/or terminates VXLAN tunnels
VLAN - Virtual Local Area Network
VM - Virtual Machine
VNI - VXLAN Network Identifier (or VXLAN Segment ID)
ACL - Access Control List
ECMP - Equal Cost Multipath
IGMP - Internet Group Management Protocol
PIM - Protocol Independent Multicast
SPB - Shortest Path Bridging
ToR - Top of Rack
TRILL - Transparent Interconnection of Lots of Links
34
