BRKARC-3445 Catalyst 4500E Deep Dive

Kedar Karmarkar
Technical Leader

Abstract
Cisco Catalyst 4500E Series modular switches enable a high performance, mobile, and secure user experience by optimizing application performance through deep visibility, unifying policy and enabling pervasive confidentiality, and driving network and service virtualization.

The Catalyst 4500E lowers total cost of ownership through maximum resiliency, In Service Software Upgrades, automation, and unparalleled investment protection.
This section will cover in depth look at the architecture and positioning of Catalyst 4500E platform and shows how Catalyst 4500E can be deployed in campus access and aggregation.

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Catalyst 4500E Chassis and Power Supplies

Catalyst 4500E Family

4507R+E
2 Supervisors 5 Line Cards

4510R+E
2 Supervisor 8 Line Cards

4503-E
1 Supervisor 2 Line Cards

4506-E
1 Supervisor 5 Line Cards

240 Ports of 10/100/1000 11 Rack Unit Height

Supervisor 6LE, 6E, 7E

384 Ports of 10/100/1000 14 Rack Unit Height

Supervisor 6E, 7E

96 Ports of 10/100/1000 7 Rack Unit Height

240 Ports of 10/100/1000 10 Rack Unit Height

Supervisor 6LE, 6E, 7E Dual Power Supplies

Cisco Public

Dual Power Supplies

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved.

Per Slot Bandwidth in 10 and 7 Slot Chassis

24G 24G 24G 24G Supervisor 6-E Supervisor 6-E 24G 6G 6G 6G 24G 24G 24G 24G Supervisor 7-E Supervisor 7-E 24G 24G 24G 24G 48G 48G 48G 48G Supervisor 7-E Supervisor 7-E 48G 48G 48G 48G

WS-C4510R-E
24G 24G Supervisor 6/6L-E Supervisor 6/6L-E 24G 24G 24G

WS-C4510R-E
24G 24G Supervisor 7-E Supervisor 7-E 24G 24G 24G

WS-C4510R+E
48G 48G Supervisor 7-E Supervisor 7-E 48G 48G 48G

WS-C4507R-E
BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

WS-C4507R-E
Cisco Public

WS-C4507R+E

Catalyst 4500E 4200 W Power Supply

Voltage
110 V

Inputs
Single Dual

Redundancy Mode
Redundant Combined Redundant

Max PoE (15W) Devices

54 98 109

Max PoEP (30W) Devices

27 50 56

Combined
220 V Single Dual Redundant Combined Redundant Combined
TECCRS-2045 2011 Cisco and/or its affiliates. All rights reserved.

198
109 198 218 384
Cisco Public

102
56 102 112 204
7

Catalyst 4500E 6000 W Power Supply

Voltage
110 V

Input s
Single Dual

Redundancy Mode
Redundant Combined Redundant

Max PoE (15W) Devices

54 98 109

Max PoEP (30W) Devices

27 50 56

Combined
220 V Single Dual Redundant Combined Redundant Combined
TECCRS-2045 2011 Cisco and/or its affiliates. All rights reserved.

198
141 257 283 384
Cisco Public

102
72 132 145 262
8

Cisco Power Calculator

http://tools.cisco.com/cpc

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Catalyst 4500E Supervisors

Catalyst 4500E Supervisor 7-E

Hardware Elements
2G DRAM Dual Core CPU

USB ports* SD Memory Card

Console and Management Port

BRKARC-3445

4 Uplinks

10GE with SFP+ 1GE with SFP

Cisco Public

2011 Cisco and/or its affiliates. All rights reserved.

*USB Type A is supported USB Type B is not supported

11

Catalyst 4500E Supervisor Comparison

Performance

Bandwidth: 280 Gbps Uplinks: 2x10G/4x1G CPU: 1 GHz DRAM: 512 MB Max Routes: 57K

Bandwidth: 320 Gbps Uplinks: 2x10G/4x1G CPU: 1.3 GHz DRAM: 512 MB Max Routes: 256 K

Bandwidth: 848 Gbps Uplinks: 4x10G/4x1G CPU: Dual Core 1.5 GHz DRAM: 2 GB Max Routes: 256 K

Supervisor 6L-E
BRKARC-3445

Supervisor 6-E
Cisco Public

Supervisor 7-E
12

2011 Cisco and/or its affiliates. All rights reserved.

Catalyst 4500E Line Cards

Cisco Catalyst 4500E 10/100/1000 Line Cards

WS-X4648-RJ45V+E WS-X4748-RJ45V+E

PoE Plus Linecards

E-Series (24G/slot) 48p 10/100/1000 RJ45 30W/ port (IEEE802.3at standard PoEP) on upto 24 ports Re-use existing chassis, power supplies PoE policing and monitoring EnergyWise Jumbo frame support

E-Series (48G/ slot) 48p 10/100/1000 RJ45 30W/ port (IEEE802.3at standard PoE-P) on 48 ports IEEE 802.1AE MACSec on all ports Re-use existing chassis, power supplies EnergyWise Jumbo frame support

WS-X4648-RJ45-E

WS-X4748-RJ45V-E

Data

E-Series (24G/slot) 48p 10/100/1000 RJ45 E-series Supervisors only Jumbo frame support

E-Series (48G/ slot) 48p 10/100/1000 RJ45 Energy Efficient Ethernet (EEE) 802.3az IEEE 802.1AE MACSec on all ports Jumbo Frame support

24G (E-Series)
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

48G (E-Series)
14

UPOE
WS-X4748-UPOE+E

UPOE
60W PoE with max. line card budget of 1500W Estimate Cable loss with intelligent diagnostics LLDP enhancement to negotiate beyond 30W Power X-Generation applications IP Turrets in financial trading floors Integrated Virtual Desktop Clients

Energy Efficient Ethernet

Compliant with IEEE 802.3az for: 100/1000 Base-T Power consumption is based on link utilization Green: Save up to 1W per link Mandatory for Energy Star Compliance*

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

15

Energy Efficient Ethernet (EEE)

Typical Server Client Traffic Profile
Utilization %
100

What does EEE do?

End Point
Application

Switch
OS
Controller SW

Burst

Burst

OS
Controller SW

Controller

Controller

Time

MAC MAC Energy Efficient Ethernet PHY

EEE

PHY

Ethernet Port is powered on all the time

Ethernet Port Power is optimized

1 Gbps Port Power Consumption No EEE 1.0 W

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved.

Wake time: ~16us

EEE 0.47W
Cisco Public

16

Energy Efficient Ethernet

4510_Sup7E#show interface gi 1/2 capabilities GigabitEthernet1/2 Model: WS-X4748-UPOE+E Type: 10/100/1000-TX Speed: 10,100,1000,auto Duplex: half,full,auto Auto-MDIX: yes EEE: yes ( 100-Tx and 1000-T auto mode ) ..<SNIP>.. 4510_Sup7E#conf t Enter configuration commands, one per line. End with CNTL/Z. 4510_Sup7E(config)#int gi 1/2 4510_Sup7E(config-if)#power efficient-ethernet auto 4510_Sup7E#show platform software interface gi 1/2 status Switch Phyport Gi1/2 Software Status EEE: Operational
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

Determine EEE Capability

Configure EEE

Verify EEE

17

Cisco Catalyst 4500E Fiber 1G/10G Line Cards

WS-X4624-SFP-E WS-X4712-SFP+E

Density

24 ports 1:1 GE SX, LX GE SFP optics

12 ports 2.5: 1, 10GE Mix and match 10G/ 1GE with SFP+ IEEE 802.1AE MacSec on all ports GLC-T, LR, ER, SR, CX1 and LRM SFP+ Optics WS-X4606-X2-E

WS-X4612-SFP-E

Density

12 ports, 1:1 GE SFP SX, LX GE SFP optics

6 ports, 2.5:1 10G Mix and match 10G/1GE with X2 (Twin-gig) LR, ER, SR, LX4 and LRM X2 optics

GE Fiber
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

10G Fiber
18

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

19

IOS XE Architecture
IOS-XE
Modern IOS to enable multi-core CPU Easy customer migration while maintaining IOS functionality and look and feel Allow hosted applications like Wireshark

Enabling Integrated Open Service Platform

IOS 15.0(2)SG IOS

Components

IOS XE 3.2.0SG IOSd Features

Components

Features

Hosted Apps

Infra

Drivers Kernels

Mgmt

Common Infrastructure / HA Management Interface Module Drivers Kernel

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

20

IOS XE Architecture
API

Common Management

Complete separation of control and data plane

Service Integration through solid API framework

Vastly improved resiliency between applications and IOSd

Hosted Apps / Services

IOSd

Connected Apps

Platform Specific Code Drivers, Dataplane

Operational Infrastructure
Linux Kernel Learn more about IOS XE in BRKARC-2007 - IOS Strategy and Evolution
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

21

Containerization

Why Wireshark on the SUP7-E?

SPAN/RSPAN Packet Forward capability No local display Need external PC/sniffer to store and decode Sits in between debug ip packet and SPAN/RSPAN

Wireshark Freeware Supports wide variety of protocols Bundled with switch Operating System Onboard Capture and decode tool Quick Analysis

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

22

Wireshark Capabilities
IOS XE on SUP7-E can host third-party apps.
Wireshark is a software process Capture filters Display filters Store packets in PCAP file that user can manually TFTP/SSH to remote server. Support for multiple active capture points

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

23

How is it done?
Local Display
View PCAP from Remote Server

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

Sample Packet Capture displays

Display packets in brief mode
Switch# show monitor capture file bootflash:mycapture.pcap
1 2 3 0.000000 0.000000 0.000000 192.85.1.3 -> 192.85.1.4 192.85.1.3 -> 192.85.1.4 192.85.1.3 -> 192.85.1.4 UDP Source port: 1024 UDP Source port: 1024 UDP Source port: 1024 Destination port: 28960 Destination port: 28960 Destination port: 28960

0.000000

192.85.1.3 -> 192.85.1.4

UDP Source port: 1024

Destination port: 28960

Display packets in hexadecimal mode

Switch# show monitor capture file bootflash:mycapture.pcap dump
0000 0010 0020 00 00 94 00 00 04 00 00 94 00 00 03 08 00 45 c0 05 1e 0f 28 00 00 ff 11 24 35 c0 55 01 03 c0 55 01 04 04 00 71 20 05 0a db 21 00 00 00 00 00 00 ..............E. ...(....$5.U...U ....q ...!......

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

25

Sample Packet Capture Displays

Display packets in detailed mode
Switch# show monitor capture file bootflash:mycapture.pcap detailed
Frame 1: 1328 bytes on wire (10624 bits), 1328 bytes captured (10624 bits) Arrival Time: Jan 1, 1970 00:00:00.000000000 Universal Epoch Time: 0.000000000 seconds <snip.snip> Frame Number: 1 Frame Length: 1328 bytes (10624 bits) Capture Length: 1328 bytes (10624 bits) <snip.snip> [Protocols in frame: eth:ip:udp:data] Ethernet II, Src: 00:00:94:00:00:03 (00:00:94:00:00:03), Dst: 00:00:94:00:00:04 (00:00:94:00:00:04) Destination: 00:00:94:00:00:04 (00:00:94:00:00:04) Address: 00:00:94:00:00:04 (00:00:94:00:00:04) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Source: 00:00:94:00:00:03 (00:00:94:00:00:03) Address: 00:00:94:00:00:03 (00:00:94:00:00:03) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) Type: IP (0x0800) Frame check sequence: 0x99c15111 [incorrect, should be 0x379d10df] Internet Protocol, Src: 192.85.1.3 (192.85.1.3), Dst: 192.85.1.4 (192.85.1.4)

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

Sample Packet Capture Displays

Internet Protocol, Src: 192.85.1.3 (192.85.1.3), Dst: 192.85.1.4 (192.85.1.4) Version: 4 Header length: 20 bytes Differentiated Services Field: 0xc0 (DSCP 0x30: Class Selector 6; ECN: 0x00) 1100 00.. = Differentiated Services Codepoint: Class Selector 6 (0x30) .... ..0. = ECN-Capable Transport (ECT): 0 .... ...0 = ECN-CE: 0 Total Length: 1310 Identification: 0x0f28 (3880) Flags: 0x00 0... .... = Reserved bit: Not set .0.. .... = Don't fragment: Not set ..0. .... = More fragments: Not set Fragment offset: 0 Time to live: 255 Protocol: UDP (17) Header checksum: 0x2435 [correct] [Good: True] [Bad: False] Source: 192.85.1.3 (192.85.1.3) Destination: 192.85.1.4 (192.85.1.4) User Datagram Protocol, Src Port: 1024 (1024), Dst Port: 28960 (28960) Source port: 1024 (1024) Destination port: 28960 (28960) Length: 1290 Checksum: 0xdb21 [validation disabled] [Good Checksum: False] [Bad Checksum: False] <snip.snip>
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

27

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

28

Catalyst 4500E Architecture

Centralized Architecture
Shared memory switch Passive Backplane

CPU / SDRAM

All forwarding, queuing, security is implemented on the Supervisor The individual line cards are considered to be transparent and contain stub ASICs and the PHYs Upgrade advantages

Forwarding Engine
Packet Processing Engine

Each 47XX-Series line card has 48 Gbps full- duplex connections to the central forwarding engine
IOS XE that can leverage multi-core CPU, and ability to host applications separately outside IOS context

Line Card Line Card Line Card Line Card Line Card

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

Catalyst 4500E Line Card Architecture

46xx and 47xx Line Card Backplane Speeds

Switch Backplane
3Gbps 3Gbps 3Gbps 3Gbps 3Gbps 3Gbps

E-Series ChassisBandwidth per Slot with 46XX series line card: 8 dedicated lanes to Supervisor
3Gbps
3Gbps

Each lane operates at 3Gbps

E-Series 46xx Line Card

Switch Backplane
6Gbps 6Gbps 6Gbps 6Gbps 6Gbps 6Gbps

E-Series ChassisBandwidth per Slot with 47xx series line cards 8 dedicated lanes to Supervisor Each lane runs at 6Gbps

E-Series 47xx Line Card

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

6Gbps

6Gbps

31

WS-X4648-RJ45V+E Block Diagram

Switch Backplane

2x3G

2x3G

2x3G

2x3G

Management FPGA

Traffic Sub-System

PoEP Sub-System

Stub ASIC

Stub ASIC

Stub ASIC

Stub ASIC

Power Brick

Power Brick

Power Brick
PoEP Module

PoEP Module
Octal PHY Octal PHY Octal PHY Octal PHY Octal PHY Octal PHY

15.4W/30W
RJ45 Ports 37-42 43-48
Cisco Public

RJ45 Ports 1-6 7-12

BRKARC-3445

RJ45 Ports 13-18 19-24

RJ45 Ports 25-30 31-36

2011 Cisco and/or its affiliates. All rights reserved.

32

WS-X4748-RJ45V+E Block Diagram

Switch Backplane

2x6G

2x6G

2x6G

2x6G

Management FPGA

Traffic Sub-System Stub ASIC MACSec Stub ASIC MACSec Stub ASIC MACSec Stub ASIC MACSec

PoEP Sub-System

Power Brick

Power Brick

Power Brick
PoEP Module

PoEP Module
Octal PHY Octal PHY Octal PHY Octal PHY Octal PHY Octal PHY

15.4W/30W
All ports are non-blocking at packet size over 200 bytes
Cisco Public

RJ45 Ports 1-6 7-12

RJ45 Ports 13-18 19-24

RJ45 Ports 25-30 31-36

RJ45 Ports 37-42 43-48

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

33

WS-X4712-SFP+E Block Diagram

Switch Backplane

2x6G

2x6G

2x6G

2x6G

Management FPGA

Traffic Sub-System Stub ASIC MACSec

Dual PHY

Stub ASIC MACSec

Dual PHY

Stub ASIC MACSec

Dual PHY

Stub ASIC MACSec

Dual PHY

SFP/SFP+ 1-3
BRKARC-3445

SFP/SFP+ 4-6

SFP/SFP+ 7-9

SFP/SFP+ 10-12
Cisco Public

1x10G + 2x1G For non-blocking performance for packets over 200 bytes
34

2011 Cisco and/or its affiliates. All rights reserved.

Catalyst 4500E Supervisor Architecture

WS-X45-SUP7-E Block Diagram

48G Switch Backplane

Packet Processor

10G

FPGA

1.5GHz CPU

Forwarding Engine

FPGA

SDRAM

2x12G 2x12G
Stub ASIC
Stub ASIC

NetFlow ASIC
BRKARC-3445

SFP SFP+

SFP SFP+

SFP SFP+

SFP SFP+
Cisco Public

SD

USB

Console Mgmt
36

2011 Cisco and/or its affiliates. All rights reserved.

Supervisor 7-E Forwarding Engine Blocks

Packet Lookup Descriptor Netflow Lookup Descriptor

S W I T C H B A C K P L A N E

Netflow Results Descriptor Packet Transmit Descriptor STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Packet Memory

PLD

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

Supervisor 7-E
S W I T C H B A C K P L A N E

Packet Memory and Queue Memory

Input Classification TCAM
Forwarding Lookup TCAM Forwarding Lookup Memory

STP Lookup

Packet Memory

PLD

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

Supervisor 7-E

Packet Memory and Queue Memory

Queue Memory
Queue 1 Queue 1 Queue 1 Queue 1 Queue 2 Queue 2 Queue N Q Entry 1

Packet Memory 32MB 128K of 256-byte cells

Q Entry 2
Q Entry 3 Q Entry 4 Q Entry 1 Q Entry 2

Q Entry N

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

39

Queue Memory
1 1 2 2 3 3 4 4

Queue Entries Allocation

CPU/Drop/Recirculation queue entries allocated Queue entry allocation for slots is divided by number of slots * Linecards divide entries equally per port Port entries divided equally per queue
Recirculation Pool 24K Q Entries Available Pool Reserve Pool 100K Q Entries 860K Q Entries

CPU/DROP 8K/8K Q Entries Line Card 1 Line Card 2 Line Card 3 Line Card 4 Line Card 5
*

Active Supervisor Port 1 Configurable allocation between queues Port 2

Queue 1

Queue 2 ...
Queue 8

Port 3
Port x

For redundant chassis , sup slots are counted as 1 slot

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

40

Queue Structures and Size

on different Supervisors

Supervisor Engines

Egress Queue and Drop Thresholds

Total Buffer Size

Total Queue Entries (Packets) per system

Queue Entries (Packets) per Default Queue per Port*

WS-X4516-10GE

1p3q1t

16MB

256K

260 Per Queue for all 4 queues. 915

WS-X45-SUP6-E

1p7q1t

17.5MB

512 K

WS-X45-SUP7-E

1p7q1t

32MB

1 Million

2079

* - configuration based on a 7-slot system with one 48-port Linecard

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

41

Catalyst 4500E Packet Walk

Supervisor 7-E Packet Packet Reception

S W I T C H B A C K P L A N E
Header Payload
PLD

STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

43

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Pass PLD to FE

S W I T C H B A C K P L A N E
Payload
PLD

STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Header

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

44

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk L2 Lookup

S W I T C H B A C K P L A N E L2 Lookup
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Payload

PLD

Header

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

45

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Input ACL/QoS

S W I T C H B A C K P L A N E
Payload

Input ACL/Marking
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

PLD

Header

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

46

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Netflow Lookup

NetFlow Lookup
STP Lookup

S W I T C H B A C K P L A N E
Payload
PLD

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor

Header
PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

47

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Input Policing

S W I T C H B A C K P L A N E Input Policing
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Payload

PLD

NLD

Packet Processor

Header
PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

48

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk - Layer 3 Lookup

Forwarding Lookup
STP Lookup

S W I T C H B A C K P L A N E

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Payload

PLD

Header

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

49

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Output ACL/QoS

S W I T C H B A C K P L A N E Output ACL/Marking
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Payload

PLD

NLD

Packet Processor

PTD

Forwarding Engine
Header

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

50

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Output Policing

S W I T C H B A C K P L A N E
Payload
PLD

Output Policing
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor

PTD

Forwarding Engine
Header

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

51

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk DBL Processing

S W I T C H B A C K P L A N E
Payload
PLD

Active Queue Mgmt avoids Congestion on Tx Qs, while protecting lo-bandwidth flows
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor

PTD

Forwarding Engine
Header

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

52

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk - Enqueue

S W I T C H B A C K P L A N E
Payload
PLD

Enqueue in Queue Memory

STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor

PTD

Forwarding Engine
Header

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

53

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Header to PP

S W I T C H B A C K P L A N E
Payload
PLD

STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NUD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Header

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

54

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Packet Rewrite

S W I T C H B A C K P L A N E Packet Header is rewritten
STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

Packet Payload Memory

PLD

NLD

Packet Processor
Header

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

55

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Packet Walk Attach Payload

S W I T C H B A C K P L A N E
Packet Memory
PLD

STP Lookup

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

NLD

Packet Processor
Header Payload

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

56

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Supervisor 7-E Forwarding Engine Blocks

S W I T C H B A C K P L A N E
Input Classification TCAM
5
Forwarding Lookup TCAM

STP Lookup 1 2

Forwarding Lookup Memory

4
PLD 3

8
NLD

Packet Memory

Packet Processor
14

PTD 13

Forwarding Engine
10 11 7

NetFlow ASIC
NRD

Queue Memory
12

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

57

Supervisor 7-E Unicast Packet Walk

1. 2. A packet enters the PHY in the line module and travels across the backplane before reaching the supervisor The packet enters the Supervisor and the Packet Processor performs parsing of VLAN tag and header and stores the packet into Packet Memory

3.
4.

The stripped header is used to construct a Packet Lookup Descriptor (PLD) and forwarded to the Forwarding Engine ASIC
The packet goes through L2 lookup. Spanning tree state is checked. Packet MAC source and MAC destination together with receive vlan ID are looked up in the L2 Hash Table. L2 lookup also determines whether the packet is destined for router functionality.

5.
6.

Input Classification is used to classify the packet via rules loaded into the Input Classification TCAM. ICC stores input ACL and QoS rules in TCAM4
A NLD (Netflow Lookup Descriptor) is created by the Forwarding Engine and fed into the NetFlow ASIC. Here new flow is created or updated; also microflow policing is done here.

7.

NRD (Netflow Result Descriptor) is created by NetFlow ASIC and passed to the Forwarding Engine ASIC. Input Aggregate policing result from VFE and Ingress Microflow policing result from NetFlow ASIC are merged, and packet policed accordingly.
Header is looked up in the FLC for L3 Lookup. FLC stores L3 (or L2 lookup) forwarding and unicast RPF check rules. Contains mainly IPv4 and IPv6 FIB entries.
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

8.

58

Supervisor 7-E Unicast Packet Walk

9.

OCC stores output ACL and QoS rules in TCAM4

10. Output policing is done at this stage. 11. DBL Hashing Memory is algorithm for avoiding congestion in the ASIC. 12. The transmit descriptor is enqueued in the queue memory 13. Packet Transmit Descriptor (PTD) is sent to the Packet Processor. A NetFlow Update Descriptor (NUD) is sent by the Forwarding Engine to the NetFlow ASIC to update Transmit Statistics for that flow.

14. Packet Processor transmits the packet across the backplane to the correct egress line card.

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

59

Supervisor 7-E Multicast Architecture

S W I T C H B A C K P L A N E
Input Classification TCAM
Forwarding Lookup TCAM

STP Lookup

Forwarding Lookup Memory

Packet Memory

PLD
Replication Queue

NLD

Packet Processor

PTD

Forwarding Engine
Replication Module

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

60

Supervisor 7-E Multicast Forwarding

S W I T C H B A C K P L A N E
9 8

STP Lookup 1

Input Classification TCAM

Forwarding Lookup TCAM

Forwarding Lookup Memory

2 NLD 5

Packet Memory

PLD
Replication Queue

Packet Processor

PTD

Forwarding Engine
Replication Module

NetFlow ASIC
NRD

Queue Memory
7

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

61

Supervisor 7-E Multicast Packet Walk

1. Packet comes in as a Source packet. The payload is copied to packet memory, and the small header or Descriptor is stripped off. The L2 table will indicate that the DMAC is a multicast MAC. The packet will be processed pretty much like a unicast packet would. 2.

3.

At some point, during the Forwarding Lookup, the destination Multicast Group address will be looked up. This will point to an Adjacency Entry in the FLC, which points to a RET Entry, in the Replication Table.
The REM consults the RET Table, it stores the information as to how many copies of this Descriptor need to be created and what are the forwarding interfaces for each copy of the Descriptor. The REM creates the Header Copies and enqueues them in the Replication Request Queue. This Descriptor traverses through the Forwarding Engine like before, but none of the Ingress Processing including Forwarding Lookups are done. It proceeds straight to OCC for applying egress features on each of those OIFs. Once the features are applied and the packets are permitted out the OIF they are enqueued into the Queue Memory. The copies are then forwarded to their respective OIFs.

4.

5. 6.

7. 8.

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

62

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

Why NetFlow?
Bandwidth/Capacity Reports

What is eating up my network resources? When do I need a capacity upgrade? What is causing congestion?

Subscriber Demographic Reports

What percentage is using P2P/gaming application? What are the usage patterns of different subscriber groups? What is the cost impact of my top subscribers?

Server Activity

What are the popular Web hosts used? What are the popular streaming sites?

Security Reports

Which subscribers are infected and attacking others? Which subscribers are spamming? Which subscriber is attacking network resources?

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

64

Supervisor 7-E Forwarding Engine Block

S W I T C H B A C K P L A N E
Input Classification TCAM
Forwarding Lookup TCAM

STP Lookup

Forwarding Lookup Memory

Packet Memory

PLD

NLD

Packet Processor

PTD

Forwarding Engine

NetFlow ASIC
NRD

Queue Memory

DHM

Replication Table

Output Classification TCAM

Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

65

NetFlow ASIC Architecture and Components

Selects Flow Mask FLT contains Flow Key Index FLT also contains flow entries Table Rx and Tx Statistics

Police, ACL drop, Redirection

Flow Key Index

Entry Entry Entry Statistics Statistics Statistics

NetFlow ASIC
Mask Mask Actions Actions

Flow Key Index Flow Key Index

64K Flow Classification Table

128K Flow Lookup Table

128K Flow Descriptor Table

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

NetFlow ASIC Architecture and Components

16 entries per Bucket 16 x 8k = 128K 0 1 Bucket
Flow Entry and Statistics Flow Entry and Statistics

1 Entry

Flow Entry and Statistics

8K Buckets

Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics

.. .. .. .. .. .. .. ..
Cisco Public

15
Flow Entry and Statistics Flow Entry and Statistics

Flow Entry and Statistics Flow Entry and Statistics

Flow Entry and Statistics

Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics

Flow Entry and Statistics

Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics Flow Entry and Statistics

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

67

Traditional NetFlow vs. Flexible NetFlow

Traditional NetFlow
SrcIf Fa1/0 Fa1/0 Fa1/0 Fa1/0 SrcIPadd DstIf 173.100.21.2 Fa0/0 173.100.3.2 Fa0/0 173.100.20.2 Fa0/0 173.100.6.2 Fa0/0 DstIPadd Protocol 10.0.227.12 11 10.0.227.12 6 10.0.227.12 11 10.0.227.12 6 SrcPort DstPort 00A2 00A2 15 15 00A1 00A1 19 19

Fixed 7 keys

Export

NetFlow Cache

Flexible NetFlow Flow Monitor 1

DstIPadd 10.0.227.12 10.0.227.12 10.0.227.12 10.0.227.12

Flow cache 1

Flow cache 2
Protocol 11 6 11 6 SrcIf Fa1/0 Fa1/0 Fa1/0 Fa1/0

Protocol 11 6 11 6

TOS 80 40 80 40

Export

Destination 1

Flow Monitor 2

Flow cache 3

TOS 80 40 80 40

Flgs 10 0 10 0 DstIf Fa0/0 Fa0/0 Fa0/0 Fa0/0

Export

Destination 2

Flow Monitor 3

SrcIPadd 173.100.21.2 173.100.3.2 173.100.20.2 173.100.6.2

Export

Destination 3

Virtual Cache For each monitor

BRKARC-3445

Only Interesting Information

Cisco Public

Monitor Types of Protocols

68

2011 Cisco and/or its affiliates. All rights reserved.

Flexible NetFlow on Catalyst 4500E

System Scalability. Up to ~100K (with 85% utilization

efficiency) cached flows for Forwarding Engine bridged flows

Bridged NetFlow. Capability of creating and tracking TCP Flags are now exported as part of the flow information.

Very useful to understand TCP flow directions and to detect denial of service attacks

Export version 9 (the most flexible) and version 5

(legacy)supported

Flexible NetFlow CLI look & feel

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

69

Flexible NetFlow Record: Key Fields

Based on Catalyst 4500 Supervisor 7-E at FCS

Interface
Input

IPv4
Source IP address Destination IP address

IPv6
Source IP address

Transport
ICMP Code ICMP Type IGMP Type TCP Source Port TCP Destination Port UDP Source Port UDP Destination Port

Destination IP address
Protocol Traffic Class Flow Label Total Length Extension Headers** DSCP Next-header* Hop-Limit

Layer 2
Dot1q priority Dot1q Vlan ID Source MAC address Destination MAC address

Protocol Precedence

DSCP
TTL Total Length

--- New Key Fields in FnF

Only first header is reported ** TBD
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved.

Is-multicast

Cisco Public

70

Flexible NetFlow Record: Non-Key Fields

Based on Catalyst 4500 Supervisor 7-E at FCS

Counters
Bytes

IPv4
TTL Minimum TTL Maximum

IPv6
Total Length Minimum Total Length Maximum

(32 bit counters)

Bytes Long (64 bit counters) Packets

Fragmentation Flags* ToS

Option Header
Hop-limit minimum Hop-limit maximum

(32 bit counters)

Packets Long (64 bit counters)

Transport
TCP Flags: ACK, FIN, PSH, RST, SYN, URG

Routing
Forwarding Status

Interface
Output

Timestamp
First Seen Last Seen

Is-multicast
--- New Non-Key Fields in FnF
*more fragment fields

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

71

4500E FnF Capabilities and Caveats

User-defined flow records supported Per-port, Per-Vlan, Per-Port-Per-Vlan Supports 64 flow masks

Two monitors (IPv4, and IPv6) can be applied simultaneously

to one interface record

Cos/ToS, TTL, and interface option not supported in one flow

Flow-based QoS (UBRL) and FnF not supported on the same

interface

Match interface out option is not supported instead use

collect interface out for getting the Transmit/Egress interface information.

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Flexible NetFlow Configuration Example

Configure the Flow Record

flow record my-app-traffic match transport tcp source-port match transport tcp destination-port match ipv4 source address match ipv4 destination address collect counter bytes collect counter packets

Configure the Exporter

flow exporter my-exporter destination 10.1.1.1

Configure the Flow Monitor

flow monitor my-monitor exporter my-exporter record my-app-traffic

Configure the Interface

Int gi1/1 ip flow monitor my-monitor input

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

73

Top Talkers
Top ten IP addresses that are sending the most packets
Switch# show flow monitor <monitor> cache aggregate ipv4 source address sort highest counter bytes top 10 format table

Top five destination addresses to which were routing most traffic from the 10.10.10.0/24 prefix
Switch# show flow monitor <monitor> cache filter ipv4 source address 10.10.10.0/24 aggregate ipv4 destination address sort highest counter bytes top 5

Top 20 sources of one-packet flows:

Switch# show flow monitor <monitor> cache filter counter packet 1 aggregate ipv4 source address sort highest counter flows top 20

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

74

Embedded Event Manager 3.2

Flexible NetFlow Event Detector
flow record <my-record> match ipv4 ttl match ipv4 source address match ipv4 destination address

flow monitor <my-monitor> record <my-record>

event manager applet security-applet event nf monitor-name "<my-monitor>" event-type create event1 entry-value 2" field ipv4 ttl entry-op lt action 1.0 syslog msg "Flow Monitor $_nf_monitor_name reported Low TTL for $_nf_source_address to $_nf_dest_address" Mar 18 22:15:08.036: %HA_EM-6-LOG: ttl: Flow Monitor ttl reported Low TTL for 10.1.3.2 to 10.1.3.102

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

75

Embedded Event Manager 3.2

Flexible NetFlow Event Detector
flow record rate match ipv4 source address match ipv4 destination address collect counter packets flow monitor <my-monitor> record <my-record> event nf monitor-name "rate" event-type update event1 entry-value "10000" field counter packets rate-interval 15 entry-op gt event2 entry-value "10.1.3.102" field ipv4 destination address entry-op eq action 1.0 syslog msg "Flow Monitor $_nf_monitor_name reported Unusually High Rate of traffic to $_nf_dest_address from $_nf_source_address" Mar 18 22:17:13.033: %HA_EM-6-LOG: rate: Flow Monitor rate reported Unusually High Rate of traffic to 10.1.3.102 from 10.1.3.2

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

76

More info on NetFlow

BRKNMS-1532
BRKNMS-3132

Introduction to Accounting Principles with NetFlow and NBAR

Advanced NetFlow

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

77

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

78

What is MACSec?
Encrypt
MACSec

Encrypt
MACSec Uplink

Encrypt
MACSec

Downlink Decrypt

Downlink

Decrypt

Decrypt

Encryption mitigates packet eavesdropping, tampering, and injection Supports 802.1AE-based strong encryption technology
128-bit AES-GCM, NIST-approved, 10Gb line-rate encryption

Hop-by-hop encryption supports data and packet inspection Works in shared media environments (IP Phones, Desktops)

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

MACSec Frame
Layer 2 SGT Frame and Cisco Meta Data Format
Authenticated Encrypted
DMAC
SMAC 802.1AE Header 802.1Q CMD ETYPE PAYLOAD ICV CRC

CMD EtherType

Version Length

SGT Opt Type

SGT Value

Other CMD Options

Cisco Meta Data

802.1AE Header CMD ICV

are the L2 802.1AE + TrustSec overhead (=~40bytes)

Tagging process prior to other L2 service such as QoS SGT namespace is managed on central policy server (ACS 5.x) No impact IP MTU/Fragmentation. Normal Ethernet Frame
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

80

More info about TrustSec

BRKSEC-1022 Introduction to Identity and TrustSec

BRKSEC-2046 Cisco Trustsec and Security Group Tagging

LTRSEC-2111 TrustSec Security Group Access Lab TECSEC-2041 Identity and Security Group Access with 802.1X and TrustSec

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

81

MACSec: Under the covers

Switch Backplane

4748 Line Card

CTS Rx
SA Check Decrypt Packet Derive SGT* to pass to PP

Stub ASIC #1 (of 4)

4712 Line Card

SA Binding SGT* Insertion Encrypt Packet

Stub ASIC #1 (of 4)

CTS Tx

802.3 Serial
SFP/SFP+ Port 1

XAUI Serial
SFP/SFP+ Port 2 SFP/SFP+ Port 3

10

11

12
Cisco Public

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

82

How To Deploy Downlink MACSec

Switch Configurations
Global Configuration Commands:

aaa new-model ! ! aaa authentication dot1x default group radius aaa authorization network default group radius aaa accounting dot1x default start-stop group radius ! aaa session-id common ! dot1x system-auth-control ! radius-server host 10.3.1.21 key XxXxXxXxXx radius-server vsa send authentication

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

83

How To Deploy Downlink MACSec

Switch Configurations
Interface Configuration Commands:

interface GigabitEthernet4/1 description AnyConnect Interface to MACSEC XP 1 switchport access vlan 903 switchport mode access mtu 9198 logging event link-status authentication priority dot1x authentication port-control auto macsec Default is should-secure, other options are must-notdot1x pae authenticator secure and must-secure mka default-policy spanning-tree portfast authentication linksec policy should-secure

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

How to Deploy Downlink MACSec

Difference from Just Dot1X
RAFALE#show authentication session interface gigabitEthernet 4/1 Interface: GigabitEthernet4/1 MAC Address: 0050.569c.0008 IP Address: 10.3.1.200 User-Name: cisco Status: Authz Success Domain: DATA Security Policy: Should Secure Security Status: Unsecure Oper host mode: single-host Oper control dir: both Authorized By: Authentication Server Vlan Policy: N/A Session timeout: N/A Idle timeout: N/A Common Session ID: 0A0301010000000B0ADAA4C0 Acct Session ID: 0x0000000D Handle: 0xC800000C Runnable methods list: Method State dot1x Authc Success

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

85

How to Deploy Downlink MACSec

After the fact
RAFALE#show authentication session interface gigabitEthernet 4/1 Interface: GigabitEthernet4/1 MAC Address: 0050.569c.0008 IP Address: 10.3.1.200 User-Name: blackbird Status: Authz Success Domain: DATA Security Policy: Must Secure Security Status: Secured Oper host mode: single-host Oper control dir: both Authorized By: Authentication Server Vlan Policy: N/A Session timeout: N/A Idle timeout: N/A Common Session ID: 0A030101000000080551CE18 Acct Session ID: 0x00000009 Handle: 0x02000009 Runnable methods list: Method State dot1x Authc Success

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

86

How To Deploy Uplink MACSec

Switch Configurations
Interface Configuration Commands on back-to-back connected interfaces:
interface GigabitEthernet3/2 switchport mode trunk cts manual no propagate sgt sap pmk 12345678 mode-list gcm-encrypt end ! Switch#show cts interface Global Dot1x feature is Disabled
CTS Layer2 Interfaces --------------------Interface Mode IFC-state dot1x-role peer-id IFC-cache -----------------------------------------------------------Gi3/2 MANUAL OPEN unknown unknown invalid CTS Layer3 Interfaces --------------------Interface IPv4 encap IPv6 encap IPv4 policy IPv6 policy ---------------------------------------------------------------------------

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

87

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

88

TCAM Overview

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

89

Packet Types

IPv4
Classification TCAM

IPv6 Layer-2
OtherL3

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

90

Lookup Types

Four Parallel Lookups

Role-based
Classification TCAM

ACL QoS
Fwd Override

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

91

TCAM Blocks
Each Classification TCAM4 has 32 Blocks

....
2048

32

32

160-bit wide for IPv4 Max is 64K entries

1
1024

Shared across all packet types

...

32

320-bit wide for IPv6 Max is 32K entries

...

2
1
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

92

Restricted Block Usage

1 212 13 14..24

2048 ... 3 2 1
BRKARC-3445

2048 ... 2 1

2048 ...

1024

1024

1024

32
2 1 2 1 2 1 2 1

IPv4 block

IPv6 block

Maximum is 12 blocks Maximum number of Access Control Entries (ACE) in all policies combined on a single ACL path cannot exceed 24K ACEs IPv6 ACEs are double the width of IPv4; you cannot have an IPv6 ACL with more than 12K ACEs
2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

93

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

94

TCAM4 Forwarding Blocks

Dual Forwarding CAMs provide 64 blocks to store IPv4 and IPv6 Unicast Multicast Routes 256,000 IPv4 Routes 128,000 IPv6 Routes

Optimized Space Allocation for IPv4 and IPv6 Configurations! Hardware Support for IPv6
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

95

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

96

Chassis + SupPacket Flow

Packet Flow SERDES

Packet Processor

Forwarding Engine

Active Supervisor

FPGA
S2W BUS

CPU

FPGA
SERDES

CPU
Forwarding Engine

Packet Processor

Standby Supervisor

Fabric Redundancy Module

BRKARC-3445

FPGA
SERDES

Linecard

Port ASIC
Cisco Public

PHY

RJ45 Ports

2011 Cisco and/or its affiliates. All rights reserved.

97

Redundant Supervisor Communication

Active Supervisor
Synchronizes: Startup Configuration Running Configuration Licenses VLAN Database BootVariables Config-Register

1G PHY

Keep-Alive

Keep-Alive - ACK

S2S (Sup2Sup) Connection Full-Duplex Gigabit Link

1G PHY

Standby Supervisor
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

98

Redundant Supervisor Uplinks

Active Ports

Inactive Ports

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

99

SSOStateful SwitchOver
SSO allows Redundant Supervisors to run a stateful IOS and stateful applications to exchange state in order to minimize outage at the time of switchover from Active to Standby Supervisor. SSO supported in Cisco IOS Release 12.2(46)SG with Sup6-E, and now with Sup7-E Default Redundancy Mode Redundant Supervisor fully initialized Upon Switchover Physical Links stay up - Protocols do not reset Traffic Interruption: Sub-Second (<40ms for Layer 2, and <200ms for Layer 3) IOS Images need to be identical
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

100

Redundancy Configuration Status - SSO

Switch#show module Chassis Type : WS-C4510R+E Power consumed by backplane : 40 Watts Mod Ports Card Type Model Serial No. ---+-----+--------------------------------------+------------------+----------2 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1418L036 3 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1352L00L 4 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1352L00Y 5 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7-E CAT1418L08C 6 4 Sup 7-E 10GE (SFP+), 1000BaseX (SFP) WS-X45-SUP7-E CAT1418L08R 7 12 10GE SFP+ WS-X4712-SFP+E CAT1413L01G 8 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1352L030 9 48 10/100/1000BaseT Premium POE E Series WS-X4748-RJ45V+E CAT1418L03A M MAC addresses Hw Fw Sw Status --+--------------------------------+---+------------+----------------+--------2 0026.9927.eaa0 to 0026.9927.eacf 0.4 Ok 3 0026.9927.c9a0 to 0026.9927.c9cf 0.3 Ok 4 0026.9927.cc10 to 0026.9927.cc3f 0.3 Ok 5 c47d.4f81.8a40 to c47d.4f81.8a43 0.8 15.0(1r)SG(0 03.00.00.1.66 Ok 6 c47d.4f81.8a44 to c47d.4f81.8a47 0.8 15.0(1r)SG(0 03.00.00.1.66 Ok 7 0026.0b79.7469 to 0026.0b79.7474 0.4 Ok 8 0026.9927.d5d0 to 0026.9927.d5ff 0.3 Ok 9 0026.9927.f9d0 to 0026.9927.f9ff 0.4 Ok Mod Redundancy role Operating mode Redundancy status ----+-------------------+-------------------+---------------------------------5 Active Supervisor SSO Active 6 Standby Supervisor SSO Standby hot
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

101

Redundancy Configuration Status - SSO

Switch#show redundancy states my state = 13 -ACTIVE peer state = 8 -STANDBY HOT Mode = Duplex Unit = Primary Unit ID = 5 Redundancy Mode (Operational) Redundancy Mode (Configured) Redundancy State Manual Swact Communications = Up client count = 64 client_notification_TMR keep_alive TMR keep_alive count keep_alive threshold RF debug mask = = = = Stateful Switchover Stateful Switchover Stateful Switchover enabled

= = = = =

240000 milliseconds 9000 milliseconds 0 18 0

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

102

SSO-Aware Features
SSO supports stateful switchover of the following Layer 2 features. The state of the features are preserved between both Active and Standby Supervisor Engines

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

103

System High Availability

NSF Recovery (Routing Protocol Recovery)

Non-Stop Forwarding (NSF) provides the capability for the routing protocols to gracefully restart after an SSO fail-over

Si

Si

The newly active redundant supervisor continues forwarding traffic using the synchronized HW forwarding tables
The NSF capable Routing Protocol requests a graceful neighbor start Routing neighbors reform with no loss of traffic

Si

Si

No Route Flaps During Recovery

BRKCRS-3032 Advanced Enterprise Campus Design: Resilient Campus Networks

BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

104

Enabling NSF ConfigurationRouting

Switch(config)#router eigrp 100 Switch(config-router)#nsf Switch(config-router)#timers nsf ? converge EIGRP time limit for convergence after switchover route-hold EIGRP hold time for routes learned from nsf peer signal EIGRP time limit for signaling NSF restart

EIGRP Example

Switch(config)#router ospf 100 Switch(config-router)#nsf ? cisco Cisco Non-stop forwarding ietf IETF graceful restart

OSPF Example

Switch(config-router)#nsf cisco ? enforce Cancel NSF restart when non-NSF-aware neighbors detected helper helper support

Switch(config-router)#nsf ietf ? helper helper support restart-interval Graceful restart interval

Switch(config-router)#bgp graceful-restart ? restart-time Set the max time needed to restart and come back up stalepath-time Set the max time to hold onto restarting peer's stale paths

BGP Example

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

105

Catalyst 4500In Service Software Upgrade

Software Maintenance Windows are significant case of downtime On redundant systems, the ISSU process allows the running IOS software to be upgraded while packet forwarding continues ISSU mechanism leverages architecture for High Availability - NSF / SSO Catalyst 4500 utilizes full image upgrades for the addition of new features, defects, and PSIRTs Increases network availability and reduces downtime caused by planned upgrades

03.01.00.SG 03.02.00.SG

Targets Planned Downtime Due to Software Upgrades

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

106

In Service Software Upgrade Process

OLD OLD NEW NEW

OLD

NEW

OLD NEW

acceptversion

OLD NEW

There is a 4-Step Traditional Method: Load Version Run Version Accept Version Commit Version
BRKARC-3445 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public

Active Standby
107

Supervisor 7-E Single Line ISSU

issu changeversion bootflash:New_Image quick
Standby Supervisor in Slot-6 is Reset and..

Boots with New Image

Initiate SSO Switchover between Active Supervisor in Slot-5 and Standby Supervisor in Slot-6 Slot-5 Slot-6 Old Image New Image New Old Image Image Active Supervisor in Slot-5 resets Standby Supervisor in Slot6 takes over as Active Supervisor Supervisor in Slot-5 boots up as a Standby Supervisor with the New_Image
Active Supervisor
BRKARC-3445

Standby Supervisor
Cisco Public

Completing the ISSU Process

108

2011 Cisco and/or its affiliates. All rights reserved.

ISSU System Status

Switch#show issu state detail Slot RP State ISSU State Operating Mode Current Image Pre-ISSU (Original) Image Post-ISSU (Targeted) Image Slot RP State ISSU State Operating Mode Current Image Pre-ISSU (Original) Image Post-ISSU (Targeted) Image = = = = = = = = = = = = = = 5 Standby Init Stateful Switchover bootflash:xo166 N/A N/A 6 Active Init Stateful Switchover bootflash:xo166 N/A N/A

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

109

Generic Online DiagnosticsWhat is it?

GOLD defines a common framework for diagnostics operations across Cisco Platforms running IOS software. The goal is to check the health of hardware components and verify proper operation of the system control and data plane at run-time and boot

Runtime Diagnostics Line Card Module, Temperature, Power Supply, Fan Tray

Power-On Diagnostics Supervisor, Backplane L2 ASIC, L3 ASIC Memory, CPU, Port

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

110

Agenda
Catalyst 4500E overview IOS XE and Wireshark Overview System Architecture and Packet walk Flexible NetFlow Secure via MACSec ACL/QoS TCAM Forwarding TCAM High Availability Summary

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

111

Catalyst 4500E Architecture - Summary

Centralized Architecture
Catalyst 4500E provides a centralized architecture
Forward and backward compatible Shared memory switch

CPU / SDRAM

Forwarding Engine
Packet Processor

Passive Backplane
All forwarding, queuing, security is implemented on the Supervisor Upgrade Advantages

The individual line cards are considered to be transparent

Sup7-E enables 848Gbps

Line Card Line Card Line Card Line Card Line Card

switching capacity with 48G/slot

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

IOS XE that can leverage multicore CPU, and ability to host applications separately outside IOS context

112

Complete Your Online Session Evaluation

Receive 25 Cisco Preferred Access points for each session evaluation you complete. Give us your feedback and you could win fabulous prizes. Points are calculated on a daily basis. Winners will be notified by email after July 22nd. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center. Dont forget to activate your Cisco Live and Networkers Virtual account for access to all session materials, communities, and ondemand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com. This is session BRKARC-3445

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

113

Thank you.

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

114

Appendix

Catalyst 4500E Portfolio

Borderless Access and Distribution

Entry to Cisco Experience
Catalyst 4500E, Sup6L-E, IP Base

Performance & Services Ready for Future

Catalyst 4500E, Sup7-E, Ent Services

Distribution

Intelligent, Resilient Services Plug-&-Play Intelligent Access

Catalyst 4500E, Sup6L-E, LAN Base
Catalyst 4500E, Sup6L-E, IP Base

Evolves with Your Business

Catalyst 4500E, Sup7-E, IP Base

Access

BRKARC-3445

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

All specifications subject to change without notice

116

Supervisor 7-E Performance & Scalability

Feature Switching Capacity Throughput Bandwidth / Slot CPU DRAM Bootflash Supervisor 7-E 848Gbps 250 Mpps ( 125 Mpps for IPv6) Upto 48G Dual Core 1.5 GHz 2G ( Upgradable to 4G) 1G Supervisor 6-E 320 Gbps 250 Mpps ( 125 Mpps for IPv6) Upto 24G Single Core 1.3 GHz 512 MB ( Upgradable to 1G ) 128 MB Supervisor 6L-E 280 Gbps 225 Mpps ( 125 Mpps for IPv6 ) Upto 24G Single Core 1 GHz 512 MB 128 MB

Chassis Support Max Routes ACL/QoS

Number of 10/100/1000 ports Number of 10GE ports Flexible NetFlow
BRKARC-3445

All Chassis 256K IPv4/125K IPv6 64K per direction

Upto 384 access Upto 4 GE uplinks Upto 96 on Line cards Upto 4 on Supervisors Native support 128K
2011 Cisco and/or its affiliates. All rights reserved.

All Chassis 256K IPv4/125K IPv6 64K per direction

Upto 384 access Upto 4 GE uplinks Upto 30 on Line cards Upto 4 on Supervisors No Support
Cisco Public

3, 6, and 7-slot 57K IPv4/25K IPv6 16K per direction

Upto 240 access Upto 4 GE uplinks Upto 30 on Line cards Upto 2 on Supervisors No Support
117