SAPT DC Network Migration
High Level Design
28 Dec 2023
Enabling Technology
Objectives
The objectives of this presentation are to discuss;
Project Scope
The existing Datacenter Network design/topology of SAPT Network architecture
High Level Design for SAPT DCN Revamp
Edge Firewall design
DC FW Design
TOR Switching Design
High Availability / Load Balancing
Management and Security Features
Proposed Design Decisions
Migration Strategy
Proposed Project Plan
Project Scope
Supply of DC/Edge/Access Network and NGFW equipment as per the required BoQ at Karachi SAPT Port
Design of DCN Architecture
Physical – HA within Primary DC
Logical – L2/L3 Topology
TOR Switching Design – Loop free topology
Optimal Edge/DC NGFW Design with no or minimal down time
High Availability
Use Cases
Traffic Flows
Migration Strategy
Phased Approach
Transit to End State
Documentation of proposed design and changes
Deployment and Migration of Edge/DC NGFW and TOR switches at SAPT Network environment
Deployment of Huawei SecoManager and AAA
Integration with existing Core, Access LAN and NMS
Industry best practices for implementing recommended security features
Existing Network Design
Currently 2 DC sites, Primary and Failover, both located at Karachi Port
Terminal
Primary DC core switching layer consists a pair of Cisco Nexus 9504
switches with VPC configuration
DC servers connected directly to the TOR switches i.e. Cisco Nexus 9372
TOR switches connects to core layer using VPC multiple 10G uplinks
A pair of DC firewalls i.e. Cisco ASA 5585 in Active/Standby mode is directly
connected with the core layer
DC Firewalls currently act as gateway for Server Farm
Existing Network Design
OSPF routing protocol is widely used, while a limited static routing is also
configured
Edge segment consist a pair of Cisco ASA 5516 firewalls in Active/Standby
mode
A partial implementation of BGP exists at Internet Routers with 3 x ISPs
(PTCL, Cybernet & TWA)
Existing Network Topology
TOWARDS HONG KONG
TOWARDS KICTL
ISP 1 ISP 2 ISP 3
PTCL CYBERNET TWA
DMZ SEGMENT
PRIMARY DATACENTER
EDGE FIREWALL
CISCO ASA 5516
B2B SEGMENT
ACTIVE/STANDBY
DC FIREWALL
CISCO ASA 5585
ACTIVE/STANDBY
CISCO NEXUS CORE SWITCH 9K IT/OT SEGMENT
F5 LB
Peer Link
Keep Alive Link
F5 LB
VPC
CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K
Peer Link Peer Link Peer Link
Keep Alive Lin k
DC FAILOVER/
Keep Alive Lin k
SECONDARY
Keep Alive Lin k
VPC VPC VPC
DATACENTER
SERVER FARM
DISTRIBUTION SWITCHES
ACCESS SWITCHES
Proposed High Level Network Topology Routers
replaced by
TOWARDS HONG KONG
ISP 1
PTCL
ISP 2
CYBERNET
ISP 3
TWA Huawei
i-Stack Switches Edge Firewalls
HUAWEI S5731
PRIMARY DATACENTER
EDGE SWITCHES
Migration
DMZ SEGMENT HUAWEI USG6635
EDGE FIREWALL
ACTIVE/STANDBY
eBis Firewall
HUAWEI USG6685F
DC FIREWALL
eBis-FW
CISCO ASA 5516
Migration
ACTIVE/STANDBY
DC Firewalls
CISCO NEXUS CORE SWITCH 9K IT/OT SEGMENT
Peer Link
F5 LB
Migration
Keep Alive Link
F5 LB
VPC
i-Stack i-Stack
TOR Switches
Migration
i-Stack
DC FAILOVER/
HUAWEI TOR SWITCH HUAWEI TOR SWITCH
CE6881 CE6881
HUAWEI TOR SWITCH
CE6863E
SECONDARY
DATACENTER
SERVER FARM
Huawei
SecoManager
SECOMANAGER - VM AAA iMaster NCE
DISTRIBUTION SWITCHES & AAA
ACCESS SWITCHES
Existing Edge Design
ISP 1 ISP 2 ISP 3
PTCL CYBERNET TWA
CISCO ASA5516
EDGE FIREWALL
DMZ SEGMENT
ACTIVE/STANDBY
CISCO NEXUS CORE SWITCH 9K
Peer Link
Keep Alive Link
VPC
Proposed Edge Design
ISP 1 ISP 2 ISP 3 INTERNET
PTCL CYBERNET TWA
i-Stack
CISCO
DUO MFA SSL REMOTE VPN
SSL REMOTE VPN
BGP AS#
HUAWEI S5731
• BGP Configuration
EDGE SWITCHES • Routing Policies
• Traffic Filtering
Eth-Trunk Eth-Trunk
HUAWEI USG6635
EDGE FIREWALL
DMZ SEGMENT • Policies/Services Migration
• SSL VPN with MFA
TOWARDS CORE SWITCH
ACTIVE/STANDBY
ACTIVE STANBY
DC FIREWALL
Existing DC FW Design CISCO ASA 5585
ACTIVE/STANDBY
CISCO NEXUS CORE SWITCH 9K
Peer Link
Keep Alive Link
VPC
CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K CISCO NEXUS TOR SWITCH 9K
Peer Link Peer Link Peer Link
Keep Alive Link Keep Alive Link Keep Alive Link
VPC VPC VPC
SERVER FARM
PROPOSED DC FW Design
HUAWEI USG6685F DC FIREWALL
FAILOVER LINK/HEARTBEAT
VRRP(Virtual Router Redundancy Protocol) STANDBY
ACTIVE
Eth-Trunk LACP
• OSPF Routing CISCO NEXUS CORE SWITCH 9K
• Policies Optimization
North South Traffic
Peer Link
Keep Alive Link
VPC
East West Traffic Default Gateway DC FW
SERVER FARM
SECOMANAGER - VM AAA iMaster NCE
PROPOSED TOR Switching Design
CISCO NEXUS CORE SWITCH 9K
Peer Link
• Virtualized Stack (i-stack)
• Ether-Trunking VPC
Keep Alive Link
• Loop-free Topology
10G 10G
Eth-Trunk LACP 10G
i-Stack i-Stack
Eth-Trunk LACP
Eth-Trunk LACP
10G
HUAWEI CE6863E HUAWEI CE6881
FIBER TOR SWITCH COPPER TOR SWITCH
SERVER FARM
SECOMANAGER - VM AAA iMaster NCE
High Availability
Failover Scenarios
Primary DC
Active Edge Firewall Fails – Traffic will continue to flow from Standby
FW
Primary ISP Link Fails - Traffic will continue to flow from secondary ISP
Internet Switch Fails - Traffic will continue to flow from secondary
switch in virtualized stack
Active DC FW Fails - Traffic will continue to flow from Standby DC FW
Primary TOR Switch Fails - Traffic will continue to flow from Secondary
switch in virtualized stack
In case of dual failure of a critical component e.g. both DC FW fail –
manual intervention may be required to run the services from
secondary/failover DC
Load Balancing
Load Balancing of traffic will be achieved at DC switching layer by utilizing
multiple 10G uplinks in LACP/Ether-Trunk mode while avoiding congestion
and giving better user experience
Load balancing of internet traffic (incoming & outgoing) will be configured
at Edge switching layer using BGP routing policies
In case of failure of any link, the available link will take the full load
No load balancing at FW layer (either DC or Edge) as HA mode is
Active/Standby
Security and Management Features
Users Authentication (AAA) via Radius / TACACS+ (iMaster NCE)
Firewall Policies Optimization (SecoManager)
BPDU / Root Guard
UDLD (Unidirectional Link Detection)
BCP38 and Bogon filtering
SNMPv3
SSHv2
Syslog
Best security practices for device hardening e.g. Management, Control and
Forwarding plane
Design Decisions
Design Decision 1: NAT will be performed at Huawei Edge Firewall. Currently it is being done at Internet Routers
Design Decision 2: BGP configuration and routing policies will be done at Huawei Edge Switches
Design Decision 3: Edge switches will form Ether-Trunk (Port Channel) with ISPs if possible, to provide full
redundancy of switch failure
Design Decision 4: Separate Ether-Trunks (Port Channels) for each DC firewall at Core layer
Design Decision 5: Virtualized stacking i.e. Huawei i-Stack will be used at TOR switching layer, forming Ether-
Trunk (Port channel) with the Cisco VPC at core switches
Migration Strategy
Phased Approach – The Strategy is to have the Parallel setup as much as possible while
maintaining the zero or minimal downtime for services migration –
Description of Multiple milestones
Phase 1 – Deployment of Internet Edge design (FW & Internet Switches) – Service
Migration
Phase 2 – Deployment of TOR switches – Service Migration
Phase 3 – Deployment of DC FW – Service Migration
Proposed Design Changes to achieve the smooth transition
End State Design
Proposed Project Plan
Proposed Project Plan
Thank you!
