Design, Manage, and Troubleshoot SD-

WAN Networks
LTRRST-2457
Speakers:
Thomas McKinnon – Solutions Architect
Brandon Lynch – Technical Leader

Santosh Vignan Mantri Pragada – Network Engineer

1 | P a g e

Table of Contents

LEARNING OBJECTIVES 5

SCENARIO 5

NETWORK DIAGRAM 6

VMANAGE ACCESS 6

CEDGE LOGIN 6

WORKSTATION1 LOGIN 6

BRANCH HOST INFORMATION 6

IMPLEMENTING CONTROL POLICY 7

TASK 1: REVIEW DEFAULT SD-WAN ENVIRONMENT 8

STEP 1: VIEW IPSEC TUNNELS 8
STEP 2: VIEW CURRENT ROUTE TABLES 8
TASK 2: CREATE AND APPLY POLICY 9
STEP 1: CREATE SITE LIST AND VPN LIST 9
STEP 2: CONFIGURE POLICY 12
STEP 3: ACTIVATE CENTRALIZED POLICY 15
STEP 4: VERIFY IPSEC TUNNELS 15
STEP 5: VERIFY OMP ROUTING 16
TASK 3: TROUBLESHOOT AND CORRECT CONTROL POLICY 17
STEP 1: REVIEW CONTROL POLICY 17
STEP 2: MODIFY CONTROL POLICY 18
STEP 10: VALIDATE OMP ROUTES 20

DEPLOYING QOS IN A DATA POLICY 21

TASK 1: CONFIGURE A LOCALIZED QOS POLICY USING THE POLICY BUILDER 21

STEP 1: NAVIGATE TO THE POLICY BUILDER 21
STEP 2: START A NEW LOCALIZED POLICY 22
STEP 3: CONFIGURE A DATA PREFIX LIST TO MATCH VOICE TRAFFIC 22
STEP 4: CONFIGURE THE CLASS-MAPS 24
STEP 5: CONFIGURE THE QOS MAP 26
STEP 6: ADD THE QUEUES AND ASSIGN BANDWIDTHS 28
STEP 7: ADD THE REMAINING QUEUES 29
STEP 8: BEGIN CREATION OF A NEW ACL 30
STEP 9: MATCH VOICE TRAFFIC AND SET ACTIONS WITH AN ACL SEQUENCE 31
STEP 10: MATCH FINANCE TRAFFIC AND SET THE CLASS 34

2 | P a g e

STEP 11: CONFIGURE THE REMAINING ACL SEQUENCE TO MATCH ALL TRAFFIC 35
STEP 12: PROCEED TO THE POLICY OVERVIEW 37
TASK 2: ATTACH THE POLICY TO THE BRANCH 3 ROUTER 39
STEP 1: DETACH THE BRANCH 3 ROUTER FROM THE TEMPLATE 39
STEP 2: EDIT THE VPN0 TLOC INTERFACE TEMPLATES 41
STEP 3: ATTACH ACL TO SERVICE-SIDE VPN INTERFACES 43
STEP 4: ATTACH THE DATA POLICY TO THE DEVICE TEMPLATE 45
STEP 5: RE-ATTACH THE DEVICE TEMPLATE TO BR3 45
STEP 6: VALIDATE THE POLICY PUSH 47
TASK 3: TROUBLESHOOT SLOW BROWSING SPEEDS 48
STEP 1: VALIDATE FLOW INFORMATION USING CFLOWD 48
STEP 2: RUN PACKET TRACE TO VERIFY DSCP MARKING 50
STEP 3: CHECK FOR OTHER INTERNET TRAFFIC 54
STEP 4: MODIFY LOCALIZED DATA POLICY FOR HTTPS AND VERIFY BEHAVIOR 55

APPLICATION-AWARE ROUTING POLICY 58

TASK 1: CREATING LISTS AND APPLICATION AWARE ROUTING POLICY 58

STEP 1: CREATE POLICY SLA LISTS 58
STEP 2: CREATE APPLICATION AWARE ROUTING POLICY 59
TASK 2: APPLY POLICIES FOR APP VISIBILITY 62
STEP 1: APPLY AAR POLICY TO CENTRALIZED POLICY 62
STEP 2: APPLY LOCAL POLICY FOR APP VISIBILITY 63
STEP 3: VALIDATE AAR POLICY 64
TASK 3: TROUBLESHOOT AND TUNE APPLICATION ROUTING 66
STEP 1: MODIFY AAR POLICY 66
STEP 2: VERIFY CITRIX TRAFFIC 69

BRANCH SECURITY WITH ZONE-BASED FIREWALL 70

TASK 1: CONFIGURE ROUTE-LEAKING BETWEEN VPN 10 AND VPN 40. 71

STEP 1: VERIFY THE ROUTING TABLE ON THE ROUTERS. 71
STEP 2: NAVIGATE TO THE POLICY BUILDER 72
STEP 3: CREATE A NEW CONTROL POLICY 72
STEP 4: CREATE SITE-LISTS 72
STEP 5: VERIFY VPN-LISTS 73
STEP 6: CREATE THE CONTROL POLICY 73
STEP 7: APPLY THE POLICY 76
STEP 8: PREVIEW THE POLICY AND SAVE IT 77
STEP 9: ACTIVATE THE POLICY 77
TASK 2: CONFIRM THAT ROUTE-LEAKING WORKS. 78
STEP 1: VALIDATE ROUTE LEAKING 78
TASK 3: CREATE THE ZONE-BASED-FIREWALL-POLICY 79
STEP 1: CONFIGURE ZONES 80
STEP 2: NAVIGATE TO POLICY 80
STEP 3: CONFIGURE THE POLICY 82

3 | P a g e

STEP 4: CONFIGURE ZONE-PAIRS 83
STEP 5: BUILD THE POLICY 83
STEP 6: APPLY THE POLICY 85
TASK 4: VALIDATE ZONE-BASED FIREWALL 86
STEP 1: CHECK CONNECTIVITY BETWEEN VPNS 86
STEP 2: RUN PACKET TRACE TO UNDERSTAND WHY WE DO NOT SEE ANY RETURN TRAFFIC 86
STEP 3: VIEW ZBFW RULES 93
STEP 3: MODIFY ZBFW RULES 94
STEP 4: VERIFY CONNECTIVITY 95

BONUS TASK: CONFIGURE A CENTRALIZED DATA POLICY FOR QOS 97

TASK 1: CREATE CENTRALIZED POLICY TO APPLY FOR QOS 97

STEP 1: BEGIN A NEW CENTRALIZED POLICY 97
STEP 2: VALIDATE THE SITE AND VPN LISTS 97
STEP 3: CONFIGURE THE MARKING ACL 99
STEP 4: DEFINE THE SITE AND VPN LISTS FOR POLICY APPLICATION 102
STEP 5: ACTIVATE THE POLICY 105
STEP 6: REMOVE THE ACL FROM SERVICE-SIDE INTERFACES 105
STEP 6: DETACH THE ACL FROM THE DATA POLICY 105
STEP 7: VALIDATE THE UPDATED CONFIGURATION 106

APPENDIX A: ADDITIONAL SD-WAN LEARNING OPPORTUNITIES 107

4 | P a g e

Learning Objectives
Upon completion of this lab, you will be able to:

• Navigate vManage for deployment and troubleshooting scenarios

• Understand the flow of a packet through a cEdge router in relation to QoS and cFlowd
• Comfortably design and deploy both a centralized policy and localized policy
• Troubleshoot your SD-WAN environment with the use of vManage GUI and CLI

Scenario
As SD-WAN customers look for methods to ease network management and design, zero-
touch deployment and robust application/transport policy come to the forefront of any solution
requirements. Using the Overlay Management Protocol (OMP), customers can shift design-
thought away from traditional DMVPN deployments to that of an extensible WAN fabric
enabled by the Cisco SD-WAN solution. Regardless of the use case, customers have the tools
to push policy via a Secure Extensible Network GUI.

This lab will provide participants with hands-on experience in a simulated environment that
focuses not only on deployment but also monitoring and troubleshooting scenarios found in
real-life use cases. The goal of this session is to provide participants with the ability to create
and maintain an extensible, resilient SD-WAN fabric that is transport agnostic and that
provides end-to-end simplicity.

This lab will be broken up into four specific use cases. In each use case, the initial design will
include a design flaw to allow troubleshooting to occur. The use cases to be shown include:

• Create a Control Policy with different requirements per VPN

• Implement QoS with a Localized Data Policy
• Apply Application Steering with Centralized App-Route Policy
• Implement Security with Zone-Based Firewall

This session will highlight the benefits of the SD-WAN solution by providing participants with
the tools needed to design, monitor, and maintain the SD-WAN fabric.

Participants should have a general understanding of route redistribution, VPN concepts, and
troubleshooting techniques.

5 | P a g e

Network Diagram

vManage Access
vManage URL https://198.18.1.10
Username admin
Password admin

cEdge Login
Username admin
Password admin

Workstation1 Login
RDP IP https://198.18.133.36
Username dcloud\administrator
Password C1sco12345

Branch Host Information

Username administrator
Password C1sco12345

6 | P a g e

Implementing Control Policy
Large scale enterprise networks often implement segmentation between different lines of
business (LOB) and/or applications. Each LOB or application may have different requirements or
security regulations for site-to-site traffic. By default, Cisco SD-WAN creates a full mesh
environment between all sites for all VPN’s. With the use of control policies, Cisco SD-WAN
provides the flexibility to configure a routing and control policy on a per-VPN basis to meet the
requirements as defined by your application and/or security team.

This section of the lab will demonstrate how to create a control policy to meet specific
application requirements. Using the vManage Policy Builder, you will create a custom control
policy to maintain a full mesh topology for corpVPN (VPN 10) while creating a hub-and-spoke
environment for replicationVPN (VPN 20).

The two key components for creating a control policy include:

1. TLOC Control, specifying what sites can build IPSec tunnels to other sites
2. Route Control, identifying what routes are allowed to be learned via OMP

Application of the control policy is applied based on site-list. Once the site-lists are applied, and
the policy is activated, you will learn how to validate tunnel connections and routing for both
corpVPN and replicationVPN via the vManage dashboard.

The Policy Builder inside of vManage provides a step-by-step workflow to define the necessary
elements of your policy and build, preview, and apply it to your devices. This also provides a
single repository for all policies active in your network that you can visualize and manage as
needed.

7 | P a g e

Task 1: Review Default SD-WAN Environment

Before we build the control policy, we want to view the current topology

Step 1: View IPSec Tunnels

Inside of the vManage GUI, navigate to Monitor > Network from the menu.

We will then select BR2-CEDGE > Tunnel, where Tunnel is listed under the WAN portion of the
dashboard.

Notice that tunnels are established for both MPLS and Internet to all other sites per color.

Step 2: View Current Route Tables

Select Real Time at the bottom left of the screen to view routes received.

8 | P a g e

Under device options, type “OMP Received Routes” and “Do Not Filter” so that we can see
routes for all VPNs.

Make note that multiple routes are received for VPN10 and VPN20.

Task 2: Create and Apply Policy

We are now ready to create and apply the policy.

The Policy Builder inside of vManage provides a step-by-step workflow to define the necessary
elements of your policy and build, preview, and apply it to your devices. This also provides a
single repository for all policies active in your network that you can visualize and manage as
needed.

Step 1: Create Site List and VPN List

The first step when creating a policy is to define Lists. Lists are used to identify a Group of
Interest, which will later be called in our policy. Required lists for a control policy, which we
will configure now, include site-lists and VPN lists.

Select Configuration > Policies > Custom Options > Lists:

9 | P a g e

Here you will see all the Lists that can be applied via centralized policy. Select Site. For the
purpose of this lab, most of the site-lists have already been created. We will create one new
site-list, encompassing all branch sites. Select New Site List, and label this site list as
ALLBranches with the site ID as 300-599

Site List Name Site ID

DC1 100

DC2 200

AllDC 100, 200

Site-Type1 300-399

Site-Type2 400-499

Site-Type3 500-599

AllBranches 300-599

10 | P a g e

Once the site-lists are created, we will then configure the remaining VPN lists. Select VPN >
New VPN List. We will create the replicationVPN as defined in the below table, selecting Add
afterwards.

VPN List Name Add VPN

corpVPN 10

replicationVPN 20

guestVPN 40

11 | P a g e

Step 2: Configure Policy
Now that we have created the lists we will need for our control policy, we are ready to build it.
When building policies, you have the option of pre-building the control policy and importing
into your Centralized Policy or building directly in the Centralized Policy. For this lab, we will
build directly in the Centralized Policy.

Click Centralized Policy at the top of the page and then select Add Policy. Notice we are now
at the Lists page. Click on both Site and VPN and verify that the lists created previously are
listed. If we had not already created the site-lists and VPN lists in the previous step, we could
do so now. Click on Next.

Here we will build our control policy, defined as Topology.

12 | P a g e

Select Add Topology > Custom Control (Route & TLOC)

Name the policy “VPN20-Hub-N-SPOKE”, with the same description “VPN20-Hub-N-SPOKE”.

Select Sequence Type and choose TLOC > Sequence Rule > Match
Site: AllDC
Choose Accept under Actions
Click Save Match and Actions

Create new sequence rule. Select Sequence Rule > Match

Site: AllBranches
Actions: Accept
Click Save Match and Actions

It is important that we are allowing all sites to build tunnels to all other sites to ensure that
VPN10 may remain full-mesh.

With IPSec tunnels now allowed, routing must be addressed and controlled to meet the
requirements.

Select Sequence Type > Route

Sequence Rule > Site: DC1

VPN : replicationVPN
Action: Accept
Save Match and Actions

Sequence Rule > Site: DC2

VPN: replicationVPN
Action: Accept
Save Match and Actions

13 | P a g e

Click Save Control Policy:

The control policy is now built. Click Next. At this time, we are not applying any traffic policies
so click on Next again. We must now name the Centralized Policy and apply the control policy.

For both name and description, use “Centralized-Policy-v01”.

Click on New Site List and set the Outbound Site List for AllBranches

Click Add > Save Policy.

14 | P a g e

Step 3: Activate Centralized Policy

While we have now created the Centralized Policy, we have not activated the policy.

In order to activate the policy, select the options for your “Centralized-Policy-v01”.

Select Edit and Click on Activate.

Step 4: Verify IPSec Tunnels

Now that we have successfully activated the Centralized Policy, we will validate the policy is
working as expected. To begin with, we will verify the IPSec Tunnels again from BR2-CEDGE1.

Navigate Monitor > Network > BR2-CEDGE1 > Tunnels

15 | P a g e

Notice that the IPSec tunnels are still formed to all remote site devices. This is what we
expected, and is required, so that VPN10 can maintain a full-mesh

Step 5: Verify OMP Routing

Continuing with the verification, we will look at the routes being learned via OMP to ensure
VPN20 is hub-and-spoke, while VPN10 remained full-mesh.

While still in the BR2-CEDGE device monitor, select Realtime and again type in OMP Received
Routes under Device Options. This time we will filter.

Select Show Filters > VPN ID: 20 and press enter:

16 | P a g e

Notice that we are now only seeing routes in VPN20 learned from DC1 due to the preference
being set by routing.

Modify the VPN ID in the filter from 20 to 10 and press enter:

Uh Oh. Notice that we are not seeing any routes in VPN 10. This means something is wrong
with our control policy, as we were receiving routes before activating the policy.

Task 3: Troubleshoot and Correct Control Policy

Step 1: Review Control Policy

The simplest way to review the policy will be via CLI output. With the vManage dashboard,
there is no need to actually login to the vSmart device.

Navigate to Configuration > Policies:

17 | P a g e

Looking at the policy options, select Preview.

Scroll down until you see the control policy. Notice that we see in sequences 21 and 31 we are
accepting routes from DC1 and DC2 for replicationVPN. The next action is the Default Action,
where we are rejecting everything:

When building the policy, we forgot to allow routes for corpVPN.

Step 2: Modify Control Policy

Click OK to get back to the Centralized Policy main page.

Select Custom Options > Topology > VPN20-Hub-N-SPOKE > Options > Edit.

18 | P a g e

In the left-hand panel, select Route:

Sequence Rule > Site: AllDC

VPN : corpVPN
Action: Accept
Save Match and Actions

Sequence Rule > Site: AllBranches

VPN: corpVPN
Action: Accept
Save Match and Actions

Click Save Control Policy > Activate.

19 | P a g e

Since the policy is already activated, the configuration will be immediately pushed to the
vSmart controllers. Following standard configuration pushes, we will click Next > Next > Click
“Confirm configuration changes on two devices” > OK:

Step 10: Validate OMP Routes

Once the configuration is successful, it is time to validate VPN10 OMP routes again.

We will navigate back to BR2-CEDGE1 to do this verification.

Monitor > Network > BR2-CEDGE1 > Real Time > IP Routes > Filter VPN 10 > Search

We now see all the routes in VPN 10 as expected in a full mesh environment.

20 | P a g e

Deploying QoS in a Data Policy
QoS is a key network element in any enterprise network but is particularly necessary at the WAN
edge. As traffic leaves a branch or hub site, it is important that traffic is properly classified so
that the Customer Edge (CE) router can police and shape traffic to the provided rate and avoid
any unintended drops by the ISP that may affect critical traffic.

In this section, you will use the vManage Policy Builder to build out all necessary elements for
QoS marking and queuing and learn how to apply the policy to a router template. Within the
vManage GUI, QoS policies are built as a Localized Policy. As you’ll see from the following Tasks,
there are four key pieces that must be configured as part of the policy:

1. Required elements, such as the class-maps that will be used for traffic placement.
2. Optional elements, such as data prefix-lists used for matching and marking traffic in a
service-side ingress ACL.
3. QoS map, where bandwidths are set per class
4. ACL, which is used to match traffic, set the DSCP (if required), and set the forwarding
class

Once the policy is applied, you will learn how to validate that QoS is working and troubleshoot
using the CLI.

Task 1: Configure a Localized QoS Policy using the Policy Builder

Step 1: Navigate to the Policy Builder

Inside of the vManage GUI, navigate to Configuration > Policies from the menu:

21 | P a g e

Step 2: Start a New Localized Policy

Once the Policy Builder page loads, select Localized Policy at the top. You’ll see that no
localized policies are currently configured:

Click on Add Policy to begin building a new policy.

Step 3: Configure a Data Prefix List to Match Voice Traffic

The first step in the Policy Builder is to configure the key elements, or Groups of Interest. This
can include policers, prefix-lists, class-maps, and various other items.

You will be working on Branch 3 for this section of the lab. As you will see from the topology,
Branch 3 consists of 3 VPN’s. The following table provides details on each VPN:

Table 1: Branch 3 VPN Summary

VPN Purpose

10 corpVPN

20 replicationVPN

40 guestVPN

22 | P a g e

Because VPN10 is dedicated to corporate traffic, we will be matching voice traffic using an
ACL and prefix-list. There is also generally call signalling that may receive a different DSCP
classification. For the purposes of this lab, you will configure a data prefix list to match the
subnet and everything will be classified as DSCP EF for egress queuing. However, depending
on the network requirements, as you’ll see, the Policy Builder offers flexibility in matching
based on the design to properly prioritize your traffic.

The first thing that is needed is to define a Data Prefix List to match the Voice application
subnet.

Click on Data Prefix:

Next, click on New Data Prefix List and the fields for the prefix list show up. The data prefix
list that you will define here will be used in an ACL to match the VPN10, or corpVPN, subnet
(10.5.11.0/24) and set both the DSCP value and class-map for proper shaping. Enter VOICE-
TRAFFIC for the name and 10.5.11.0/24 for the subnet as shown below:

23 | P a g e

After entering this information, click Add. You will see the finalized prefix list entry:

Step 4: Configure the Class-Maps

Select the Class Map tab:

24 | P a g e

Click New Class List and a new box is displayed to define the class name and queue number.
Enter VOICE for the class name and select Queue 0 for the queue:

By default on hardware SD-WAN routers, Queue 0 is reserved for control and LLQ traffic. For
this reason, voice traffic should be configured to use Queue 0.

As you can see, the remaining class-maps in the table below have been pre-built for the lab:

25 | P a g e

Table 2: Queue Names and Numbers

Name Queue

VIDEO 1

BULK-DATA 2

FINANCE 3

BEST-EFFORT 4

Once the VOICE class-map has been created, vManage should show all classes as follows
(note that the ordering may be different):

Click Next to move to the Forwarding Class/QoS section.

Step 5: Configure the QoS Map

With the class maps configured, you now need to configure the QoS Map that will be enabled
on the XE SDWAN router for outbound queuing. In a typical Cisco configuration, QoS is
configured on the WAN interface in a hierarchical model:

<Parent policy-map applied to the WAN interface>

shape <provided egress bandwidth rate>
<Child policy-map for proper queuing based on DSCP>

26 | P a g e

When configuring the QoS Map in this section, this will result in the child policy-map definition
on the model above and as you’ll see once the policy is applied and the configuration is
previewed later in this scenario, the remaining components will be addressed.

Click on Add QoS Map to begin building the QoS policy and select Create New:

Name the QoS Map WAN-QOS and provide a description WAN QoS Policy - Branch 3:

27 | P a g e

You will notice that Queue 0 is preconfigured by default. This queue cannot be modified and as
you add more queues with bandwidth allocations, vManage automatically adjusts the Queue 0
bandwidth to ensure the total allocation doesn’t exceed 100%.

Step 6: Add the Queues and Assign Bandwidths

You will now add the respective queues for the QoS Map and assign a bandwidth to each
queue. Click on Add Queue. A new dialog box is revealed that allows you to specify the
queue, bandwidth allocation, and drop method:

There are a couple of things to note about the configuration options for XE SDWAN routers:

• While the box presents an option to configure bandwidth or buffer percentages,

XE SDWAN routers do not support buffer percentage mappings. This is unique to
vEdge devices. For this reason, a bandwidth percentage should be specified.
This maps to the bandwidth percent command inside of the policy-map.
• The default drop type is tail drops, which must still be specified but doesn’t
involve any additional configuration inside of the policy-map. Selecting Random
Early maps to random-detect inside of the policy-map, also known as WRED
(Weighted Random Early Detection). This is generally the preferred drop
mechanism to eliminate peaks and valleys in queue depths and transmission
rates but as with any design, consider the traffic as part of the queue and decide
what is best for your network.

Select 1 for the Queue, Random Early for Drops, and assign 20% bandwidth to the queue.
Click Save Queue to save the configuration.

28 | P a g e

Step 7: Add the Remaining Queues
Once Queue 1 is saved, add the remaining queues per the following table:

Table 3: Queue Parameters

Queue Number Bandwidth

2 20

3 20

4 10

Once complete, the queue structure should show as follows inside of the QoS Map
configuration:

Click Save Policy and the high-level view of the QoS Map is displayed:

29 | P a g e



Click Next to move to the ACL configuration.

Step 8: Begin Creation of a New ACL

Click Add Access Control List Policy and select Add IPv4 ACL Policy:

Name the ACL QoS-Marking-ACL and provide a description:

Name: QoS-Marking-ACL

Description: QoS Marking ACL

30 | P a g e



Step 9: Match Voice Traffic and Set Actions with an ACL Sequence
You will now create the sequences in the ACL to match the corresponding traffic based on the
data-prefix-list defined earlier in Step 3 as well as on the received DSCP marking and/or
ports. The ACL should be defined according to the branch design. For example, if QoS
marking is done by a LAN-side device prior to the XE SDWAN router, it will likely be sufficient
to match on this marking on ingress of the router and set the class accordingly. In many cases
with video and voice, the endpoint may provide its own DSCP marking. However, in cases
such as L2-based designs, there may be no QoS marking up to the branch edge. In these
situations, the XE SDWAN router can be used to both mark and set the class. In this lab, you
will define sequences that do both to demonstrate how each can be accomplished.

Click on Add ACL Sequence:

31 | P a g e



For the first sequence, we will match the VOICE-TRAFFIC data-prefix-list that was created
earlier in this task. This will allow the XE SDWAN router to match any source IP in this subnet
and properly classify the traffic since the voice endpoints aren’t setting the DSCP today. Click
on Sequence Rule to add a new sequence to the ACL and under Match, click Source Data
Prefix:

After clicking in the Select a data prefix list dialog box, you’ll notice that the data-prefix-list
that you previously configured is pre-populated. Any data-prefix-lists that were created in
Step 1 of the Policy Builder will be present here. Select this list.

32 | P a g e

After selecting the list, click on Actions and choose DSCP. Set the DSCP value of 46 (EF) for
this traffic:

Next, click on Class and click the dialog box that is presented. You’ll notice that all of your
previously-created classes are populated here. Select VOICE:

Click Save Match and Actions. Once done, you’ll see the high-level overview of the sequence
that you just created:

33 | P a g e



Notice that because this is a summary view, not all of the actions are displayed. If you wish to
see the full output of the sequence, click on the down arrow on the left side of the sequence
box to expand the output:

Step 10: Match Finance Traffic and Set the Class

Now that voice traffic has been matched correctly, you will create a new ACL sequence to
match traffic from the company’s primary finance application. This application communicates

34 | P a g e

to the server that lives at the primary DC on port 45000, meaning this will be the destination
port for this traffic. Click Sequence Rule to add a new sequence that matches on a
Destination Port of 45000, sets the DSCP value to 26, and sets the Class to FINANCE. Once
complete, the ACL state should show the following:

Step 11: Configure the Remaining ACL Sequence to Match All Traffic
The remaining ACL sequences will utilize different match parameters to correctly classify the
traffic. Table 3 below provides the DSCP mapping for each class along with the match criteria
to use inside of the ACL :

Table 4: Class-to-DSCP Mappings with Match Criteria for Remaining Classes

Class Name DSCP (Decimal) Match Criteria

VIDEO 32, 34 DSCP

BULK-DATA 10 Dst. Port = 80

Once done, your ACL should look like the following:

35 | P a g e



There are a couple of things to notice about the output above:

1) The last sequence above (Sequence 5) doesn’t contain any match parameters but sets
the class to BEST-EFFORT. This will act as a “match-all” condition to match all other
traffic and put this in the BEST-EFFORT queue, which is mostly for scavenger traffic in
this design.
2) You’ll also notice that each sequence either shows the DSCP or Class value by default
in this view. As noted earlier, you can expand each box to see all action/match criteria.
For VIDEO and BEST-EFFORT, no DSCP marking is required but can still be
implemented depending on design.

Finally, click on Default Action. You will see that the default action is set to Drop:

36 | P a g e

Like the implicit ‘deny all’ in an ACL, the default ‘drop’ action will drop anything not matching
the previous sequences. For the purposes of this lab, this action doesn’t need to be modified
due to the fact that you already incorporated a ‘match-all’ condition to place all remaining
traffic in the BEST-EFFORT queue. Depending on the use case for the ACL, this may require
modifying to support the branch or hub design.

Click Save ACL Policy and the ACL is presented in the current list:

Step 12: Proceed to the Policy Overview

From the ACL screen, click Next. You will be taken to the Route Policy step. Because this
Localized Policy is specific to QoS, no Route Policy is required. Click Next again to be taken to
the Policy Overview section, which is the last section in the Policy Builder. Name the policy
BRANCH-3-QOS-POLICY and provide a description as seen below:

37 | P a g e



Notice that Netflow and Application are checked above. This enables application- and flow-
visibility under the policy, providing cFlowd capabilities to see per-flow details and behaviour
based on both DSCP and application. As you’ll see in the Troubleshooting Task, this is a useful
tool when troubleshooting flow or application performance and to understand how traffic is
routing through a XE SDWAN router.

Prior to saving the policy, it’s often useful to preview the configuration to ensure that it
matches the intent. Click Preview to see the CLI configuration that will be provisioned on the
router once the policy is attached to the device template:

If anything is found that was misconfigured, you can click Back and move back to the
appropriate section and edit that piece of the policy. In this case, the policy should match the

38 | P a g e

intent of the QoS design. Click Save Policy and the policy now displays under the Localized
Policy tab:

At this point, you’re now ready to attach the policy to the device template and push the policy
out to the router.

Task 2: Attach the Policy to the Branch 3 Router

Now that the policy is configured, it needs to be attached to the Branch 3 router template to
activate it. Before doing that, it is important to understand the differences between a Localized
Policy and Centralized Policy.

In a Centralized Policy, which you’ll optionally create in the Bonus Task in this section as well
as in the next section, the policy is activated on the vSmarts and pushed out based on the VPN
and Site Lists defined. When the policy is downloaded on the XE SDWAN router, it is applied
globally in the forwarding path according to the defined direction and doesn’t require direct
application on interfaces.

With a Localized Policy, because the policy is applied via the router’s template, it is configured
globally under the policy section but is not applied until explicitly done in other parts of the
template. In this section, you’ll see how to both attach the policy to the template but also apply
the corresponding parts of the policy to the interfaces for it to take effect.

Step 1: Detach the Branch 3 Router from the Template

When applying configuration changes to a router template, it is not required to detach the
template from the device. The reason that we’re doing it is here is because of the current
behavior within vManage. When a device is attached to a template and a change is made to
the template either globally (i.e., Policy) or via one of the feature templates referenced,

39 | P a g e

vManage then tries to push the changes. This prevents multiple template edits simultaneously
with a single push.

To reduce the number of template pushes in the lab, you’ll first detach the device from the
template and then make the necessary edits.

Navigate to Configuration > Devices. Click on Change Mode and select CLI mode:

Select BR3-CEDGE1, click the right-facing arrow to move it to the CLI Mode table, and click
Update to CLI Mode:

40 | P a g e

A task is then generated to move the device to CLI mode. After you see Success, the device
is now in CLI mode.

Step 2: Edit the VPN0 TLOC Interface Templates

Navigate to Configuration > Templates. Beside the BranchType3Template-CSR template,
click Edit:

Scroll down to the Transport & Management VPN section. From this, you need to verify the
names of the feature templates for the VPN0 TLOC interfaces. These templates will need to be
edited to specify both the shaping rate as well as the QoS Map defined in the previous task:

41 | P a g e

Note the feature template names above:

• BR3-INET-TLOC
• BR3-MPLS-TLOC
• BR3-LTE-TLOC

For this lab, we will only edit the BR3-MPLS-TLOC template (in a true deployment, it is best
practice to apply QoS on all transports). Scroll to the top of the window and click on Feature. In
the search box, enter BR3-MPLS-TLOC to locate this template. After the search returns, click
the options button and select Edit:

Click on ACL/QoS in the menu bar to move to this section. You’ll notice an entry for both the
Shaping Rate (in Kbps) and QoS Map. For the MPLS TLOC, the egress rate provisioned by the
ISP is 50Mbps. Click the Entry Settings Box (defaults to a blue checkmark) and select Global
and configure the shaping rate and QoS Map:

42 | P a g e

Click Update to complete the feature template change.

Step 3: Attach ACL to Service-Side VPN Interfaces

Now that the VPN0 MPLS interface has been updated, you still need to attach the ACL on the
service-side interfaces to ensure proper marking of traffic and queue placement. Similar to the
previous step, you now need to identify the feature templates for the VPN10 and VPN20
service-side interfaces.

If you’re still under the Feature tab underneath the Configuration > Templates section, simply
click on Device to move back to the device templates. Otherwise, navigate back to
Configuration > Templates. Click the options button and select Edit again. Once inside the
device template, click Service VPN in the menu bar:

43 | P a g e

Again, note the VPN10 and VPN20 feature template names above:

• BR3-VPN10-Interface-NoVRRP
• BR3-VPN20-Interface-NoVRRP

Scroll back to the top and click on Feature. Search for BR3-VPN10-Interface-NoVRRP first to
edit this template, click the options button, and select Edit. Click on ACL/QoS to move to this
section. Edit the Entry Setting to Global for Ingress ACL – IPv4, select On, and enter the
name of the ACL (QoS-Marking-ACL) created when the policy was defined:

Click Update. Repeat this step for the VPN20 interface feature template.

44 | P a g e

Step 4: Attach the Data Policy to the Device Template
Now that the feature templates have been updated, the device template must be updated to
reference the data policy you created earlier. Similar to traditional IOS-XE configuration, the
policy-map and ACLs are defined globally, which in this case is brought in by the data policy
application in the device template. The interfaces then reference the respective elements to
enable the features.

Navigate back to Configuration > Templates or select Device if you’re still in this section.
Select Edit for the BranchType3Template-CSR device template and click on Additional
Templates. In this section, you’ll see an entry for Policy. This entry allows you to select a
Localized Policy that you’ve previously created to be attached to the device template and
pushed to the device. Click on the option box and select BRANCH-3-QOS-POLICY that you
already defined:

Click Update to finalize the device template edit. Typically, if the device template was attached
to a device, vManage will immediately move to a screen that allows you to edit device values
prior to when vManage populates the configuration before pushing to the device. However,
because the template was detached at the beginning of this section, it reverts to the device
template page.

Step 5: Re-attach the Device Template to BR3

Click on the options button next to the Branch 3 device template and select Attach Devices.
Select BR3-CEDGE1, click the right-facing arrow to move it into the Selected Devices table,
and click Attach:

45 | P a g e

The next screen presents an option to edit the device variables. These will be pre-populated
and do not require edits. Click Next to move to the preview and attach window. On the left-
hand pane, the device will be presented. Click this to see a configuration preview of what will
be pushed to the device. The default tab presents the final configuration that will exist on the
device after the push is complete. To see a view of differences between what is present vs.
what will exist after the push, click on Config Diff at the top:

When scrolling down, you should see the QoS and ACL additions followed by the localized
policy. These will be highlighted in green, meaning these are additions to the configuration.

46 | P a g e

Red lines indicate lines that will be removed or replaced. In the sample output above, these
are internal parameters that aren’t inherently necessary to device operation and can be
ignored. A sample of some of the added configuration is provided below:

Once the review is complete, click Configure Devices. If completed correctly, the
configuration should now be successfully pushed to the device.

Step 6: Validate the Policy Push

Now that the policy has been pushed, you may wish to validate the configuration on the
device. SSH to the BR3-CEDGE1 router and issue show sdwan run policy:

47 | P a g e

Above, you can see a segment of the configuration pushed. You can also check the interface
and QoS configurations using show sdwan run.

Task 3: Troubleshoot Slow Browsing Speeds

After deploying the site over a weekend, users in the production and replication VPNs (10 and
20) have started complaining about slow response times in browsing. You know that a QoS
policy was applied based on application specifications from the business. Recall that HTTP
traffic is being classified as BULK-DATA and has been provisioned with 20% of the egress
bandwidth on any TLOC it exits. This section will walk you through several troubleshooting
tools and techniques to help validate forwarding through XE SDWAN routers and identify root
cause.

Step 1: Validate Flow Information Using cFlowd

In Task 1, before saving the policy, the Netflow box was checked. As the lab guide noted at
that time, enabling this feature turns on flow-visibility inside of the local policy configuration.
This provides per-flow data to allow you to confirm characteristics and forwarding for these
flows.

Navigate to Tools > SSH Terminal in vManage. Select BR3-CEDGE1 to SSH into the router.
The default credentials are admin/admin:

48 | P a g e

Execute show sdwan app-fwd cflowd flows to see the flow data recorded currently:

Notice in the output above that a flow is displayed showing a destination of port 80 from
VPN20 but the DSCP marking is 0. The reason for this is where cFlowd falls in the forwarding
path. In this case, the flow is recorded before the marking is put in place so you cannot
confirm if the correct marking is present. To confirm, filter on all flows with a destination port
of 80 with show sdwan app-fwd cflowd flows | in dest-port 80:

49 | P a g e

As you can see from this output, there are multiple HTTP flows and all show the same thing.
You need to be able to verify that these flows are being marked correctly. Fortunately, there is
a tool available both in IOS-XE as well as XE SDWAN routers that provides detailed datapath
forwarding information that will allow you to determine how features are interacting with the
packet. This tool is known as Packet Trace. It provides the ability to inspect the packet inside
of the FIA (Feature Invocation Array) and understand how the packet is both modified and
forwarded.

Step 2: Run Packet Trace to Verify DSCP Marking

In order to run Packet Trace, there are three steps required:

1) Move the device to CLI mode.

2) Configure an ACL to match the traffic in question and serve as a filter.
3) Configure Packet Trace and verify the data recorded.

As was done in Task 2, Step 1, move the device to CLI mode. After the device is in CLI mode,
you now need to configure an ACL to match traffic with a destination port of 80 and a source
IP in the VPN20 subnet (10.5.21.0/24). Configure this as follows:

50 | P a g e

Notice that the ACL contains only a single entry to match this traffic. Because this ACL isn’t
applied to an interface and will be applied as a traffic filter for the capture, you want to ensure
this is restricted to just the traffic of interested to avoid diluting the capture.

With the ACL created, you now need to configure Packet Trace:

In this configuration, there are 4 key commands:

1) debug platform packet-trace packet 64 fia-trace

• This command defines the number of packets in the buffer (64) and enables FIA
tracing, which provides granular detail about the path of the packet through the
router. Note the output above shows 8192, which is the maximum size but may
exceed memory thresholds on some platforms. Use 64 for this lab.

51 | P a g e

 2) debug platform packet-trace copy packet input l2 size 2048
• This command dictates to the router what information you want to capture about
the packet. In this case, the router will collect information starting with the L2
header and the size of each buffer is set to the maximum (2048 bytes).
3) debug platform condition ipv4 access-list HTTP both
• This command determines what type of traffic (IPv4 or IPv6) to collect, applies
the ACL as the filter for that traffic, and sets the direction to capture on.
Optionally, if you want to capture only on a select interface, this can be provided
as input into the command.
4) debug platform condition start
• This enables the condition, effectively enabling Packet Trace to run.

Once Packet Trace is enabled, you now need to verify if any traffic is being captured. Execute
show platform packet-trace summary to get a summary view of any packets collected:

In the output above, you see packets forwarding out of Gig2, which is the MPLS TLOC, and
also some packets dropped due to WRED. Note that the egress TLOC may vary depending on
how the traffic is hashed. You also know that with congestion, it’s expected to see drops with
WRED enabled so for now, you can ignore these outputs as you need to validate the marking
on this traffic first.

Run show platform packet-trace packet <#> for one of the packets forwarded outbound to
check the details of the packet:

52 | P a g e

The first thing to notice is the L3 and L4 information of the frame. You can confirm this is HTTP
traffic destined to an IP in the DC.

Further down in the FIA trace, you can also see the SDWAN ACL IN action. Within this section,
the ACL matched this traffic to sequence 3. In this case, the sequences start at a base of 0,
meaning that the first sequence is 0, the next will be 1, and so on. This is important to note
when correlating this sequence number to the actual sequence number in the ACL.

You can then confirm that the DSCP value was defined as 10. This confirms that the traffic is
being marked properly. To further confirm, scroll down in the output:

Here, you can see the queue is set to 2 as expected for QoS.

53 | P a g e

Step 3: Check for Other Internet Traffic
Although the business requirement was to match HTTP traffic, after seeing the correct marking
on the traffic, it may be worthwhile to look for other internet traffic, namely on port 443 as it’s
possible the impacted traffic isn’t HTTP but HTTPS instead.

First, check cFlowd for dest-port 443:

Immediately, you can see there are a lot of HTTPS streams active as well. From the earlier
section, you know that this should be falling into the DEFAULT queue because this isn’t being
matched or marked. To confirm, stop Packet Trace, modify the ACL, and start it again:

Next, check the SDWAN ACL IN and SDWAN QoS Output FIA entries to validate queuing:

54 | P a g e

You can see that no marking is being applied as this matches the DEFAULT class and this is
placed into queue 4 for QoS. Because the bandwidth in this queue is lower, you suspect that
this is the source of the slowness reported. You now need to modify the ACL to include port
443 for correct marking.

Step 4: Modify Localized Data Policy for HTTPS and Verify Behavior
Navigate back to Configuration > Policies and in the top right corner, click on Custom
Options and select Access Control Lists under Localized Policy:

55 | P a g e

Beside the QoS-Marking-ACL, click the options button and select Edit:

Click the pencil icon for sequence 4 (or the sequence covering HTTP traffic) and add 443 to
the Destination Port section:

56 | P a g e

After editing, click Save Match and Actions and then Save ACL Policy.

Once done, re-attach the template to the device as defined earlier in the QoS section and
check Packet Trace again, following the instructions in Step 3, to ensure the marking and
queue has now been updated.

After doing so, you confirm with the branch that the user experience has improved and is now
performing as expected.

57 | P a g e

Application-Aware Routing Policy
As enterprise networks continue to grow, the need to maintain healthy across the WAN is ever
increasing. As a result, having the ability to monitor the health of a circuit while avoiding brownout
and blackout conditions is key. With the use of Application Aware Routing (AAR), customers can
set predictable SLAs for all critical applications. This provides multiple hybrid active-active link
scenarios, while enabling real-time enforcement around network problems.

This section of the lab demonstrates how to create an AAR policy and apply it to your activated
Centralized Policy. When creating the application policy, it is important to understand the order
of operations for traffic flow through the router. As you can see in the figure below, traffic will
first be marked via Localized Policy, if applicable. After passing through the local policy, traffic
will then look at the AAR policy to see which path is preferred, if any. Finally, traffic traverses the
Centralized Data Policy where forwarding classes can be set. It is also possible to match and
mark traffic in the Centralized Data Policy rather than the Localized Policy. If such is the case,
then you will want to match traffic via the same DSCP, ACL, and/or Application in both your AAR
Policy and Data Policy.

Local Policy Centralized AAR Centralized Data

Since the Localized Policy was created in the previous section, we could make our AAR policy
to perform all matches based on DSCP. Instead, we will create the AAR policy to plan for the
future where our QoS policy is utilizing DPI, and all match is performed via the Centralized Data
Policy. For this, we will use the same match sequences that were used in the previous QoS
section. Application of the AAR policy will be applied based on the previously created site-lists.

Task 1: Creating Lists and Application Aware Routing Policy

Step 1: Create Policy SLA Lists

As there is no application policy currently built, traffic is performing equal cost multipath
routing (ECMP) as the default behaviour. As stated above, our policy will be created using the
same match sequences found in the local policy. Before we can begin, we must first define the
SLA’s that will be used in the network. These will be created inside the policy generator.

To begin, navigate to the custom traffic policy generator

Navigate Configuration > Policies > Custom Options > Lists > SLA Class

We will build four (4) different SLA classes, as shown in the table below. When building your
SLA classes, it is important to remember that the BFD probes run in echo mode. This means in
a full-mesh environment, you will be looking at the round trip time when setting SLA. This is
reflected in the table below, where voice requires 150ms of one way latency, meaning that our
SLA is set to 300ms.

58 | P a g e

 SLA Class Name Loss (%) Latency (ms) Jitter (ms)

VOICE 2 300 60

VIDEO 2 300 40

CRITICAL 5 400

NON-CRITICAL 10 500

To configure the SLA Classes select New SLA Class List and fill in the VOICE SLA class based
on the above table and click Add. You will notice the other 3 classes have been pre-
configured:

Step 2: Create Application Aware Routing Policy

We are now ready to build the AAR Policy.

Select Custom Options > Traffic Policy

59 | P a g e

At this point, you can see there are three types of policies that can affect traffic: Application
Aware Routing, Traffic Data, and Cflowd. We will focus only on Application Aware Routing
at this time.

Select Add Policy > Create New.

Name the policy AAR-v01 and use the same in the description

Select Sequence Type and click on the three option dots for App Route. Select Rename and
name this entry Voice. This is a best practice so that you can easily modify your policy later for
specific traffic classes.

Next click on Sequence Rule and choose DSCP as match condition. Set the DSCP value to 46
to match voice traffic as it is standard to trust phone markings.

Select Action > SLA Class List. The SLA Class List will be VOICE as we had previously
configured with a Preferred Color of mpls. Click on Save Match and Actions.

60 | P a g e

Repeat the steps for the remaining entries in the table below by adding a sequence type,
renaming the sequence to the associated application group and match sequences.

Notice that we are matching on the same values as matched in our QoS policy. While we did
mark DSCP in the previous section, we are setting our AAR policy for any future considerations
of using a Centralized Data Policy.

When matching on multiple values, if entered in the same entry (multiple DSCP values), they
will operate as an “OR” statement. If entered for multiple match conditions (Source Port,
Protocol, etc.), they operate as an “AND” statement.

SLA Class Preferred

Sequence Name Match Value
Color
DSCP VOICE mpls
- 46
Voice Source Data
Prefix
VOICE-TRAFFIC

DSCP VIDEO mpls

Video
- 32 34
DSCP CRITICAL mpls
- 26
Critical
Destination
Port 45000
Destination NON- biz-internet
Bulk-Data Port 80 CRITICAL

NON-
Default
CRITICAL

Notice that for Default, we are not setting a match value nor a preferred color. Like was shown
in the previous task, we are matching on everything that does not match a previous sequence.
By not setting a preferred color, traffic will still do ECMP, but now with an SLA to provide
brownout detection.

61 | P a g e

Once all sequences have been created, click on Save Application Aware Routing Policy at
the bottom of the page.

Task 2: Apply Policies for App Visibility

Step 1: Apply AAR Policy to Centralized Policy

Now that we have created the AAR policy, we need to attach it to the Centralized Policy. This
can be done by simply attaching the template to the already activated policy.

Click on Centralized Policy at the top of the page to bring us back to the main policy page.

Select the Option dots for the centralized policy and click Edit.

Inside the policy, navigate to Traffic Rules at the top of the page. Once again we are at the
Application Aware Routing section by default.

Select Add Policy > Import Existing > AAR-v01 > Import

62 | P a g e

Navigate back to Policy Application and then select Application Aware Routing. Here we will
apply the AAR policy based on Site-List and VPN List.

Select New Site List and VPN List

Site List > AllBranches AllDC

VPN List > corpVPN

Select Add > Save Policy Changes > Activate B.

Step 2: Apply Local Policy for App Visibility

To ensure that applications are visible for validation and to check cFlowd, app-visibility must
be configured. To do this, we will copy the policy created for Branch3 in the previous section
to use on Branch1.
*As a function of this exercise, we will NOT be configuring QoS on the interfaces.

Navigate to Configuration > Policies > Localized Policy.

63 | P a g e

Select BRANCH-3-QOS-POLICY. Next, select Option > Copy and name the copy BRANCH-
1-QOS-POLICY.

We will now apply the local policy to Branch1:

Navigate to Configuration > Templates > BranchType1Template-CSR > Edit > Additional
Templates > Local Policy > BRANCH-1-QOS-POLICY > Update > Next > Configure Devices
and ensure you select the box to configure two devices.

Step 3: Validate AAR Policy

With the app-route policy now applied and activated, it is time to validate traffic is flowing as
desired. Since cFlowd is no longer exported to vManage, we will verify using the SSH Terminal
inside vManage.

64 | P a g e

Navigate Tools > SSH Terminal > BR1-CEDGE1 and the login will be admin/admin:

Once logged into the device, issue the command “show sdwan app-fwd cflowd flows”.
Notice that traffic is being identified based on both DSCP and applications.

Another way to look at the output is using the table view, with the command show sdwan
app-fwd cflowd flows format table:

65 | P a g e



Task 3: Troubleshoot and Tune Application Routing

Step 1: Modify AAR Policy

You have received a report that Citrix traffic is not performing as desired. If we look at the
initial requirements Citrix was being categorized in the Default Traffic class, performing ECMP
across both available paths. It has been decided to move Citrix traffic to Critical for the AAR
policy, and to prefer the MPLS path at all times unless the MPLS Circuit is down.

Before we make the change, it’s important to look at the flows. This can be done inside
vManage using Troubleshooting Commands. Moving back to the device commands will allow
access to these tools: Monitor > Network > BR1-CEDGE1 > Troubleshooting > App Route
Visualization

66 | P a g e

Once in the App Route Visualization, select DC1-VEDGE1 | 10.1.0.1 as the Device. Click on the
down arrow for traffic filter and select DPI > Application > Citrix > Go

By default, the graph will show the traffic visualization over the last 24 hours. Click on 1h
instead to see a break down for the past hour. Looking at the graph, you can see where traffic
is utilizing both mpls and biz-internet paths.

Navigating back to the Policy Screen, we will first create an application list to match Citrix. Go
to Configuration > Policy > Custom Options > Lists.

By default, Application List is first, so click on New Application List. In this option, we can
choose Application or Application Family. We are only worried about one specific application,
Citrix, so we will stick with Application.

Name the Application List Citrix and in the Search Bar, we will select Citrix. Once completed,
click Add.

We can now navigate to the app-route policy by selecting Custom Options > Traffic Policy.

67 | P a g e

Once in the Traffic Policy, select Options > Edit for AAR-v01.

Select Critical > Sequence Rule with the following Match and Action
Match: Application/Application Family List > Citrix
Action: SLA Class > Critical
Preferred Color > mpls
‘strict’ box is checked

The strict option indicates that if the mpls is out of SLA, we will fail over to one of the alternate
paths as long as they are within SLA. However, if all paths are out of SLA for this particular
traffic, the traffic will be dropped.

We can now save the AAR policy. Again, since the policy is activated, the configuration will be
pushed to both vSmarts as a configuration change, so we need to select the box to push to
multiple devices. Select Save Match and Actions > Save Application Aware Routing Policy >
Activate > Next > Next:

68 | P a g e

Step 2: Verify Citrix Traffic
With the new policy in place, we can now check for Citrix traffic.

Navigate to Tools > SSH Terminal > BR1-CEDGE1 and again login as admin/admin.

Perform the command show sdwan app-fwd cflowd flows format table. If a Citrix flow is
running you will be able to see traffic going out of GigabitEthernet4, which is the mpls circuit:

After some time, you can look again at the App Route Visualization tab to see the trend move
from the biz-internet path to the mpls path.

With Citrix traffic flowing over mpls, complaints have stopped and employees are now having
no issues with Citrix traffic.

69 | P a g e

Branch Security with Zone-Based Firewall
Consider a scenario where we have two different VPN’s configured on a single SDWAN device
or across multiple SDWAN devices. In one of the VPN’s, we have shared resources that we
want to restrict access to. The shared resources can be printers or confidential customer data.
We want the users in the Data VPN to be able to access the shared resources but we do not
want the data traffic to flow in the other direction. We can use a Zone-Based Firewall Policy to
achieve this.

Zone-Based Firewalls are a type of Localized Data Policy that allows stateful inspection of
TCP, UDP, and ICMP data traffic flows. Traffic flows that originate in a given zone are allowed
to proceed to another zone based on the policy between the two zones. A zone is a grouping
of one or more VPNs. Grouping VPNs into zones allows you to establish security boundaries in
your overlay network so that you can control all data traffic that passes between zones.

Zone configuration consists of the following components:

1. Source Zone — A grouping of VPNs where the data traffic flows originate.

2. Destination Zone — A grouping of VPNs where the data traffic flows terminate. A VPN
can be part of only one zone.

3. Zone-Based Firewall Policy — A data policy, similar to a Localized Data Policy, that
defines the conditions that the data traffic flow from the source zone must match to
allow the flow to continue to the destination zone. Zone-Based Firewalls can match IP
prefixes, IP ports, and the protocols TCP, UDP, and ICMP. Matching flows can be
accepted, inspected, or dropped and the packet headers can be logged. Nonmatching
flows are dropped by default.

4. Zone Pair — A container that associates a source zone with a destination zone and that
applies a ZBFW policy to the traffic that flows between the two zones.

Matching flows that are accepted can be processed in two different ways:

• Inspect — The packet's header can be inspected to determine its source address and
port. The address and port are used by the NAT device to allow traffic to be returned
from the destination to the sender.

• Pass — Allow the packet to pass to the destination zone without inspecting the packet's
header at all. With this action, the NAT device blocks return traffic that is addressed to
the sender.

For this lab, we will create a new control policy rather than modify the existing policy for
clearer understanding.

70 | P a g e

Task 1: Configure Route-Leaking between VPN10 and VPN40.
By default, in Cisco XE SDWAN routers, we do not export routes between two different service
VPN’s. The policy builder in vManage can be used to build, preview and manage the different
polices you would like to apply in your network. Before we build the policy to export the routes
between the two VPN’s, let’s verify the routing table on the routers.

Step 1: Verify the Routing Table on the Routers.

The first thing to do is verify the current routing table for both VPN10 and VPN40. Navigate to
the SSH Terminal of vManage by choosing Tools > SSH Terminal > BR3-CEDGE1 and use the
credentials admin/admin.

Once logged into the device, run show ip route vrf <Service VPN ID>:

Note that in both VPN 10 and VPN 40, we are only seeing routes for the respective VPN.

71 | P a g e

Step 2: Navigate to the Policy Builder
Inside the vManage GUI, navigate to Configuration > Policy > Add Policy:

Step 3: Create a New Control Policy

Once the policy builder loads, we should land on the Centralized Policy tab. Once you are
there, click on Add Policy:

Step 4: Create Site-Lists

In the Create Groups of Interest tab we will verify the Site-Lists and VPN Lists previously
created. In the Cisco SD-WAN policy framework, we apply policies to sites based on site-lists.
We can also use site-lists as a part of a match condition in a control policy. Select Site from the
options displayed.

We can see that the CSR located in Branch 3 is already part of the site-list Site-Type3. If you
want to create one, you can create one exclusively for Branch 3.

72 | P a g e

Step 5: Verify VPN Lists
Next, we will verify that we have both the guestVPN (VPN40) and corpVPN (VPN10) VPN lists
created in our environment:

Click on Next to proceed to the Topology and VPN Membership section.

Step 6: Create the Control Policy

In the Configuring Topology and VPN Membership section, we will configure the match and
actions to accomplish route-leaking.

73 | P a g e

Click on Add Topology > Custom Control (Route & TLOC):

In the next screen, fill in the Name field with Leak-routes and specify the Description as
Policy to leak routes between vpn 10 and vpn 40. Click on Sequence Type > Route:

As with any policy language, the control policy has a Match and Action framework. We have
many match conditions which we can use for every sequence and a variety of actions which
can be performed. In route-leaking, we are going to match on routes belonging to a VPN
coming from a site and then export it to the other VPN. Our Match condition is going to be
based on Site and VPN. In our Lab, Branch 3 has VPN40 and VPN10. We are going to match
on VPN10 routes from Branch 3 and export them to VPN40.

From the Match parameters displayed, select Site and VPN. Fill in the name of the Site List
(Site-Type3) and the VPN List (corpVPN) and then move to the Actions tab where you will
select Accept > Export To > guestVPN. Then click on Save Match and Actions:

74 | P a g e

Following the same procedure as explained above, create the second sequence to export the
VPN40 (guestVPN) routes from Branch 3 to VPN10 (corpVPN). Click on Save Match and
Actions to save the second sequence:

At this point, we have exported the two VPNs to each other. However, the Default Action of a
control policy is Reject. To ensure that the TLOCs establish between all devices, and routes
are receieved, we will need to adjust this behavior. One option is to setup extra sequences as
we did in an earlier section. Another option, which we will use in this example, is to change the
Default Action.

Select Default Action > Edit > Accept > Save Match and Actions:

75 | P a g e

Save the control policy by clicking on Save Control Policy. This then takes you to the next
section where you can configure App-Route and Data Policies. Because we are not concerned
about those for this task, click on Next until you land on the Apply Policies to Sites and VPN’s
section.

Step 7: Apply the Policy

After we are done creating the Site-Lists, VPN Lists, and the policy, we can then proceed to
apply it to the respective sites. A control policy can be applied in two directions: inbound and
outbound. When you specify the direction of the control policy, you should look at it from the
perspective of the vSmart. If you have applied it in the inbound direction, this means the rules
defined in the policy are applied when the vSmart receives the Routes/TLOCS from the router.
We know that the vSmart is the central brain of the network and it works as a route-reflector to
reflect the routes from one router to another. When we do route-leaking, we need to apply the
policy in the inbound direction so that the routes from VPN40 get exported into the VPN10
routing table on the vSmart and vice-versa. The exported routes are then populated to the
other SDWAN nodes in the network as a part of an OMP update.

Name the policy Policy-to-leak-routes with a description of Policy to leak routes between
vpn 10 and vpn 40. In the Topology tab, click on New Site List. Select the Inbound Site List
and select Site-Type3, then click Add.

76 | P a g e

Step 8: Preview the Policy and Save
Click on Preview and ensure the policy has no errors:

If everything looks correct, click on Save Policy.

Step 9: Activate the Policy

After the policy gets saved, you will see it present under the Configuration > Policies tab.
Click on Options in the right-most column and from the drop down that appears, click on
Activate:

77 | P a g e



Make sure that the policy push is successful.

Task 2: Confirm that Route-Leaking is Operational

Look at the routing table on your devices by running show ip route vrf <VRF name>. You
should see the 10.5.40.0/24 route now in VPN10 and the 10.5.10.0/24 route in VPN40.

Step 1: Validate Route Leaking

Navigate to Tools > SSH Terminal > BR3-CEDGE1 and login using the credentials
admin/admin:

78 | P a g e

Verification of route-leaking must occur behind the XE SDWAN device. To verify, we will SSH
to the VPN40 host and ping the VPN10 host.

Using the command ssh -vrf 40 -l administrator 10.5.40.10 and a password of C1sco12345,
SSH from the router to reach the VPN40 host. Once logged in, enter ping 10.5.10.10 from the
terminal:

Task 3: Create the Zone-Based Firewall Policy

When we try to configure a Zone-Based Firewall Policy in Cisco SD-WAN, we need to do the
following:

1. Configure Zones (Source Zone and Destination Zone)

2. Policy, which contains sequences with match and action clauses
3. Zone-Pairs (define what policy should be applied for traffic between zones)

79 | P a g e

Step 1: Configure Zones
We first need to create two zones. The first is the Source-Zone (VPN where traffic is
originated) and the other is the Destination-Zone (VPN where traffic is destined). In the
vManage GUI, navigate to Configuration > Security. You will see a Custom Options dropdown
in the top-right of the screen. In the dropdown, select Lists > Zones. You can see that Zone10
has already been created. We will be creating Zone40 by clicking New Zone List and creating
Zone40 with the Entry as 40 and then clicking Add:

Step 2: Navigate to Policy

Navigate to Configuration > Security on the vManage GUI and then click on Add Security
Policy:

80 | P a g e

Select Compliance > Proceed from the list which is displayed:

In the Add Security Policy tab, click on Add Firewall Policy and select Create New. Click
Next:

81 | P a g e

Step 3: Configure the Policy

We need to configure the zone-pair, ame of the policy, and the sequences in this step. Provide
the Name (ZBFW-vpn10-and-vpn40) and Description (Policy to restrict traffic between vpn
10 and vpn 40) of the policy and then click on Sequence Rule to create the first sequence.

In this exercise, we are trying to allow guests sitting in the guestVPN (VPN40) to reach a single
printer in the corpVPN (VPN10). To demonstrate this, we are going to use ping to verify that
we have connectivity from VPN10 to VPN40 and also from VPN40 to VPN10. We are going to
create a policy which is going to pass ping traffic but deny anything else.

In the first sequence of the policy, under the Match section, select Protocol as the match field.
Input 1 (ICMP protocol number). Next, click Actions, where you will change from the default
Drop action to Pass before selecting Save Match and Actions:

82 | P a g e

Step 4: Configure Zone-Pairs
A Zone-Pair is used to make the ZBFW policy take effect. For the Zone-Pair configuration, we
need to define the following: Source-Zone, Destination-Zone and Zone-Based-Policy.

Source-Zone – VPN from which traffic is going to be originated.

Destination-Zone – VPN to which the traffic is destined to.

Click on the Apply Zone-Pairs radio button on the top of the screen.

In the corresponding window, we need to define the Source Zone and Destination Zone.

In our case, we are only going to allow traffic from guestVPN (VPN40) to the corpVPN (VPN10).
Select Zone10 to be our Source Zone and Zone40 to be our Destination Zone and then click
Save. Select Save Firewall Policy:

Step 5: Build the Policy

You will not be configuring Intrusion Prevention in this session. Click Next until you get to
Policy Summary.

83 | P a g e

In the Policy Summary section, fill in the details for the Name and Description:

You can preview the policy by clicking on Preview. When everything is correct, click on Save
Policy:

84 | P a g e

Step 6: Apply the Policy
The Zone-Based Firewall policy is a Localized Policy which must be applied locally on the
router. It will not be pushed from the vSmart centrally. We must include the policy as a part of
the device template that is pushed or attached to the XE SDWAN router.

Navigate to Configuration > Templates on vManage and select the template which is attached
to the router in question. Click Edit:

Go to the Additional Templates section by clicking on the Additional Templates tab. Select
Security Policy and from the drop-down, select the Zone-based-Policy1 which we just
created and then click Update.

85 | P a g e

Once you click on Update, the configuration should be pushed to the router.

Task 4: Validate Zone-Based Firewall

At this time, you are ready to validate the ZBFW that you just created.

Step 1: Check Connectivity between VPNs

Connect back to the VPN40 host at Branch 3 via BR3-CEDGE1 as we did previously.

Connect via Tools > SSH Terminal > BR3-CEDGE1 with login admin/admin.

Once connected, SSH to the VPN40 host again using the command ssh -vrf 40 -l
administrator 10.5.40.10 and using the password C1sco12345. Then, issue ping 10.5.10.10
as we did previously. You can see that while packets are being transmitted, none are received:

Step 2: Run Packet Trace to Check Return Traffic

In order to run Packet Trace, there are three steps required:

1) Move the device to CLI mode.

86 | P a g e

 2) Configure an ACL to match the traffic in question and serve as a filter.
3) Configure Packet Trace and verify the data recorded.

Move the device to CLI mode. Once that is done, configure an ACL to match the traffic
between 10.5.40.10 (VPN40 host) and 10.5.10.10 (VPN10 host). Configure this as follows:

With the ACL created, you now need to configure Packet Trace:

In this configuration, there are 4 key commands:

87 | P a g e

 1. debug platform packet-trace packet 64 fia-trace
• This command defines the number of packets in the buffer (64) and enables FIA
tracing, which provides granular detail about the path of the packet through the
router. Note the output above shows 8192, which is the maximum size but may
exceed memory thresholds on some platforms. Use 64 for this lab.
2. debug platform packet-trace copy packet input l2 size 2048
• This command dictates to the router what information you want to capture about
the packet. In this case, the router will collect information starting with the L2
header and the size of each buffer is set to the maximum (2048 bytes).
3. debug platform condition ipv4 access-list zbfw both
• This command determines what type of traffic (IPv4 or IPv6) to collect, applies
the ACL as the filter for that traffic, and sets the direction to capture on.
Optionally, if you want to capture only on a select interface, this can be provided
as input into the command.
4. debug platform condition start
• This enables the condition, effectively enabling Packet Trace to run.

Once Packet Trace is enabled, you now need to verify if any traffic is being captured. Execute
show platform packet-trace summary to get a summary view of the packets collected:

88 | P a g e

In the above output, you can see there are packets being forwarded and dropped.

Run show platform packet-trace packet <#> for one of the packets forwarded outbound to
check the details of the packet:

89 | P a g e

Looking at the Layer 3 information for this packet, you can determine that this is the ICMP echo
request packet from the host in VPN40 to the host in VPN10. Moving down the FIA trace, we
can also see the ZBFW section. Within this section, we can see that the Action here is FWD
and it also shows us the zone-pair we defined in our policy:

90 | P a g e

Let’s take a look at why the ICMP echo reply is getting dropped. Run show platform packet-
trace packet <#> for a dropped packet:

91 | P a g e

From the Layer 3 information in the above packet, we can determine that this is the ICMP echo
reply from the host in VPN10. Let’s walk down the FIA trace to determine why this is getting
dropped:

92 | P a g e

We can see that the ZBFW feature is dropping the ICMP echo reply and the reason provided is
Same zone without Policy. You can also see that this traffic does not match any zone-pair
defined. For a ZBFW policy, you must have a zone-pair to account for traffic coming in the
reverse direction also.

Step 3: View ZBFW Rules

Navigate to the Security Policy that we created to review our rules via Configuration >
Security > Edit:

Once in the Security Policy, select Firewall at the top and then Options > Edit for the
previously created Firewall Policy:

93 | P a g e

If we look at the rules, there is only one created, which allows traffic from VPN40 to VPN10.
We did not create a rule allowing return traffic.

Step 3: Modify ZBFW Rules

At this point, we have realized that a second rule is needed to allow return traffic. We will do
so by creating a new Zone-Pair Rule with a Source Zone of Zone10 and Destination Zone of
Zone40.

Click on Apply Zone-Pairs at the top of the screen and then click on the Add sign before
filling in the new pair before clicking Save:

94 | P a g e



At this point, you should see two sources for the Rule we have created. Click on Save Firewall
Policy.

Save the changes we made to the policy and attach the device to the template so that we can
push the new modified policy.

Step 4: Verify Connectivity

Log back into BR3-CEDGE1 via the CLI by using the SSH Terminal:

Tools > SSH Terminal > BR3-CEDGE1 > admin/admin

Once in the CLI, we will once again connect to the VPN40 host using the command ssh -vrf 40
-l administrator 10.5.40.10 and then test connectivity using ping ping 10.5.10.10:

95 | P a g e



If we exit from the host (exit), we can then check the ZBFW zonepair-statistics via the CLI
using show sdwan zbfw zonepair-statistics and see that our counters for “Inspect Pass”
have increased. Note that the values may be different depending on the number of packets
sent.

Now, the branch is setup to allow guest users to print to a corporate printer and our use case
is finalized.

96 | P a g e

Bonus Task: Configure a Centralized Data Policy for QoS
In the previous tasks, you learned how to create a local policy to define both the marking ACL
as well as a QoS map to define queuing for branch traffic. In many deployments, marking
characteristics will be the same across multiple (or all) branches. Because of that, it’s possible
(and potentially more desirable) to create a centralized data policy that sets the forwarding
class for QoS that can be applied to multiple sites at once through application on the
vSmart(s). While the local QoS policy is still required to define the class-maps and queuing
structure, this approach of using a centralized data policy enables you to minimize the local
policy configuration to QoS and tweak the class bandwidth parameters as required while
allowing the ACL marking to be applied once through the vSmarts for all sites and remain
static.

This task will show you how to define the necessary VPN and Site Lists, create the ACL,
activate the centralized policy on the vSmarts, and validate the configuration locally on the
device.

Task 1: Create Centralized Policy to Apply for QoS

Step 1: Begin a New Centralized Policy

Inside of the vManage GUI, navigate to Configuration > Policy from the menu:

Click on Add Policy.

Step 2: Validate the Site and VPN Lists

Once the Groups of Interest tab loads, click on Site:

97 | P a g e

When configuring a Centralized Policy, the Site List is used to define which sites the policy will
be applied to from the vSmart(s). For example, in a large deployment with site numbers 1-
1000, you may wish to create a policy that applies to sites 100-150, a different policy that
applies to sites 200-250, and so on. As you’ll see in the last step of the Policy Builder for this
exercise, the Site List is specified when the policy is applied.

In the list above, you’ll notice that a Site List exists covering Branch 3, which you know from
the topology is Site 500. If a covering list doesn’t already exist, click on New Site List to add a
list with the respective site ID(s).

In addition to the Site List, you also need to confirm a VPN list exists for both VPN10 and
VPN20 so that the policy can be applied to both. Click on VPN:

98 | P a g e

Again here, you can see that a VPN list exists for both 10 and 20 so no new list is required.
However, it can be added in the same way as other lists if required.

Click Next to move to the Configure Topology and VPN Membership tab. Neither of these are
required for this Centralized Policy. Click Next again to move to the Configure Traffic Rules
tab.

Step 3: Configure the Marking ACL

Once you’re on the Configure Traffic Rules tab, click Traffic Data. Next, click Add Policy and
select Create New:

99 | P a g e

Similar to the last section, the data policy configuration template is displayed. Name the policy
QoS-Marking-ACL and provide a description:

Click on Sequence Type and the following dialog box is displayed:

100 | P a g e

Select QoS as you’ll be defining the ACL to set the forwarding-class based on DSCP. After
that is selected, click Sequence Rule to add a new sequence. Choose Source Data Prefix as
seen below and select VOICE-TRAFFIC, which was created in the previous section:

Next, click Actions and select DSCP and Forwarding Class. Enter 46 for the DSCP value and
VOICE for the Forwarding Class:

101 | P a g e

After clicking Save Match and Actions, you should see the following:

Repeat Task 1, Steps 10-11 while using Table 3 as a reference for the DSCP values if needed.
Once all sequences have been added, click Save Data Policy and on the following screen,
click Next to move to the Apply Policy to Sites and VPNs section.

Step 4: Define the Site and VPN Lists for Policy Application
In the final step of the Policy Builder, the Site and VPN Lists that were confirmed earlier in this
task must be applied so that when the policy is activated on the vSmarts, the controller knows
which site(s) should receive the policy and which VPNs should be programmed for it.

102 | P a g e

Because the policy created is a Data Policy, this will show up under the Traffic Data tab. Click
on this tab and the QoS-Marking-ACL shows as an entry for policy application. Name the
policy QOS-CENTRALIZED-POLICY and provide a description as seen below:

Click on New Site List and VPN List. Select the lists that were previously confirmed on the
Groups of Interest tab:

Since the policy will be applied for traffic coming from the service-side, select the From
Service radio button and click Add.

103 | P a g e

After clicking Preview, you’ll see a similar ACL created as was done in the previous task. The
key difference is that the set action defines forwarding-class instead of class for each action
statement.

Scroll down to the bottom of the preview to see the lists and how the policy will be applied:

It’s always recommended to validate the configuration before saving a policy to ensure the
intent matches what you expect for network application. In this case, you can see that for sites
500-599 (Branch 3 is Site 500), each will receive the policy and it will be applied for VPNs 10
and 20 from the service-side. In this case, the configuration aligns with the QoS requirements.
Click Save Policy and the policy now shows in the Centralized Policy section:

104 | P a g e

You now need to edit the current Data Policy and Device/Feature templates in preparation for
policy application.

Step 5: Activate the Policy

Generally, when converting policies, this would be done inside of a maintenance window to
ensure no performance impact. However, this section will walk through this in a way that
should ensure marking is preserved as the new policy is activated the original policy is
modified. First, under Configuration > Policies, click the options button next to the policy just
created and click Activate:

vManage reports that the policy will be activated on both vSmarts.

Step 6: Remove the ACL from Service-Side Interfaces

In Task 2 > Step 3, you attached the ACL to the service-side interfaces. Because the
Centralized Data Policy will now take care of classification and marking, this ACL will no longer
be used. Repeat this step, this time removing the ACL from each service-side interface feature
template. You can either do this one at a time, pushing the changes to the device after each
update, or move the device to CLI mode and then make all edits at once before reattaching the
template as was done earlier in Task 2.

Step 6: Detach the ACL from the Data Policy

Navigate back to Configuration > Policies. Click on Localized Policy and then the options
button beside the policy you created. Click Edit and then Access Control Lists. Beside QoS-
Marking-ACL, click the options button and select Detach. Click on Policy Overview and click
Save Policy Changes. Click Next to move past the device variables screen and apply the
policy. If you check the configuration preview, you’ll see the ACL will be removed from the
router as part of this configuration push.

105 | P a g e

Step 7: Validate the Updated Configuration
Now that the policy changes have been updated, you may first want to validate the
configuration was pushed correctly from the vSmarts. To do this, SSH to the router from the
GUI and run show sdwan policy from-vsmart:

The policy is displayed. To confirm the ACL configuration was removed from the Localized
Policy, check show sdwan run policy. The same validation commands that were executed
previously can now be used to confirm QoS marking and shaping are working as designed.

106 | P a g e

APPENDIX A: Additional SD-WAN Learning Opportunities
DevNet Sessions
DEVWKS-1671: Programmability with SD-WAN
DEVWKS-1218: SD-WAN APIs overview
DEVWKS-2179: Hands-on Viptela Automation Using Ansible and VIRL
DEVWKS-1189: Introduction to the Cisco SD-WAN vManage API
DEVWKS-1682: SD-WAN Automation with Ansible
Breakout Sessions
BRKRST-2558: Cisco SD-WAN as a Managed Service
BRKRST-2091: Cisco SD-WAN (Viptela) Data Center and Branch Integration Design
BRKCRS-2110: Delivering Cisco Next Generation SD-WAN with Viptela
BRKARC-2112: Harnessing the power of Software Defined Branch and SD-WAN
BRKSPG-2017: SD-WAN in Service Provider networks
BRKARC-1004: SD-WAN on Cisco IOS XE Routers: An End-to-End solution view
BRKRST-2559: 3 Steps to Deploy Cisco SD-WAN On-Prem
BRKCRS-2818: Build a Software Defined Enterprise with Cisco SD-WAN and Cisco SD-Access
BRKRST-2791: Building and Using Policies with Cisco SD-WAN
BRKCRS-2113: Cloud-Ready WAN for IAAS and SAAS with Cisco Next-Gen SD-WAN
BRKRST-2093: Next-Gen SD-WAN (Viptela) Deployment, Monitoring, and Troubleshooting
BRKRST-2095: SD-WAN Routing Migrations
BRKRST-2377: SD-WAN Security

Walk-in Labs
LABCRS-1005: Introductory Cisco SD-WAN (Viptela) Topics – Zero Touch Deployment and Configuration
Templates
LABCRS-2000: Cisco SD-WAN Topics: Controlling the Overlay via Centralized Policy

107 | P a g e