Simple Multihoming

ISP Workshops

Last updated 9th December 2015

Agenda
p Why

Multihome?
p The Multihoming Toolset
p How to Multihome Options
p Multihoming to the same AS
p Multihoming to different ASes

Why Multihome?
p Redundancy
n

One connection to internet means the network

is dependent on:
Local router (configuration, software, hardware)
p WAN media (physical failure, carrier failure)
p Upstream Service Provider (configuration, software,
hardware)
p

Why Multihome?
p Reliability

Business critical applications demand

continuous availability
n Lack of redundancy implies lack of reliability
implies loss of revenue
n

Why Multihome?
p Supplier

Diversity

Many businesses demand supplier diversity as

a matter of course
n Internet connection from two or more suppliers
n

With two or more diverse WAN paths

p With two or more exit points
p With two or more international connections
p Two of everything
p

Why Multihome?
p
p

Changing upstream provider

With one upstream, migration means:
n
n
n
n
n

Disconnecting existing connection

Moving the link to the new upstream
Reconnecting the link
Reannouncing address space
Break in service for end users (hours, days,...?)

With two upstreams, migration means:

n
n
n

Bring up link with new provider (including BGP and

address announcements)
Disconnect link with original upstream
No break in service for end users
6

Why Multihome?
p Not

really a reason, but oft quoted

p Leverage:
n

Playing one ISP off against the other for:

Service Quality
p Service Offerings
p Availability
p

Why Multihome?
p Summary:

Multihoming is easy to demand as requirement

of any operation
n But what does it really mean:
n

In real life?
p For the network?
p For the Internet?
p

And how do we do it?

Multihoming Definition
p More

than one link external to the local

network
Two or more links to the same ISP
n Two or more links to different ISPs
n

p Usually
n

two external facing routers

One router gives link and provider redundancy

only

Multihoming
p The

scenarios described here apply equally

well to end sites being customers of ISPs
and ISPs being customers of other ISPs
p Implementation details may be different,
for example:
End site ISP
n ISP1 ISP2
n

Configuration on End-Site
ISPs share config

10

Autonomous System Number

(ASN)
p

Two ranges
0-65535
65536-4294967295

Usage:
0 and 65535
1-64495
64496-64511
64512-65534
23456
65536-65551
65552-4199999999
4200000000-4294967295

(original 16-bit range)

(32-bit range RFC6793)
(reserved)
(public Internet)
(documentation RFC5398)
(private use only)
(represent 32-bit range in 16-bit world)
(documentation RFC5398)
(public Internet)
(private use only)

32-bit range representation specified in RFC5396

n

Defines asplain (traditional format) as standard notation

11

Autonomous System Number

(ASN)
p

ASNs are distributed by the Regional Internet

Registries
n

Current 16-bit ASN assignments up to 64297

have been made to the RIRs
n
n

Around 43000 16-bit ASNs are visible on the Internet

Around 200 left unassigned

Each RIR has also received a block of 32-bit ASNs

n

They are also available from upstream ISPs who are

members of one of the RIRs

Out of 12000 assignments, around 9200 are visible on

the Internet

See www.iana.org/assignments/as-numbers
12

Private AS Application
p

An ISP with customers

multihomed on their
backbone (RFC2270)
-orA corporate network
with several regions
but connections to the
Internet only in the
core
-orWithin a BGP
Confederation

65001
193.0.32.0/24

C
1880
193.0.34.0/24

65002
193.0.33.0/24

65003
193.0.35.0/24

193.0.32.0/22 1880

13

Private-AS Removal
p Private

ASNs MUST be removed from all

prefixes announced to the public Internet
n

Include configuration to remove private ASNs

in the eBGP template

p As

with RFC1918 address space, private

ASNs are intended for internal use
n

They should not be leaked to the public

Internet

p Cisco

IOS

neighbor x.x.x.x remove-private-AS

14

More Definitions
p Transit

Carrying traffic across a network

n Usually for a fee
n

p Peering

Exchanging routing information and traffic

n Usually for no fee
n Sometimes called settlement free peering
n

p Default
n

Where to send traffic when there is no explicit

match in the routing table
15

Configuring Policy
p Assumptions:

Prefix-lists are used throughout

n Easier/better/faster than access-lists
n

p Three

BASIC Principles

Prefix-lists to filter prefixes

n Filter-lists to filter ASNs
n Route-maps to apply policy
n

p Route-maps

can be used for filtering, but

this is more advanced configuration
16

Policy Tools
p Local
n

preference

Outbound traffic flows

p Metric
n

(MED)

Inbound traffic flows (local scope)

p AS-PATH
n

prepend

Inbound traffic flows (Internet scope)

p Subdividing
n

Aggregates

Inbound traffic flows (local & Internet scope)

p Communities
n

Specific inter-provider peering

17

Originating Prefixes: Assumptions

p MUST

announce assigned address block to

Internet
p MAY also announce subprefixes
reachability is not guaranteed
p Current minimum IPv4 allocation is /24
Several ISPs filter RIR blocks on published
minimum allocation boundaries
n Several ISPs filter the rest of address space
according to the IANA assignments
n This activity is called Net Police by some
n

18

Originating Prefixes
p

The RIRs publish their minimum allocation sizes per /8 address block
n
n
n
n
n
n

IANA publishes the address space it has assigned to end-sites and

allocated to the RIRs:
n

AfriNIC:
www.afrinic.net/library/policies/126-afpub-2005-v4-001
APNIC:
www.apnic.net/db/min-alloc.html
ARIN:
www.arin.net/reference/ip_blocks.html
LACNIC:
lacnic.net/en/registro/index.html
RIPE NCC:
www.ripe.net/ripe/docs/smallest-alloc-sizes.html
Note that AfriNIC only publishes its current minimum allocation size, not
the allocation size for its address blocks

www.iana.org/assignments/ipv4-address-space

Several ISPs use this published information to filter prefixes on:

n
n

What should be routed (from IANA)

The minimum allocation size from the RIRs

Net Police prefix list issues

p
p
p
p
p

Meant to punish ISPs who pollute the routing table with

specifics rather than announcing aggregates
Impacts legitimate multihoming especially at the Internets
edge
Impacts regions where domestic backbone is unavailable or
costs $$$ compared with international bandwidth
Hard to maintain requires updating when RIRs start
allocating from new address blocks
Dont do it unless consequences understood and you are
prepared to keep the list current
n
n

Consider using the Team Cymru or other reputable bogon BGP

feed:
www.team-cymru.org/Services/Bogons/routeserver.html
20

How to Multihome
Some choices

21

Transits
p

Transit provider is another autonomous system

which is used to provide the local network with
access to other networks
n
n

Might be local or regional only

But more usually the whole Internet

Transit providers need to be chosen wisely:

n

Only one
p

Too many
p
p
p

No redundancy
More difficult to load balance
No economy of scale (costs more per Mbps)
Hard to provide service quality

Recommendation: at least two, no more

than three

Common Mistakes
p

ISPs sign up with too many transit providers

n
n
n

Lots of small circuits (cost more per Mbps than larger

ones)
Transit rates per Mbps reduce with increasing transit
bandwidth purchased
Hard to implement reliable traffic engineering that
doesnt need daily fine tuning depending on customer
activities

No diversity
n
n

Chosen transit providers all reached over same satellite

or same submarine cable
Chosen transit providers have poor onward transit and
peering

Peers
p

A peer is another autonomous system with which

the local network has agreed to exchange locally
sourced routes and traffic
Private peer
n

Public peer
n

Private link between two providers for the purpose of

interconnecting
Internet Exchange Point, where providers meet and
freely decide who they will interconnect with

Recommendation: peer as much as possible!

Common Mistakes
p
p

Mistaking a transit providers Exchange

business for a no-cost public peering point
Not working hard to get as much peering as
possible
n
n

Physically near a peering point (IXP) but not present at

it
(Transit sometimes is cheaper than peering!!)

Ignoring/avoiding competitors because they are

competition
n

Even though potentially valuable peering partner to give

customers a better experience

Multihoming Scenarios
p Stub

network
p Multi-homed stub network
p Multi-homed network
p Multiple Sessions to another AS

26

Stub Network
AS101
AS100

p
p
p
p

No need for BGP

Point static default to upstream ISP
Upstream ISP advertises stub network
Policy confined within upstream ISPs policy
27

Multi-homed Stub Network

AS65530
AS100

p
p
p
p

Use BGP (not IGP or static) to loadshare

Use private AS (ASN > 64511)
Upstream ISP advertises stub network
Policy confined within upstream ISPs policy
28

Multi-homed Network
Global Internet
AS200

AS300
AS100

Many situations possible

n
n
n
n

Multiple sessions to same ISP

Secondary for backup only
Load-share between primary and secondary
Selectively use different ISPs
29

Multiple Sessions to an ISP

p Several

options

ebgp multihop
n bgp multipath
n cef loadsharing
n bgp attribute manipulation
n

ISP

AS 100
30

Multiple Sessions to an AS
ebgp multihop
p

Use ebgp-multihop
n
n

Run eBGP between loopback addresses

eBGP prefixes learned with loopback address as
next hop

Cisco IOS
router bgp 100
neighbor 1.1.1.1 remote-as 200
neighbor 1.1.1.1 ebgp-multihop 2
!
ip route 1.1.1.1 255.255.255.255 serial 1/0
ip route 1.1.1.1 255.255.255.255 serial 1/1
ip route 1.1.1.1 255.255.255.255 serial 1/2

AS 200

Common error made is to point remote

loopback route at IP address rather than
specific link

1.1.1.1
B

AS 100

Multiple Sessions to an AS
ebgp multihop
p

One serious eBGP-multihop

caveat:
n
n

R1 and R3 are eBGP peers

that are loopback peering
Configured with:

neighbor x.x.x.x ebgp-multihop 2

n

If the R1 to R3 link goes

down the session could
establish via R2

Usually happens when

routing to remote loopback
is dynamic, rather than
static pointing at a link

R1

R3

AS 100

AS 200

R2

Desired Path
Used Path

Multiple Sessions to an ISP

ebgp multihop
p Try

and avoid use of ebgp-multihop

unless:
Its absolutely necessary or
n Loadsharing across multiple links
n

p Many

ISPs discourage its use, for

example:

We will run eBGP multihop, but do not support it as a standard offering

because customers generally have a hard time managing it due to:
routing loops
failure to realise that BGP session stability problems are usually due
connectivity problems between their CPE and their BGP speaker
33

Multiple Sessions to an AS
bgp multi path
p
p
p

Three BGP sessions required

Platform limit on number of paths
(could be as little as 6)
Full BGP feed makes this unwieldy
n

3 copies of Internet Routing Table

goes into the FIB

router bgp 100

neighbor 1.1.2.1 remote-as 200
neighbor 1.1.2.5 remote-as 200
neighbor 1.1.2.9 remote-as 200
maximum-paths 3

AS 200

AS 100
34

Multiple Sessions to an AS
bgp attributes & filters
p
p
p

Simplest scheme is to use

defaults
Learn/advertise prefixes for
better control
Planning and some work
required to achieve loadsharing
n
n
n

Point default towards one ISP

Learn selected prefixes from
second ISP
Modify the number of prefixes
learnt to achieve acceptable load
sharing

No magic solution

AS 200
C

AS 100
35

Basic Principles of
Multihoming
Lets learn to walk before we try
running

36

The Basic Principles

p

Announcing address space attracts traffic

n

(Unless policy in upstream providers interferes)

Announcing the ISP aggregate out a link will

result in traffic for that aggregate coming in that
link
Announcing a subprefix of an aggregate out a link
means that all traffic for that subprefix will come
in that link, even if the aggregate is announced
somewhere else
n

The most specific announcement wins!

37

The Basic Principles

p

To split traffic between two links:

n
n
n

Announce the aggregate on both links ensures

redundancy
Announce one half of the address space on each link
(This is the first step, all things being equal)

Results in:
n
n
n

Traffic for first half of address space comes in first link

Traffic for second half of address space comes in second
link
If either link fails, the fact that the aggregate is
announced ensures there is a backup path

38

The Basic Principles

p The

keys to successful multihoming

configuration:
Keeping traffic engineering prefix
announcements independent of customer iBGP
n Understanding how to announce aggregates
n Understanding the purpose of announcing
subprefixes of aggregates
n Understanding how to manipulate BGP
attributes
n Too many upstreams/external paths makes
multihoming harder (2 or 3 is enough!)
n

39

IP Addressing &
Multihoming
How Good IP Address Plans
assist with Multihoming

40

IP Addressing & Multihoming

p
p

IP Address planning is an important part of

Multihoming
Previously have discussed separating:
n
n
n
n

Customer address space

Customer p-t-p link address space
Infrastructure p-t-p link address space
Loopback address space
101.10.0.0/21

101.10.0.1

101.10.5.255

Customer Address & p-t-p links

101.10.6.255 /24

Infrastructure Loopbacks
41

IP Addressing & Multihoming

p

ISP Router loopbacks and backbone point to point

links make up a small part of total address space
n

Links from ISP Aggregation edge to customer

router needs one /30
n
n

And they dont attract traffic, unlike customer address

space

Small requirements compared with total address space

Some ISPs use IP unnumbered

Planning customer assignments is a very

important part of multihoming
n

Traffic engineering involves subdividing aggregate into

pieces until load balancing works
42

Unplanned IP addressing
p

ISP fills up customer IP addressing from one end

of the range:
101.10.0.0/21

12345
Customer Addresses
p

ISP

Customers generate traffic

n

n
n

Dividing the range into two pieces will result in one /22
with all the customers, and one /22 with just the ISP
infrastructure the addresses
No loadbalancing as all traffic will come in the first /22
Means further subdivision of the first /22 = harder work
43

Planned IP addressing
p

If ISP fills up customer addressing from both

ends of the range:
101.10.0.0/21

13579

2 4 6 810

Customer Addresses

Customer Addresses

Scheme then is:

n

ISP

First customer from first /22, second customer from

second /22, third from first /22, etc

This works also for residential versus commercial

customers:
n
n

Residential from first /22

Commercial from second /22

44

Planned IP Addressing
p This

works fine for multihoming between

two upstream links (same or different
providers)
p Can also subdivide address space to suit
more than two upstreams
n

Follow a similar scheme for populating each

portion of the address space

p Dont

forget to always announce an

aggregate out of each link
45

Basic Multihoming
Lets try some simple worked
examples

46

Basic Multihoming
p No

frills multihoming
p Will look at two cases:
Multihoming with the same ISP
n Multihoming to different ISPs
n

p Will

keep the examples easy

Understanding easy concepts will make the

more complex scenarios easier to comprehend
n All assume that the site multihoming has a /19
address block
n

47

Basic Multihoming
p This

type is most commonplace at the

edge of the Internet
Networks here are usually concerned with
inbound traffic flows
n Outbound traffic flows being nearest exit is
usually sufficient
n

p Can

apply to the leaf ISP as well as

Enterprise networks

48

Two links to the same ISP

One link primary, the other link
backup only

49

Two links to the same ISP

(one as backup only)
p Applies

when end-site has bought a large

primary WAN link to their upstream and a
small secondary WAN link as the backup
n

For example, primary path might be an E1,

backup might be 64kbps

50

Two links to the same ISP

(one as backup only)
primary
C

AS 100
E

AS 65534
D

backup

p AS100

removes private AS and any

customer subprefixes from Internet
announcement
51

Two links to the same ISP

(one as backup only)
p Announce
n

/19 aggregate on each link

primary link:
Outbound announce /19 unaltered
p Inbound receive default route
p

backup link:
Outbound announce /19 with increased metric
p Inbound received default, and reduce local
preference
p

p When

one link fails, the announcement of

the /19 aggregate via the other link
ensures continued connectivity
52

Two links to the same ISP

(one as backup only)
p

Router A Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 description RouterC
neighbor 122.102.10.2 prefix-list aggregate out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
53

Two links to the same ISP

(one as backup only)
p

Router B Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.6 remote-as 100
neighbor 122.102.10.6 description RouterD
neighbor 122.102.10.6 prefix-list aggregate out
neighbor 122.102.10.6 route-map med10-out out
neighbor 122.102.10.6 prefix-list default in
neighbor 122.102.10.6 route-map lp-low-in in
!

..next slide
54

Two links to the same ISP

(one as backup only)
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
!
route-map med10-out permit 10
set metric 10
!
route-map lp-low-in permit 10
set local-preference 90
!

55

Two links to the same ISP

(one as backup only)
p

Router C Configuration (main link)

router bgp 100
neighbor 122.102.10.1 remote-as 65534
neighbor 122.102.10.1 default-originate
neighbor 122.102.10.1 prefix-list Customer in
neighbor 122.102.10.1 prefix-list default out
!
ip prefix-list Customer permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0

56

Two links to the same ISP

(one as backup only)
p

Router D Configuration (backup link)

router bgp 100
neighbor 122.102.10.5 remote-as 65534
neighbor 122.102.10.5 default-originate
neighbor 122.102.10.5 prefix-list Customer in
neighbor 122.102.10.5 prefix-list default out
!
ip prefix-list Customer permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0

57

Two links to the same ISP

(one as backup only)
p

Router E Configuration
router bgp 100
neighbor 122.102.10.17
neighbor 122.102.10.17
neighbor 122.102.10.17
!
ip prefix-list Customer

p
p

remote-as 110
remove-private-AS
prefix-list Customer out
permit 121.10.0.0/19

Router E removes the private AS and customers

subprefixes from external announcements
Private AS still visible inside AS100
58

Two links to the same ISP

With Loadsharing

59

Loadsharing to the same ISP

p More

common case
p End sites tend not to buy circuits and
leave them idle, only used for backup as
in previous example
p This example assumes equal capacity
circuits
n

Unequal capacity circuits requires more

refinement see later

60

Loadsharing to the same ISP

Link one
C

AS 100
E

AS 65534
D

Link two

Border router E in AS100 removes private AS and any

customer subprefixes from Internet announcement

61

Loadsharing to the same ISP

(with redundancy)
p
p

Announce /19 aggregate on each link

Split /19 and announce as two /20s, one on each
link
n
n

p
p

Basic inbound loadsharing

Assumes equal circuit capacity and even spread of traffic
across address block

Vary the split until perfect loadsharing achieved

Accept the default from upstream
n
n

Basic outbound loadsharing by nearest exit

Okay in first approximation as most ISP and end-site
traffic is inbound
62

Loadsharing to the same ISP

(with redundancy)
p

Router A Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 prefix-list as100-a out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as100-a permit 121.10.0.0/20
ip prefix-list as100-a permit 121.10.0.0/19
!
ip route 121.10.0.0 255.255.240.0 null0
ip route 121.10.0.0 255.255.224.0 null0

63

Loadsharing to the same ISP

(with redundancy)
p

Router B Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 122.102.10.6 remote-as 100
neighbor 122.102.10.6 prefix-list as100-b out
neighbor 122.102.10.6 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as100-b permit 121.10.16.0/20
ip prefix-list as100-b permit 121.10.0.0/19
!
ip route 121.10.16.0 255.255.240.0 null0
ip route 121.10.0.0 255.255.224.0 null0

64

Loadsharing to the same ISP

(with redundancy)
p

Router C Configuration
router bgp 100
neighbor 122.102.10.1 remote-as 65534
neighbor 122.102.10.1 default-originate
neighbor 122.102.10.1 prefix-list Customer in
neighbor 122.102.10.1 prefix-list default out
!
ip prefix-list Customer permit 121.10.0.0/19 le 20
ip prefix-list default permit 0.0.0.0/0

p
p

Router C only allows in /19 and /20 prefixes from

customer block
Router D configuration is identical
65

Loadsharing to the same ISP

(with redundancy)
p

Router E Configuration
router bgp 100
neighbor 122.102.10.17
neighbor 122.102.10.17
neighbor 122.102.10.17
!
ip prefix-list Customer

remote-as 110
remove-private-AS
prefix-list Customer out
permit 121.10.0.0/19

Private AS still visible inside AS100

66

Loadsharing to the same ISP

(with redundancy)
p Default
n

route for outbound traffic?

Originate the default route in the IGP on the

Border routers
Rely on IGP metrics for nearest exit
p IGP originates default route as long as BGP puts
default route in RIB
p

e.g. on router A using OSPF:

router ospf 65534
default-information originate

e.g. on router A using ISIS:

router isis as65534
default-information originate

67

Loadsharing to the same ISP

(with redundancy)
p Loadsharing

configuration is only on
customer router
p Upstream ISP has to
Remove customer subprefixes from external
announcements
n Remove private AS from external
announcements
n

p Could
n

also use BGP communities

See BGP Community presentation

68

Two links to the same ISP

Multiple Dualhomed Customers
(RFC2270)

69

Multiple Dualhomed Customers

(RFC2270)
p Unusual

for an ISP just to have one

dualhomed customer
Valid/valuable service offering for an ISP with
multiple PoPs
n Better for ISP than having customer multihome
with another provider!
n

p Look

at scaling the configuration

Simplifying the configuration

n Using templates, peer-groups, etc
n Every customer has the same configuration
(basically)
n

70

Multiple Dualhomed Customers

(RFC2270)
C

AS 100
E

A1

AS 65534

B1
D

A2

AS 65534

B2
p

Border router E in AS100

removes private AS and any
customer subprefixes from
Internet announcement

A3

AS 65534

B3
71

Multiple Dualhomed Customers

(RFC2270)
p Customer

announcements as per previous

example
p Use the same private AS for each
customer
Documented in RFC2270
n Address space is not overlapping
n Each customer hears default only
n

p Router

An and Bn configuration same as

Router A and B previously
72

Multiple Dualhomed Customers

(RFC2270)
p

Router A1 Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 prefix-list as100-a out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as100-a permit 121.10.0.0/20
ip prefix-list as100-a permit 121.10.0.0/19
!
ip route 121.10.0.0 255.255.240.0 null0
ip route 121.10.0.0 255.255.224.0 null0

73

Multiple Dualhomed Customers

(RFC2270)
p

Router B1 Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 122.102.10.6 remote-as 100
neighbor 122.102.10.6 prefix-list as100-b out
neighbor 122.102.10.6 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as100-b permit 121.10.16.0/20
ip prefix-list as100-b permit 121.10.0.0/19
!
ip route 121.10.0.0 255.255.224.0 null0
ip route 121.10.16.0 255.255.240.0 null0

74

Multiple Dualhomed Customers

(RFC2270)
p

Router C Configuration
router bgp 100
neighbor bgp-customers peer-group
neighbor bgp-customers remote-as 65534
neighbor bgp-customers default-originate
neighbor bgp-customers prefix-list default out
neighbor 122.102.10.1 peer-group bgp-customers
neighbor 122.102.10.1 description Customer One
neighbor 122.102.10.1 prefix-list Customer1 in
neighbor 122.102.10.9 peer-group bgp-customers
neighbor 122.102.10.9 description Customer Two
neighbor 122.102.10.9 prefix-list Customer2 in
75

Multiple Dualhomed Customers

(RFC2270)
neighbor 122.102.10.17 peer-group bgp-customers
neighbor 122.102.10.17 description Customer Three
neighbor 122.102.10.17 prefix-list Customer3 in
!
ip
ip
ip
ip
p

prefix-list
prefix-list
prefix-list
prefix-list

Customer1 permit 121.10.0.0/19 le 20

Customer2 permit 121.16.64.0/19 le 20
Customer3 permit 121.14.192.0/19 le 20
default permit 0.0.0.0/0

Router C only allows in /19 and /20 prefixes from

customer block
76

Multiple Dualhomed Customers

(RFC2270)
p

Router D Configuration
router bgp 100
neighbor bgp-customers peer-group
neighbor bgp-customers remote-as 65534
neighbor bgp-customers default-originate
neighbor bgp-customers prefix-list default out
neighbor 122.102.10.5 peer-group bgp-customers
neighbor 122.102.10.5 description Customer One
neighbor 122.102.10.5 prefix-list Customer1 in
neighbor 122.102.10.13 peer-group bgp-customers
neighbor 122.102.10.13 description Customer Two
neighbor 122.102.10.13 prefix-list Customer2 in
77

Multiple Dualhomed Customers

(RFC2270)
neighbor 122.102.10.21 peer-group bgp-customers
neighbor 122.102.10.21 description Customer Three
neighbor 122.102.10.21 prefix-list Customer3 in
!
ip
ip
ip
ip
p

prefix-list
prefix-list
prefix-list
prefix-list

Customer1 permit 121.10.0.0/19 le 20

Customer2 permit 121.16.64.0/19 le 20
Customer3 permit 121.14.192.0/19 le 20
default permit 0.0.0.0/0

Router D only allows in /19 and /20 prefixes from

customer block
78

Multiple Dualhomed Customers

(RFC2270)
p

Router E Configuration
Assumes customer address space is not part of
upstreams address block
router bgp 100
neighbor 122.102.10.17 remote-as 110
neighbor 122.102.10.17 remove-private-AS
neighbor 122.102.10.17 prefix-list Customers out
!
ip prefix-list Customers permit 121.10.0.0/19
ip prefix-list Customers permit 121.16.64.0/19
ip prefix-list Customers permit 121.14.192.0/19
n

Private AS still visible inside AS100

79

Multiple Dualhomed Customers

(RFC2270)
p

If customers prefixes come from ISPs address

block
n
n

Do NOT announce them to the Internet

Announce ISP aggregate only

Router E configuration:
router bgp 100
neighbor 122.102.10.17 remote-as 110
neighbor 122.102.10.17 prefix-list aggregate out
!
ip prefix-list aggregate permit 121.8.0.0/13

80

Multihoming Summary
p Use

private AS for multihoming to the

same upstream
p Leak subprefixes to upstream only to aid
loadsharing
p Upstream router E configuration is
identical across all situations

81

Basic Multihoming
Multihoming to Different ISPs

82

Two links to different ISPs

p

Use a Public AS
n
n

Address space comes from

n
n
n

Or use private AS if agreed with the other ISP

But some people dont like the inconsistent-AS which
results from use of a private-AS
Both upstreams
or
Regional Internet Registry
NB. Very hard to multihome with address space from
both upstreams due to typical operational policy in force
to day

Configuration concepts very similar to those used

for two links to the same AS
83

Inconsistent-AS?
p

Viewing the prefixes

originated by AS65534 in
the Internet shows they
appear to be originated by
both AS210 and AS200
n
n

AS 65534

This is NOT bad

Nor is it illegal

AS 200
AS 210

Cisco IOS command is

show ip bgp inconsistent-as

Internet
84

Two links to different

ISPs
One link primary, the other link
backup only

85

Two links to different ISPs

(one as backup only)
Internet

AS 110

AS 120
C

Announce /19 block

A

Announce /19 block

with longer AS PATH

AS 100
86

Two links to different ISPs

(one as backup only)
p Announce

/19 aggregate on each link

Primary link makes standard announcement

n Backup link lengthens the AS PATH by using
AS PATH prepend
n

p When

one link fails, the announcement of

the /19 aggregate via the other link
ensures continued connectivity

87

Two links to different ISPs

(one as backup only)
p

Router A Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 100
neighbor 122.102.10.1 prefix-list aggregate out
neighbor 122.102.10.1 prefix-list default in
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
88

Two links to different ISPs

(one as backup only)
p Router B Configuration
router bgp 100
network 121.10.0.0 mask 255.255.224.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list aggregate out
neighbor 120.1.5.1 route-map as120-prepend out
neighbor 120.1.5.1 prefix-list default in
neighbor 120.1.5.1 route-map lp-low in
!
...next slide...

89

Two links to different ISPs

(one as backup only)
ip route 121.10.0.0 255.255.224.0 null0
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
route-map as120-prepend permit 10
set as-path prepend 100 100 100
!
route-map lp-low permit 10
set local-preference 80
!

90

Two links to different ISPs

(one as backup only)
p Not

a common situation as most sites tend

to prefer using whatever capacity they
have
n

(Useful when two competing ISPs agree to

provide mutual backup to each other)

p But

it shows the basic concepts of using

local-prefs and AS-path prepends for
engineering traffic in the chosen direction

91

Two links to different

ISPs
With Loadsharing

92

Two links to different ISPs

(with loadsharing)
Internet

AS 110

AS 120
C

Announce first
/20 and /19 block

Announce second
/20 and /19 block

AS 100
93

Two links to different ISPs

(with loadsharing)
p Announce

/19 aggregate on each link

p Split /19 and announce as two /20s, one
on each link
n

Basic inbound loadsharing

p When

one link fails, the announcement of

the /19 aggregate via the other ISP
ensures continued connectivity

94

Two links to different ISPs

(with loadsharing)
p

Router A Configuration
router bgp 100
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.1 remote-as 110
neighbor 122.102.10.1 prefix-list as110-out out
neighbor 122.102.10.1 prefix-list default in
!
ip route 121.10.0.0 255.255.224.0 null0
ip route 121.10.0.0 255.255.240.0 null0
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as110-out permit 121.10.0.0/20
ip prefix-list as110-out permit 121.10.0.0/19

95

Two links to different ISPs

(with loadsharing)
p

Router B Configuration
router bgp 100
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list as120-out out
neighbor 120.1.5.1 prefix-list default in
!
ip route 121.10.0.0 255.255.224.0 null0
ip route 121.10.16.0 255.255.240.0 null0
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list as120-out permit 121.10.0.0/19
ip prefix-list as120-out permit 121.10.16.0/20

96

Two links to different ISPs

(with loadsharing)
p Loadsharing

in this case is very basic

p But shows the first steps in designing a
load sharing solution
Start with a simple concept
n And build on it!
n

97

Two links to different

ISPs
More Controlled Loadsharing

98

Loadsharing with different ISPs

Internet

AS 110

AS 120
C

Announce /19 block

A

Announce /20 subprefix,

and /19 block with
longer AS path

AS 100
99

Loadsharing with different ISPs

p Announce

/19 aggregate on each link

On first link, announce /19 as normal

n On second link, announce /19 with longer AS
PATH, and announce one /20 subprefix
n

Controls loadsharing between upstreams and the

Internet

p Vary

the subprefix size and AS PATH

length until perfect loadsharing achieved
p Still require redundancy!
100

Loadsharing with different ISPs

p Router A Configuration
router bgp 100
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 110
neighbor 122.102.10.1 prefix-list default in
neighbor 122.102.10.1 prefix-list as110-out out
!
ip route 121.10.0.0 255.255.224.0 null0
!
ip prefix-list as110-out permit 121.10.0.0/19
!
ip prefix-list default permit 0.0.0.0/0
101

Loadsharing with different ISPs

p Router B Configuration
router bgp 100
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list default in
neighbor 120.1.5.1 prefix-list as120-out out
neighbor 120.1.5.1 route-map agg-prepend out
!
ip route 121.10.0.0 255.255.224.0 null0
ip route 121.10.16.0 255.255.240.0 null0
!
...next slide...

102

Loadsharing with different ISPs

route-map agg-prepend permit 10
match ip address prefix-list aggregate
set as-path prepend 100 100
!
route-map agg-prepend permit 20
!

ip prefix-list default permit 0.0.0.0/0

!
ip prefix-list as120-out permit 121.10.0.0/19
ip prefix-list as120-out permit 121.10.16.0/20
!
ip prefix-list aggregate permit 121.10.0.0/19
!
103

Loadsharing with different ISPs

p This

example is more commonplace

p Shows how ISPs and end-sites subdivide
address space frugally, as well as use the
AS-PATH prepend concept to optimise the
load sharing between different ISPs
p Notice that the /19 aggregate block is
ALWAYS announced

104

Summary

105

Summary
p Previous

examples dealt with simple case

p Load balancing inbound traffic flow
Achieved by modifying outbound routing
announcements
n Aggregate is always announced
n

p We

have not looked at outbound traffic

flow
n

For now this is left as nearest exit

106

Simple Multihoming
ISP Workshops

107