Microservices Network Architecture 101

v
Microservices Network Architecture 101
Dinesh G Dutt
May 26, 2016

Agenda
Microservices: Evolution of Application
Architecture
Container Networking 101
Container Networking 101-Alt
May 29, 2016 Cumulus Networks Confidential 2

Key Takeaways
Microservices are the latest step in evolution
of application architecture
 Make app development faster, apps more scalable,
adaptable
Container networking has two primary
modes: single host and multi-host
Using existing well-known techniques, multi-
host container networking can be made
quite easy

•Evolution of Microservices

Monolithic App Running On Mainframe
May 29, 2016 cumulusnetworks.com 5

Client-Server Architecture: Monolithic Server
LAN

Web Front-end
Application
Database
Internet
Client-Server Architecture: Server Split Into Three, Handle
More Clients

Web Front-end
Application
Database
Web Front-end
Application
Database
Web Front-end
Application
Database
Internet
Server Virtualization, Multiple Instances of Servers

Internet
Microservices Architecture: Devolving The Application Even
More

Microservices Are Popular Now Because …
They address a set of issues that plague the
fast-paced, highly scalable, ever evolving
world of modern software
 Reliability
 Scalability
 Faster development and release cycles
 Reduces barrier to entry for new frameworks and
technologies

•Distributed system design, and
therefore communications, is at the
heart of Microservices

For More About Microservices
https://www.nginx.com/blog/introduction-to-
microservices/
 Excellent 7 part article about Microservices
 Doesn’t cover networking

Linux Containers (LXC)
If Microservices are the genotype,
containers are the phenotype
Containers a.k.a. LXC is a system isolation
method provided in Linux that’s lighter than
VM, but heavier than solutions such as
chroot or systemd-nspawn
 Doesn’t assume entire OS is isolated
 Underlying OS is common across all containers

Container Landscape
Linux kernel
(cgroups, namespaces)
Docker Rocket (CoreOS)OpenVZ
App App App

•Container Communication

Topology is
simulated in Vagrant
 Switches are running
Cumulus VX 3.0.0 box
 Hosts are running
williamyeh/ubuntu-
trusty64-docker box
 BGP between leaf and
spine (L3 fabric)
spine-1 spine-2
leaf-1
host-11
host-12
host-21
host-22

Docker Networking Options
Single Host
 Bridged (default)
 Host mode
 None
 MacVlan
 IPVlan
Multi Host
 Overlay
• Flannel
 MacVlan
 IPVlan
 Bridged

Bridged Mode With Docker Container: Default
10.1.20.1
10.1.20.11
eth0
docker0
172.17.0.4
172.17.0.1
172.17.0.3 172.17.0.2
 All containers connected to the
bridge can talk to each other
and to the host
 Host bridge interface (docker0)
is the gateway to talk to host
 Entities outside the host
cannot communicate with the
container
 IPAM is default assignment by
docker from subnet specified
 Can start container with a specific
IP address in bridge subnet

Default Allowed Communication In Bridged Mode
10.1.20.1
10.1.20.11
eth0
docker0
172.17.0.4
172.17.0.1
172.17.0.3 172.17.0.2
docker1
172.16.1.1
172.16.1.2
X
• Router cannot ping 172.16.1.x or 172.17.0.x, but 172.16.1.x or 172.17.0.x can ping 10.1.2.1
• Router is not aware of the reachability to 172.x network, by default
• Can start docker such that containers are also isolated from each other

Single Host Networking: NAT always for External
Communication

Multi-host Networking
Introduced in docker version
Uses VxLAN overlay to connect L2 networks
Complex setup:
 Build a machine with a key-value store daemon
 All machines to be in multi-host mode need access
to this KV store
 Need docker swarm

•A Networking Dude looks at Multi-
host Networking

What NAT ?
NAT exists to allow docker containers to run
in cloud VMs and not expose themselves via
private IP addresses
This is not typically the use case for folks
deploying containers in their own DC
So, what can we do for such folks ?

Simplified Multi-host Routing
Turn off NAT
Assign non-default IP addresses to docker
bridges
Run routing protocol on the host to
announce these addresses
 Or use a giant L2 network with each host doing
IPAM over an allocated range

Simplified Multi-host Routing
In brief:
 Using the straight up linux bridging driver for
containers coupled with routing on the host gives
you multi-host container communication
 Using straight up Linux iptables provides the
microsegmentation between containers

Why More Choices ?
Problems with existing models:
 Bridging the default way involves NAT for external
communication
• NAT is not scalable, performance issues
 Ability to assign IP addresses to containers without
regard to bridge subnet

MacVlan Model
Constructs multiple MAC
addresses based on
original MAC address of
parent interface (eth0 in
picture)
Containers attach to the
parent interface directly, no
bridge
Containers have an IP
address allocated from the
bridge subnet where bridge
is upstream (typically a
bridge/VLAN in the ToR)
10.1.20.1
10.1.20.11
eth0
10.1.20.3 10.1.20.210.1.20.4
• docker network create –d macvlan
–-subnet=10.1.20.0/24 –-gateway=10.1.20.1
-o parent=eth0 mcv1
• docker run –net=mcv1 –it ubuntu bash

IPVlan: L3 Mode
 Ipvlan interface is created
attached to an existing interface
(eth0, for eg)
 Container interfaces attach to
this ipvlan interface
 Container IPs MUST be allocated
manually (no IPAM)
 Containers connected to same
parent can communicate with
each other, even if they’re in
different subnets
 All broadcast/multicast dropped
10.1.20.1
10.1.20.11
eth0
20.1.1.12/24 12.2.1.2/3010.0.1.11/24
• docker network create –d ipvlan
–-subnet=10.1.20.0/24 --subnet=20.1.1.0/24
-o ipvlan_mode=l3 –o parent=eth0 ipv1
• docker run –net=ipv1 –ip=10.1.20.11 –it ubuntu bash

•Container Frameworks

Why Do We Need Container Frameworks ?
Multi-host containers require more than
communication simplification:
 Cluster management
 Job scheduling
 Policy (simple or complex ones)
 Service discovery
 Event-driven models
 Lifecycle management

Swarm/Kubernetes/Mesos/Weave
Microservices Landscape
Linux kernel
App App App

Swarm/Kubernetes/Mesos/Weave
Microservices Landscape
Linux kernel
App App App
Calic
o

What Did I Not Cover
Network plugins
Host mode
Flannel use
 Came out of CoreOS, similar to overlay mode in
docker
 Apparently not used much with docker
 One use case is with Kubernetes, but its not really
required there, as per our last webinar

Summary
Microservices are the next evolution in
building scalable, resilient, agile
applications
Container networking has many new and
moving parts, but at the heart linux bridging
and iptables are good enough
Running a routing protocol on the host
simplifies multi-host container networking

•Operationalizing Linux VRF in the
Data Center
•Guest: David Ahern, Primary VRF
Author in the Linux kernel
•When: June 23
Next Month’s Webinar

© 2016 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in
the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive
licensee of Linus Torvalds, owner of the mark on a world-wide basis.
ThankYou!
Cumulus Networks Confidential 37May 29, 2016

Microservices Network Architecture 101

More Related Content

What's hot

Viewers also liked

Similar to Microservices Network Architecture 101

More from Cumulus Networks

Recently uploaded

Microservices Network Architecture 101