Amazon Virtual Private Cloud
Deep Dive
Randall Hunt – Developer Evangelist, AWS
©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
Related Presentations – Videos online
https://www.youtube.com/user/AmazonWebServices
• ARC205 – VPC Fundamentals and Connectivity
• ARC401 – Black Belt Networking for Cloud Ninja
– Application centric, network monitoring, management, floating IPs
• ARC403 – From One to Many: Evolving VPC Design
• SDD302 – A Tale of One Thousand Instances
– Example of EC2-Classic customer adopting VPC
• SDD419 – Amazon EC2 Networking Deep Dive
– Network performance, placement groups, enhanced networking
• SDD422 – Amazon VPC Deep Dive (this talk)
Topics today
Virtual networking options
EC2-Classic Default VPC VPC
Simple to get started – The best of both Advanced virtual
all instances have networking services:
Internet connectivity, Get started using the ENIs and multiple IPs
auto-assigned private EC2-Classic routing tables
and public IP addresses experience egress security groups
network ACLs
Inbound security groups If and when needed, private connectivity
begin using any VPC
feature you require Enhanced networking
And more to come...
Virtual networking options
EC2-Classic Default VPC VPC
Simple to get started – The best of both Advanced virtual
all instances have networking services:
Internet connectivity, Get started using the ENIs and multiple IPs
auto-assigned private EC2-Classic routing tables
and public IP addresses experience egress security groups
network ACLs
Inbound security groups
All accounts created after If and when needed, private connectivity
12/4/2013 support VPC begin using any VPC
feature you require Enhanced networking
only and have a default
VPC in each region And more to come...
Confirming your default VPC
describe-account-attributes
VPC only
1. Routing & private connections
Implementing a hybrid architecture
Corporate Data Center
Create VPC
Corporate Data Center
aws ec2 create-vpc --cidr 10.10.0.0/16
aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.1.0/24 --a us-west-2a
aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.2.0/24 --a us-west-2b
Create VPN connection
Corporate Data Center
aws ec2 create-vpn-gateway --type ipsec.1
aws ec2 attach-vpn-gateway --vpn vgw-f9da06e7 --vpc vpc-c15180a4
aws ec2 create-customer-gateway --type ipsec.1 --public 54.64.1.2 --bgp 6500
aws ec2 create-vpn-connection --vpn vgw-f9da06e7 --cust cgw-f4d905ea --t ipsec.1
Launch instances
Corporate Data Center
aws ec2 run-instances --image ami-d636bde6 --sub subnet-d83d91bd --count 3
aws ec2 run-instances --image ami-d636bde6 --sub subnet-b734f6c0 --count 3
Using AWS Direct Connect
Corporate Data Center
aws directconnect create-connection --loc EqSE2 --b 1Gbps --conn My_First
aws directconnect create-private-virtual-interface --conn dxcon-fgp13h2s --new
virtualInterfaceName=Foo, vlan=10, asn=60, authKey=testing,
amazonAddress=192.168.0.1/24, customerAddress=192.168.0.2/24,
virtualGatewayId=vgw-f9da06e7
Configuring route table
Corporate Data Center
192.168.0.0/16
Each VPC has a single
routing table at creation time,
used by all subnets
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id vgw-f9da06e7
Remote connectivity best practices
Availability Zone Availability Zone
Each VPN connection
consists of 2 IPSec
tunnels. Use BGP for
failure recovery.
Corporate Data Center
Remote connectivity best practices
Availability Zone Availability Zone
A pair of VPN
connections (4 IPSec
tunnels total) protects
against failure of your
customer gateway
Corporate Data Center
Remote connectivity best practices
Availability Zone Availability Zone
Redundant AWS Direct
Connect connections
with VPN backup
Corporate Data Center
VPC with private and public connectivity
Corporate Data Center
192.168.0.0/16
aws ec2 create-internet-gateway
aws ec2 attach-internet-gateway --internet igw-5a1ae13f --vpc vpc-c15180a4
aws ec2 delete-route --ro rtb-ef36e58a --dest 0.0.0.0/0
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
aws ec2 create-route --ro rtb-ef36e58a --dest 192.168.0.0/16 --gateway-id vgw-f9da06e7
Automatic route propagation from VGW
Corporate Data Center
192.168.0.0/16
Used to automatically update routing
table(s) with routes present in the VGW
aws ec2 delete-route --ro rtb-ef36e58a --dest 192.168.0.0/16
aws ec2 enable-vgw-route-propagation --ro rtb-ef36e58a --gateway-id vgw-f9da06e7
Isolating connectivity by subnet
Corporate
192.168.0.0/16
Subnet with connectivity only
to other instances and the
Internet via the IGW
aws ec2 create-subnet --vpc vpc-c15180a4 --cidr 10.10.3.0/24 --a us-west-2b
aws ec2 create-route-table --vpc vpc-c15180a4
aws ec2 associate-route-table --ro rtb-fc61b299 --subnet subnet-60975a17
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
Software VPN for VPC-to-VPC connectivity
# VPC A
aws ec2 modify-network-interface-attribute --net eni-f832afcc --no-source-dest-check
aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc
# VPC B
aws ec2 modify-network-interface-attribute --net eni-9c1b693a --no-source-dest-check
aws ec2 create-route --ro rtb-67a2b31c --dest 10.10.0.0/16 –-instance-id i-9c1b693a
Software VPN for VPC-to-VPC connectivity
Software VPN
between these
instances
Software VPN for VPC-to-VPC connectivity
Enabling communication
between instances in these
subnets; adding routes to the
default routing table
Software firewall to the Internet
Routing all traffic from subnets
to the Internet via a firewall is
conceptually similar
# Default routing table directs traffic to the NAT/firewall instance
aws ec2 create-route --ro rtb-ef36e58a --dest 0.0.0.0/0 --instance-id i-f832afcc
# Routing table for 10.10.3.0/24 directs to the Internet
aws ec2 create-route --ro rtb-67a2b31c --dest 0.0.0.0/0 --gateway-id igw-5a1ae13f
2. VPC peering
Shared services VPC using VPC peering
• Common/core services
– Authentication/directory
– Monitoring
– Logging
– Remote administration
– Scanning
Provides infrastructure zoning
• Dev: VPC B
• Test: VPC C
• Production: VPC D
VPC peering for VPC-to-VPC connectivity
VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16
vpc-c15180a4 vpc-062dfc63
aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63
aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87
VPC A> aws ec2 create-route --ro rtb-ef36e58a --des 10.20.0.0/16 --vpc-peer pcx-ee56be87
VPC B> aws ec2 create-route --ro rtb-67a2b31c --des 10.10.0.0/16 --vpc-peer pcx-ee56be87
VPC peering across accounts
VPC B - 10.20.0.0/16
VPC A - 10.10.0.0/16 vpc-062dfc63
vpc-c15180a4 Account ID 472752909333
aws ec2 create-vpc-peering-connection --vpc-id vpc-c15180a4 --peer-vpc vpc-062dfc63
--peer-owner 472752909333
# In owner account 472752909333
aws ec2 accept-vpc-peering-connection --vpc-peer pcx-ee56be87
VPC peering – Additional considerations
• Security groups not supported across peerings
– Workaround: specify rules by IP prefix
• No “transit” capability for VPN, AWS Direct
Connect, or 3rd VPCs
– Example: Cannot access VPC C from VPC A via VPC B
– Workaround: Create a direct peering from VPC A to VPC C
• Peer VPC address ranges cannot overlap
– But, you can peer with 2+ VPCs that themselves overlap
– Use subnets/routing tables to pick the VPC to use
VPC peering with software firewall
VPC A - 10.10.0.0/16 VPC B - 10.20.0.0/16
# Default routing table directs Peer traffic to the NAT/firewall instance
aws ec2 create-route --ro rtb-ef36e58a --dest 10.20.0.0/16 --instance-id i-f832afcc
# Routing table for 10.10.3.0/24 directs to the Peering
aws ec2 create-route --ro rtb-67a2b31c --dest 10.20.0.0/16 --vpc-peer pcx-ee56be87
3. Enhanced networking
Latency: Packets per second
Instance 1 Instance 2
...........
Packet processing in Amazon EC2:
VIF
Instance Virtual NICs
eth1
eth0
Physical NIC
Virtualization layer
Packet processing in Amazon EC2:
SR-IOV
Instance
eth1
eth0
Physical NIC
VF Driver
VF
Virtualization layer
Inter-instance latency
SR-IOV: Is this thing on?
It may already be!
For many newer AMIs, enhanced
networking is already on:
• Newest Amazon Linux AMIs
• Windows Server 2012 R2 AMI
No need to configure
SRIOV: Is this thing on? (Linux)
No Yes!
[ec2-user@ip-10-0-3-70 [ec2-user@ip-10-0-3-70 ~]$
~]$ ethtool -i eth0 ethtool -i eth0
driver: vif driver: ixgbevf
version: version: 2.14.2+amzn
firmware-version: firmware-version: N/A
bus-info: vif-0 bus-info: 0000:00:03.0
… …
SRIOV: Is this thing on? (Windows)
No Yes!
AMI/instance support for SR-IOV
• C3, C4, I2, D2, R3 instance families: 23 types
• HVM virtualization type
• Required kernel version
– Linux: 2.6.32+
– Windows: Server 2008 R2+
• Appropriate VF driver
– Linux: ixgbevf 2.14.2+ module
– Windows: Intel® 82599 Virtual Function driver
Walkthrough: Enabling enhanced networking
(Amazon Linux)
amzn-ami-hvm-2012.03.1.x86_64-ebs
hvm
Walkthrough: Enabling enhanced networking
(Amazon Linux)
--attribute
sriovNetSupport
Not yet!
InstanceId i-37c5d1d9
Walkthrough: Enabling enhanced networking
(Amazon Linux)
OS update
[ec2-user@ip-10-0-3-125 ~]$ sudo yum update
Walkthrough: Enabling enhanced networking
(Amazon Linux)
reboot-instances
Reboot
(OS update)
Walkthrough: Enabling enhanced networking
(Windows)
Walkthrough: Enabling enhanced networking
(Windows)
Add to Windows driver store
Walkthrough: Enabling enhanced networking
All EBS-backed instances
stop-instances
Stop the instance
Walkthrough: Enabling enhanced networking
All EBS-backed instances
stop-instances
--sriov-net-support
simple
Enable SRIOV
Cannot be undone
Walkthrough: Enabling enhanced networking
All EBS-backed instances
start-instances
Start
Walkthrough: Enabling enhanced networking
All EBS-backed instances
start-instances
--attribute
sriovNetSupport
InstanceId i-37c5d1d9
Value simple
We’re on
Subnet A Subnet A2 Subnet C
us-east-1a us-east-1a us-east-1c
10.0.1.0/24 10.0.2.0/24 10.0.3.0/24
Instance
1
10.0.2.50
10.0.1.100 Instance
3 10.0.3.99
Instance
4
Instance
2
10.0.2.51
10.0.1.101
Elastic
network
interface
Subnet A Subnet A2 Subnet C
us-east-1a us-east-1a us-east-1c
10.0.1.0/24 Placement group 10.0.2.0/24 10.0.3.0/24
Instance
1
10.0.2.50
10.0.1.100 Instance
3 10.0.3.99
Instance
4
Instance
2
10.0.2.51
10.0.1.101
elastic
network
interface
Placement Groups
• ~1.5-3x better inter-instance ping (YMMV)
• Cannot span AZs
• Cannot be applied to running instances
• Only available for certain instance types
• Not great for things that scale horizontally
(capacity limited)
4. VPC for EC2-Classic customers
Adopting VPC
• Customers tell us they want to adopt VPC
• Have significant EC2-Classic
infrastructure
• Where do I start?
Start simple
• One subnet per AZ
• Each instance has a public
IP address and Internet
connectivity
• Use security groups to
control access
Add features at your own pace
• Multiple interfaces per
instance
• Multiple IPs per interface
• Enhanced networking
• Private connectivity
• VPC peering
• …
VPC ClassicLink
• Incremental adoption of VPC
• Private IP communication
between EC2-Classic and VPC
instances
• Security groups between EC2-
Classic and VPC instances
• Designed for the largest
deployments
ClassicLink
Route53
ELB
RDS DB
Instance
ClassicLink
Route53
ELB
RDS DB
Instance
ClassicLink
Route53
ELB
RDS DB
Instance
ClassicLink
Route53
ELB
RDS DB
Instance
ClassicLink
Route53
ELB
RDS DB
Instance
ClassicLink
Route53
RDS DB
Instance
ClassicLink
Route53
RDS DB
Instance
ClassicLink
• Preparation: Create VPC and configure for ClassicLink
• Create VPC security groups and deploy VPC components
• Add EC2-Classic instances to your VPC security groups
• Deploy components in stages in VPC
• Clean up un-used EC2-Classic instances
Pros Cons
(Potentially) No disruptive maintenance Additional complexity during migration
Direct private IP connectivity and security group Still need to replace EC2-Classic
integration instances with new VPC instances
Designed for the largest deployments
ClassicLink – Component stages
• Start with AWS-managed
infrastructure
– RDS, ElastiCache, Redshift
RDS DB ElastiCache Elastic Load • Next ELB
EC2-Classic Instance Cache Node Balancer
• Then instances
ClassicLink
RDS DB ElastiCache Elastic Load
Instance Cache Node Balancer
ClassicLink – Additional considerations
• VPC address ranges for use with ClassicLink
– 10.0.0.0/15, or any other range outside 10.0.0.0/8
– Why? EC2-Classic instance private IP addresses are in 10.2.0.0 – 10.255.255.255
• VPC also can’t have extra route table entries to 10.0.0.0/8
• ClassicLink instances use EC2-Classic for all Internet traffic. No
access from VPN/Direct Connect or a VPC peer to a ClassicLink
instance.
• ClassicLink must be enabled after instance launch (Run) or Start
• VPC instance DNS names do not resolve from EC2-Classic, and vice-
versa
ClassicLink APIs & CLI
Enabling ClassicLink
vpc-4325f426
To use ClassicLink the VPC must
have this feature enabled. Can
be restricted with IAM policy.
Attaching a EC2-Classic instance to a VPC
i-2b3ecd1c
vpc-4325f426 sg-da107fbf
Link this specific instance to
the VPC using the specified
VPC security groups
Attaching a EC2-Classic instance to a VPC
i-2b3ecd1c
vpc-4325f426 sg-da107fbf
Link required after Run (new
instance launch) or Start
(stopped instance)
ClassicLink and other services
• Elastic Load Balancing
– EC2-Classic instances can be backends of VPC balancers
• Spot
– Running spot instances can be linked
• Auto Scaling
– Configure to link classic instances following launch
