IPSEC-INTERNET PROTOCOL SECURITY
Dr.© KIRUBAVATHI.G
McGraw-Hill The McGraw-Hill Companies, Inc., 2000
IP Level Security: IPSec
IP Security (IPSec) is a collection of protocols designed
by the IETF (Internet Engineering Task Force) to provide
security for a packet at the IP level.
IPSec does not define the use of any specific
encryption or authentication method.
IPSec provides a framework and a mechanism; it
leaves the selection of the encryption, authentication,
and hashing methods to the user.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec as a VPN Protocol
A VPN (Virtual Private Network)
is the overall mechanism for
creating a secure “tunnel” through
an untrusted network (like the
Internet).
IPSec (Internet Protocol
Security) is one of the main
protocol suites used to
implement that tunnel.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Applications of IPSec
Secure branch office
connectivity over the Internet.
Secure remote access over the
Internet.
Establsihing extranet and
intranet connectivity with
partners.
Enhancing electronic commerce
security.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
VPN
VPN –provides a secure connection over an
insecure medium such as Internet.
VPN creates a secure tunnel that allow remote
users to access organizational private network.
TYPES OF VPN
site-to-site VPN
client-to-site VPN (remote user)
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Site to Site VPN
A Site-to-Site VPN is also called as Router-to-Router
VPN and is commonly used in the large companies.
Companies or organizations, with branch offices in
different locations, use Site-to-site VPN to connect the
network of one office location to the network at
another office location.
Intranet based VPN: When several offices of the
same company are connected using Site-to-Site
VPN type, it is called as Intranet based VPN.
Extranet based VPN: When companies use Site-
to-site VPN type to connect to the office of another
company, it is called as Extranet based VPN.
In Site-to-site VPN one router acts as a VPN Client
and another router as a VPN Server as it is based on
Router-to-Router communication.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Remote user VPN
Remote user VPN permits a user to connect to a
private network and access all its services and
resources remotely.
The connection between the user and the private
network occurs through the Internet and the
connection is secure and private.
Remote Access VPN is useful for home users and
business users.
An employee of a company, while he/she is out of
station, uses a VPN to connect to his/her company’s
private network and remotely access files and
resources on the private network.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Layers of the osi model in which IPSec
operates
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPsec at a Glance
IPsec uses a combination of the following techniques
to provide its services
Diffie-Hellman key exchange to establish keys
between peers.
Encryption algorithms like DES to provide
confidentiality.
Hash algorithms like MD5 and SHA-1 to provide
message authentication.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPsec components
IPsec consists of two important protocol
components
The first, defines the information that needs to
be added to the IP packet to achieve the required
services. These are classified further as
Authentication Header (AH) and
Encapsulating Security Protocol (ESP).
The second, Internet Key Exchange, which
negotiates security association between two
peers and exchanges keying material.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec cont…
IPsec can be used on many different
devices, it’s used on routers, firewalls,
hosts and servers.
Between two routers to create a site-to-
site VPN that “bridges” two LANs
together.
Between a firewall and windows host for
remote access VPN.
Between two Linux servers to protect
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSEC TUNNEL
To establish an IPsec tunnel, use a
protocol called IKE (Internet Key
Exchange).
There are two phases to build an
IPsec tunnel:
IKE phase 1
IKE phase 2
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE phase 1
The collection of parameters that
the two devices will use is called a
SA (Security Association).
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE phase 1 cont…
The IKE phase 1 creates a tunnel by
negotiating the Security
Associations b/w two peers.
Use this tunnel as a secure method
to establish the second tunnel
called the IKE phase 2 tunnel or
IPsec tunnel for secure
transmission.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE phase 2
An IKE phase 2 tunnel (or IPsec
tunnel) can used to protect our
user data. The user data will be
sent through the IKE phase 2 tunnel
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE phase 1
Two peers will negotiate about the
encryption, authentication, hashing and
other protocols that they want to use
and some other parameters that are
required.
In this phase, an ISAKMP (Internet
Security Association and Key
Management Protocol) session is
established. This is also called the
ISAKMP tunnel or IKE© phase
McGraw-Hill 1 tunnel.
The McGraw-Hill Companies, Inc., 2004
Phase 1 cont…
The main purpose of IKE phase 1 is
to establish a secure tunnel that we
can use for IKE phase 2.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
HAGLE
Break down phase 1 in
three simple steps:
Step 1: Negotiation
The peer that has traffic that should be protected will initiate
the IKE phase 1 negotiation. The two peers will negotiate
about the following items:
Hashing: to verify the integrity, we use MD5 or SHA for this.
Authentication: each peer has to prove who he is. Two
commonly used options are a pre-shared key or digital
certificates.
DH (Diffie Hellman) group: the DH group determines the
strength of the key that is used in the key exchange process.
Lifetime: Each vendor uses a different lifetime, a common
default value is 86400 seconds (1 day).
Encryption: what algorithm do we use for encryption? For
© The McGraw-Hill Companies, Inc., 2004
example, DES, 3DES or AES.
McGraw-Hill
Cont…
Step 2: DH Key Exchange
Once the negotiation has succeeded, the two peers will know
what policy to use. They will now use the DH group that they
negotiated to exchange keying material. The end result will be
that both peers will have a shared key.
Step 3: Authentication
The last step is that the two peers will authenticate each other
using the authentication method that they agreed upon on in
the negotiation. When the authentication is successful, we
have completed IKE phase 1. The end result is a IKE phase 1
tunnel (aka ISAKMP tunnel) which is bidirectional. This means
that both peers can send and receive on this tunnel.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Three steps completed
using 2 diff modes
Main mode
Aggressive mode
Main mode uses six messages while
aggressive mode only uses three
messages. Main mode is considered
more secure.
https://www.cloudshark.org/captures/
ff740838f1c2
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
a nonce is an arbitrary number that can be used just once in a
cryptographic communication
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Phase 1: The goal of this phase is to create a secure, encrypted, and
authenticated channel.
• Main Mode: In this mode, a series of message exchanges take place to
establish this secure communication.
1.MM1 & MM2 (Messages 1 and 2):
1. Initiator (MM1) sends encryption and hashing algorithm proposals to
the Responder.
2. Responder (MM2) responds by selecting the appropriate proposal that
it can support.
3. Purpose: Both parties negotiate cryptographic settings (encryption,
hashing).
2.MM3 & MM4 (Messages 3 and 4):
1. Initiator (MM3) sends its public Diffie-Hellman values and a nonce (a
unique random value).
2. Responder (MM4) responds with its Diffie-Hellman values and its own
nonce.
3. Purpose: This stage involves the exchange of secret key materials
through the Diffie-Hellman mechanism to agree on a shared secret for
encryption.
3.MM5 & MM6 (Messages 5 and 6):
1. Initiator (MM5) sends its identity encrypted along with proof of its
identity (authentication).
2. Responder (MM6) does the same.
3. Purpose: Both parties prove their identity using previously shared
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
keys.
Main mode
IKEv1 main mode uses 6 messages.
Msg 1: The initiator (peer that
wants to build the tunnel) will send
the first message. This is a proposal
for the security association.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE USES UDP PORT 500
SPI-Security Parameter Index initiat
It’s a unique value that’s identifies S
DOI: IPSec(first proposal)
Attributes - SA
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 2
When the responder receives the
first message from the initiator, it
will reply.
This message is used to inform the
initiator that we agree upon the
attributes in the transform payload.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 3
Since peers agree on the security
association to use, the initiator will
start the Diffie Hellman key
exchange.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 4
The responder will also send his/her
Diffie Hellman nonces to the
initiator, now the two peers can
calculate the Diffie Hellman shared
key. (a nonce is an arbitrary
number that can be used just once
in a cryptographic communication)
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 5
The last two messages are encrypted
so we can’t see its contents
anymore. These two are used for
identification and authentication of each
peer. The initiator starts.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 6
we have the 6th message from the
responder with its identification and
authentication information. IKEv1
main mode has now completed and
we can continue with IKE phase 2.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Aggressive Mode
IKEv1 aggressive mode only
requires three messages to
establish the security association.
It’s quicker than main mode since it
adds all the information required
for the DH exchange in the first two
messages.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 1
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 2
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Message 3
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Phase-2 (Quick Mode)
In Quick Mode, 3 messages are exchanged which basically contains the
parameters/ protocols/ algorithms to be used for the secure data
transfer. Once this negotiation is completed, User traffic (data plane) can
be started moving on the IPSEC tunnel.
• Quick Mode Message-1: Initiator sends this packet which contains
majorly 3 important things. One is the protocol of encapsulation to
be used for data plane whether ESP or AH, Second is the Encryption
algorithm to be used for encrypting data plane traffic and third is the
Hashing Algorithm to be used for integrity check of data plane traffic.
This message is sent in an encrypted packet.
• Quick Mode Message-2: Responder sends this packet to the Initiator
containing its own parameters/protocols/algorithms for
encapsulation, encryption, hashing. This message is also encrypted.
• Quick Mode Message-3: Once the parameters match on both ends for data
plane traffic exchanged in Quick Mode Messages 1 and 2, the initiator
sends a message to start the data traffic. This message is also encrypted.
After these 3 messages of Quick Mode, User traffic can be sent over an
IPSEC tunnel using the parameters NEGOTIATED IN Quick Mode.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IKE Phase 2 cont….
There is only one mode to build the
IKE phase 2 tunnel which is called
quick mode.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Msg 2 & 3
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Protocols
IKE builds the tunnels but it doesn’t
authenticate or encrypt user data. For
this:
AH (Authentication Header)
ESP (Encapsulating Security Payload)
Both offer authentication and integrity
but only ESP supports encryption.
Both protocols support two different
modes:
Transport mode
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Modes of Ipsec
Transport mode
• It protects the nw layer payload, the payload is
encapsulated in the nw layer.
• It does not protect the IP header
• Ipsec header is added to the information coming
from the transport layer.
• Normally used when we need host-to-host protection
of data.
Transport mode in action
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Tunnel mode
• It protects the entire IP packet.
• It takes an IP packet, including the header, applies
IPSec security methods to the entire packet, then
adds a new IP header.
• Normally used b/w routers, b/w routers & host, b/w
host & router.
Tunnel mode in action
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Tunnel mode
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Figure 32.3 Transport mode and tunnel modes of IPSec protocol
32.55
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Authentication Header
Protocol
AH offers authentication and
integrity, but it doesn’t offer any
encryption.
It protects the IP packet by
calculating a hash value over
almost all fields in the IP header.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
AH cont…
An authentication header is added to
the payload with the authentication
data field set to 0.
Padding may be added to the total
length even for a particular hashing
algo.
Hashing is based on the total packet.
Authen. Data are inserted in the AH.
IP header is added after changing the
protocol field to 51.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
ESP (Encapsulating
Security Payload)
Provides src authentication,
integrity & privacy.
ESP adds header & trailer.
ESP au. data added at the end of
the packet, so that the calculation
is easy.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec Tunnel Mode
IPSec tunnel mode is the default mode. With tunnel mode,
the entire original IP packet is protected by IPSec. This
means IPSec wraps the original packet, encrypts it, adds a
new IP header and sends it to the other side of the VPN
tunnel (IPSec peer).
Tunnel mode is most commonly used between gateways
(Cisco routers or ASA firewalls), or at an end-station to a
gateway, the gateway acting as a proxy for the hosts
behind it.
Tunnel mode is used to encrypt traffic between secure
IPSec Gateways, for example two Cisco routers connected
over the Internet via IPSec VPN.
In this example, each router acts as an IPSec Gateway for
their LAN, providing secure connectivity to the remote
network:
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec Tunnel Mode
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPSec Transport Mode
IPSec Transport mode is used for end-to-end communications, for
example, for communication between a client and a server or
between a workstation and a gateway (if the gateway is being
treated as a host). A good example would be an encrypted Telnet or
Remote Desktop session from a workstation to a server.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
IPsec is a suite of protocols for securing networks. Briefly outline how it
provides confidentiality, integrity and authentication.
Confidentiality: through Encryption of data
Integrity: Routers at each end of the tunnel calculate checksum or
hash. Integrity (sequence integrity): Anti-replay protection through
sequence numbers.
Authentication: It involves Signatures and Certificates
Briefly explain what is by an Authentication Header (AH) and an
Encapsulating Security Payload (ESP).
Draw a diagram to show where IPSec fits in the TCP/IP model
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
Employees are increasingly connecting to company networks remotely via mobile
devices such as laptops, tablets and smartphones. Remote access needs to satisfy
five essential requirements to be efficient and secure. Identify and briefly explain each
of these FIVE (5) requirements.
Authentication – validates that the data was sent from the sender.
Access Control – preventing unauthorized users from accessing the network.
Confidentiality – preventing the data from being read or copied as the data is
being transported.
Data Integrity – ensuring that the data has not been altered.
Availability – ensuring that a connection can be made.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
There are several methods of achieving secure remote access. One important method
is to use a VPN. Explain if/ how a VPN achieves each of the requirements in part (a)
5
Authentication of the user, typically via RADIUS.
Access control – only authorized users can access, e.g. through access
rights on the authentication server.
Confidentiality via encryption of data across the insecure network
Integrity –via MAP, replay attack prevention
Availability cannot be provided by a VPN through protocols, but by capacity
and resilience of the architecture.
McGraw-Hill © The McGraw-Hill Companies, Inc., 2004
