Chapter 2

Application
Layer

Computer
Networking: A Top
Down Approach
7th Edition, Global Edition
Jim Kurose, Keith Ross
Pearson
April 2016
Application Layer 2-1
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming with
UDP and TCP

Application Layer 2-2

 Chapter 2: application
layer
our goals:
 conceptual,  learn about protocols by
implementation aspects examining popular
of network application application-level
protocols protocols
• transport-layer • HTTP
service models • FTP
• SMTP / POP3 / IMAP
• client-server
• DNS
paradigm
 creating network
• peer-to-peer
applications
paradigm
• socket API
• content distribution
networks

Application Layer 2-3

Some network apps
 e-mail  Voice over IP
 web (e.g., Skype)
 text messaging  real-time video
 conferencing
remote login
 social
 P2P file sharing
networking
 multi-user network  search
games
 …
 streaming stored video
(YouTube, Hulu,  …
Netflix)

Application Layer 2-4

Creating a network app application
transport
network
data link
physical

write programs that:

 run on (different) end
systems
 communicate over
network
 e.g., web server software
communicates with
browser software application
transport
network
data link application

no need to write physical transport

network

software for network-

data link
physical

core devices
 network-core devices do
not run user applications
 applications on end
systems allows for rapid Application Layer 2-5
Application architectures
possible structure of applications:
 client-server
 peer-to-peer (P2P)

Application Layer 2-6

 Client-server architecture
server:
 always-on host
 permanent IP address
 data centers for scaling

clients:
 communicate with
server
client/server
 may be intermittently
connected
 may have dynamic IP
addresses
 do not communicate
directly with each
other Application Layer 2-7
P2P architecture
 no always-on server peer-peer
 arbitrary end systems
directly communicate
 peers request service
from other peers,
provide service in return
to other peers
• self scalability – new
peers bring new
service capacity, as
well as new service
demands
 peers are intermittently
connected and change
IP addresses
• complex Application Layer 2-8
2.1.2 Processes communicating

process: program clients, servers

running within a client process: process
host that initiates
 within same host, two communication
processes server process:
communicate using process that waits to
inter-process be contacted
communication  aside: applications
(defined by OS)
with P2P architectures
 processes in different
have client processes
hosts communicate
by exchanging & server processes
messages

Application Layer 2-9

 Sockets
 The interface between the process and the computer
network
 process sends/receives messages to/from its socket
 socket analogous to door
• sending process shoves message out door
• sending process relies on transport infrastructure
on other side of door to deliver message to socket
at receiving process
 It is also referred to as the API (Application
application
Programming Interface)
application socket
process
controlled by
process app developer

transport transport
network network controlled
link
by OS
link Internet
physical physical

Application Layer 2-10

Addressing processes
 to receive messages,  identifier includes both
process must have IP address and port
identifier numbers associated
 host device has unique with process on host.
32-bit IP address  example port numbers:
 Q: does IP address of • HTTP server: 80
host on which process • mail server: 25
runs suffice for  to send HTTP message
identifying the process? to gaia.cs.umass.edu
 A: no, many web server:
• IP address:
processes can be 128.119.245.12
running on same • port number: 80
host  more shortly…

Application Layer 2-11

App-layer protocol defines
 types of messages open protocols:
exchanged,  defined in RFCs
• e.g., request, response  allows for
 message syntax: interoperability
• what fields in messages  e.g., HTTP, SMTP
& how fields are proprietary protocols:
delineated  e.g., Skype
 message semantics
• meaning of information
in fields
 rules for when and how
processes send & respond
to messages

Application Layer 2-12

What transport service does an app
need?
Reliable data transfer throughput
 some apps (e.g., file  some apps (e.g.,
transfer, web multimedia) require
transactions) require minimum amount of
100% reliable data throughput to be
transfer “effective”
 other apps (e.g., audio)  other apps (“elastic
can tolerate some loss apps”) make use of
timing whatever throughput
 some apps (e.g., security
they get
Internet telephony,  encryption, data
interactive games) integrity, …
require low delay to be
“effective”
Application Layer 2-13
Transport service requirements: common
apps

application data loss throughput time sensitive

file transfer no loss elastic no

e-mail no loss elastic no
Web documents no loss elastic no
real-time loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec
audio/video video:10kbps-5Mbps yes, few secs
stored audio/video loss-tolerant same as above yes, 100’s msec
interactive games loss-tolerant few kbps up yes and no

text messaging no loss elastic

Application Layer 2-14

Internet transport protocols
services
TCP service: UDP service:
 connection-oriented:  unreliable data
setup required between transfer between
client and server sending and receiving
processes process
 reliable transport  does not provide:
between sending and reliability, flow
receiving process control, congestion
 flow control: sender control, timing,
won’t overwhelm throughput
receiver guarantee, security,
 congestion control: or connection setup,
throttle sender when
network overloaded Q: why bother? Why is
 does not provide: there a UDP?
timing, minimum Application Layer 2-15
Internet apps: application, transport
protocols
application underlying
application layer protocol transport protocol

e-mail SMTP [RFC 2821] TCP

remote terminal access Telnet [RFC 854] TCP
Web HTTP [RFC 2616] TCP
file transfer FTP [RFC 959] TCP
streaming multimedia HTTP (e.g., YouTube), TCP or UDP
RTP [RFC 1889]
Internet telephony SIP, RTP, proprietary
(e.g., Skype) TCP or UDP

Application Layer 2-16

Securing TCP

TCP & UDP SSL is at app layer

 no encryption  apps use SSL
 cleartext passwds libraries, that “talk”
sent into socket to TCP
traverse Internet in SSL socket API
cleartext  cleartext passwords
SSL sent into socket
 provides encrypted traverse Internet
TCP connection encrypted
 data integrity  see Chapter 8
 end-point
authentication
Application Layer 2-17
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming
with UDP and TCP

Application Layer 2-18

Web and HTTP
First, a review…
 web page consists of objects
 object can be HTML file, JPEG image,
Java applet, audio file,…
 web page consists of base HTML-file
which includes several referenced
objects
 each object is addressable by a URL,
www.someschool.edu/someDept/pic.gif
e.g.,
host name path name

Application Layer 2-19

 HTTP overview
HTTP: hypertext
transfer protocol HT
TP
req
 Web’s app layer protocol PC running HT
ues
t
Firefox browser TPr
 client/server model esp
ons
e
• client: browser that
t
requests, receives, u es
req server
P se
(using HTTP) and HT
T
es po
n running
r Apache Web
“displays” Web objects HT
T P
server
• server: Web server
sends (using HTTP) iPhone running
Safari browser
objects in response to
requests
Application Layer 2-20
HTTP overview (continued)
uses TCP: HTTP is “stateless”
 client initiates TCP  server maintains no
connection (creates information about
socket) to server, port past client requests
80
 server accepts TCP aside
connection from client protocols that maintain
 HTTP messages (app- “state” are complex!
layer protocol  past history (state) must
messages) exchanged be maintained
between browser (HTTP  if server/client crashes,
client) and Web server their views of “state”
may be inconsistent,
(HTTP server) must be reconciled
 TCP connection closed
Application Layer 2-21
 HTTP connections
non-persistent HTTP persistent HTTP
 at most one object （ default ）
sent over TCP  multiple objects can
connection be sent over single
• connection then TCP connection
closed between client,
 downloading multiple server
objects required
multiple connections

Application Layer 2-22

 Non-persistent HTTP
suppose user enters URL: (contains text,
www.someSchool.edu/someDepartment/home.index references to 10
jpeg images)
1a. HTTP client initiates TCP
connection to HTTP server
(process) at 1b. HTTP server at host
www.someSchool.edu on www.someSchool.edu
port 80 waiting for TCP
connection at port 80.
2. HTTP client sends HTTP “accepts” connection,
request message notifying client
(containing URL) into TCP 3. HTTP server receives
connection socket. request message, forms
Message indicates that response message
client wants object containing requested
someDepartment/home.i object, and sends
time ndex message into its socket
Application Layer 2-23
 Non-persistent HTTP (cont.)
4. HTTP server closes TCP
connection.
5. HTTP client receives
response message
containing html file,
displays html. Parsing html
file, finds 10 referenced jpeg
objects
time
6. Steps 1-5 repeated for
each of 10 jpeg objects

Note: each TCP connection transports exactly

one request message and one response
message.
11 TCP connections are generated in this
example. Application Layer 2-24
Non-persistent HTTP: response
time
RTT (Round-Trip Time): time
for a small packet to travel
from client to server and
back initiate TCP
connection
HTTP response time: RTT
 one RTT to initiate TCP request
connection file
 one RTT for HTTP request RTT
time to
transmit
and first few bytes of HTTP file
response to return file
received
 file transmission time
 non-persistent HTTP time time
response time =
2RTT+ file transmission
time
Application Layer 2-25
Persistent HTTP

non-persistent HTTP persistent HTTP:

issues:  server leaves connection
 requires 2 RTTs per open after sending
object response
 OS overhead for each  subsequent HTTP
TCP connection messages between same
 browsers often open client/server sent over
parallel TCP open connection
connections to fetch  client sends requests as
referenced objects soon as it encounters a
referenced object
 as little as one RTT for all
the referenced objects
 Typically, the HTTP server
closes a connection after
a timeout interval
Application Layer 2-26
 HTTP request message
 two types of HTTP messages: request, response
 HTTP request message:
• ASCII (human-readable format)
carriage return character
line-feed character
request line
(GET, POST, HEAD,GET /index.html HTTP/1.1\r\n
PUT, Delete) Host: www-net.cs.umass.edu\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
header Accept-Language: en-us,en;q=0.5\r\n
lines Accept-Encoding: gzip,deflate\r\n
carriage return, Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n

Connection: keep-alive\r\n (connection: close)

line feed at start Keep-Alive: 115\r\n
of line indicates \r\n
end of header lines

* Check out the online interactive exercises for more

examples: http://gaia.cs.umass.edu/kurose_ross/interactive/ Application Layer 2-27
HTTP request message: general
format
method sp URL sp version cr lf request
line
header field name value cr lf
header
~
~ ~
~ lines

header field name value cr lf

cr lf

~
~ entity body ~
~ body

Application Layer 2-28

Uploading form input
POST method:
 web page often includes form input
 input is uploaded to server in entity
body

URL method:
 uses GET method
 input is uploaded in URL field of request
line:

www.somesite.com/animalsearch?monkeys&banana

Application Layer 2-29

Method types
HTTP/1.0: HTTP/1.1:
 GET  GET, POST, HEAD
 POST  PUT
 HEAD • uploads file in
• asks server to entity body to
leave requested path specified in
object out of URL field
response  DELETE
• deletes file
specified in the
URL field

Application Layer 2-30

 HTTP response message
status line
(protocol status
code status HTTP/1.1 200 OK\r\n
phrase) Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n
Server: Apache/2.0.52 (CentOS)\r\n
Last-Modified: Tue, 30 Oct 2007 17:00:02 GMT\r\
n
header ETag: "17dc6-a5c-bf716880"\r\n
Accept-Ranges: bytes\r\n
lines Content-Length: 2652\r\n
Keep-Alive: timeout=10, max=100\r\n
Connection: Keep-Alive\r\n
Content-Type: text/html; charset=ISO-8859-1\r\n
\r\n
data, e.g., data data data data data ...
requested
HTML file

* Check out the online interactive exercises for more

examples: http://gaia.cs.umass.edu/kurose_ross/interactive/ Application Layer 2-31
HTTP response status codes
 status code appears in 1st line in server-to-client
response message.
 some sample codes:
200 OK
• request succeeded, requested object later in this msg
301 Moved Permanently
• requested object moved, new location specified later in
this msg (Location:)
400 Bad Request
• request msg not understood by server
404 Not Found
• requested document not found on this server
505 HTTP Version Not Supported

Application Layer 2-32

 Trying out HTTP (client side) for
yourself
1. Telnet to your favorite Web server:
telnet gaia.cs.umass.edu 80 opens TCP connection to port 80
(default HTTP server port)
at gaia.cs.umass. edu.
anything typed in will be sent
to port 80 at gaia.cs.umass.edu

2. type in a GET HTTP request:

GET /kurose_ross/interactive/index.php HTTP/1.1
Host: gaia.cs.umass.edu by typing this in (hit carriage
return twice), you send
this minimal (but complete)
GET request to HTTP server

3. look at response message sent by HTTP server!

or use Wireshark to look at captured HTTP request/response)
Application Layer 2-33
User-server state: cookies
many Web sites use example:
cookies  Susan always access
four components: Internet from PC
1) cookie header line  visits specific e-
of HTTP response commerce site for first
message time
 when initial HTTP
2) cookie header line
in next HTTP request requests arrives at site,
message site creates:
• unique ID
3) cookie file kept on
user’s host, • entry in backend
managed by user’s database for ID
browser
4) back-end database
Application Layer 2-34
at Web site
 Cookies: keeping “state” (cont.)
client server

ebay 8734
usual http request msg Amazon server
cookie file creates ID
usual http response
1678 for user create backend
ebay 8734
set-cookie: 1678 entry database
amazon 1678
usual http request msg
cookie: 1678 cookie- access
specific
usual http response msg action

one week later:

access
ebay 8734 usual http request msg
amazon 1678 cookie: 1678 cookie-
specific
usual http response msg action
Application Layer 2-35
在 chrome 中查看
cookie
chrome://
settings/
content/
cookies

Application Layer 2-36

Cookies (continued)
what cookies can be aside
cookies and privacy:
used for:
  cookies permit sites
authorization
 shopping carts to learn a lot about
 recommendations you
 user session state (Web  you may supply
e-mail) name and e-mail to
sites
how to keep “state”:
 protocol endpoints: maintain state at
sender/receiver over multiple transactions
 cookies: http messages carry state

Application Layer 2-37

 2.2.5 Web caches (proxy
server)
goal: satisfy client request without involving
origin server
user sets browser:
Web accesses via
cache
 browser sends all HT proxy
TP u est
req server req
HTTP requests to HT
client TP
ues
t H TTP
o nse
res p origin
cache pon P res
se T server
HT
• object in cache: ues
t
cache returns req e
TT P o ns
p
object H res
T TP
H
• else cache
requests object client origin
from origin server, server
then returns
object to client
Application Layer 2-38
More about Web caching
 cache acts as both why Web caching?
client and server  reduce response time
• server for original for client request
requesting client
• client to origin server  reduce traffic on an
 typically cache is institution’s access
installed by ISP link
(university,  Internet dense with
company, caches: enables
residential ISP) “poor” content
providers to
effectively deliver
content (so too does
P2P file sharing)
Application Layer 2-39
Caching example:
assumptions:
 avg object size: 100K bits
origin
 avg request rate from servers
browsers to origin public
servers:15/sec Internet
 avg data rate to browsers:
1.50 Mbps
 RTT from institutional router
to any origin server: 2 sec 1.54 Mbps
 access link rate: 1.54 Mbps access link

problem! institutional
consequences: network
 LAN utilization: 0.15% 1 Gbps LAN
 access link utilization = 99%
 total delay = Internet delay
+ access delay + LAN delay
= 2 sec + minutes + usecs
Application Layer 2-40
 Caching example: fatter
access link
assumptions:
 avg object size: 100K bits
origin
 avg request rate from servers
browsers to origin public
servers:15/sec Internet
 avg data rate to browsers:
1.50 Mbps
 RTT from institutional router
to any origin server: 2154
sec 1.54 Mbps
154 Mbps
 access link rate: 1.54 Mbps access link
Mbps
institutional
consequences: network
 LAN utilization: 0.15%9.9% 1 Gbps LAN
 access link utilization = 99%
 total delay = Internet delay +
access delay + LAN delay
msecs + usecs
= 2 sec + minutes
Cost: increased access link speed (not cheap!)
Application Layer 2-41
 Caching example: install local
cache
assumptions:
 avg object size: 100K bits
origin
 avg request rate from servers
browsers to origin public
servers:15/sec Internet
 avg data rate to browsers:
1.50 Mbps
 RTT from institutional router
to any origin server: 2 sec 1.54 Mbps
 access link rate: 1.54 Mbps access link
institutional
consequences: network
 ?
LAN utilization: 15% 1 Gbps LAN
 ?
access link utilization = 100%
Howdelay
to compute linkdelay + local web
 total = Internet
utilization, cache
access delay + delay?
LAN delay
= 2 sec + minutes + usecs
Cost: web cache (cheap!)
Application Layer 2-42
Caching example: install local
cache
Calculating access link
utilization, delay with
origin
cache: servers
 suppose cache hit rate is 0.4 public
• 40% requests satisfied at cache, Internet
60% requests satisfied at origin
 access link utilization:
 60% of requests use access link
 data rate to browsers over access 1.54 Mbps
access link
link
= 0.6*1.50 Mbps = 0.9 Mbps institutional
 utilization = 0.9/1.54 = 0.58 network
1 Gbps LAN
 total delay
 = 0.6 * (delay from origin servers) local web
+0.4 * (delay when satisfied at cache
cache)
 = 0.6 (2.01) + 0.4 (~msecs) = ~
1.2 secs
 less than with 154 Mbps link (and Application Layer 2-43
cheaper too!)
 2.2.6 Conditional GET
Cache server
 Goal: don’t send object
if cache has up-to-date
cached version HTTP request msg
object
If-modified-since: <date>
• no object transmission not
delay modified
• lower link utilization HTTP response
after
HTTP/1.0
 cache: specify date of 304 Not Modified <date>
cached copy in HTTP
request
If-modified-since:
<date> HTTP request msg
 server: response If-modified-since: <date> object
contains no object if modified
after
cached copy is up-to- HTTP response
HTTP/1.0 200 OK <date>
date:
<data>
HTTP/1.0 304 Not
Modified Application Layer 2-44
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming
with UDP and TCP

Application Layer 2-45

Electronic mail outgoing
message queue
user mailbox
Three major user
agent
components:
 user agents mail user
server
 mail servers agent

 simple mail transfer SMTP mail user

protocol: SMTP server agent
SMTP
User Agent SMTP user
 a.k.a. “mail reader” mail
agent
 composing, editing, server
user
reading mail messages agent
 e.g., Outlook, user
Thunderbird, iPhone mail agent
client
 outgoing, incoming Application Layer 2-46
Electronic mail: mail servers
mail servers: user
agent
 mailbox contains
incoming messages for mail user
server
user agent
 message queue of SMTP mail user
outgoing (to be sent) server agent
mail messages SMTP
 SMTP protocol between
SMTP user
mail servers to send agent
email messages mail
server
• client: sending mail user
server agent

• “server”: receiving user

agent
mail server
Application Layer 2-47
Electronic Mail: SMTP [RFC
2821]
 uses TCP to reliably transfer email message
from client to server, port 25
 direct transfer: sending server to receiving
server
 three phases of transfer
• handshaking (greeting)
• transfer of messages
• closure
 command/response interaction (like HTTP)
• commands: ASCII text
• response: status code and phrase
 messages must be in 7-bit ASCII
Application Layer 2-48
Scenario: Alice sends message to
Bob
1) Alice uses UA to 4) SMTP client sends
compose message “to” Alice’s message over
[email protected] the TCP connection
2) Alice’s UA sends 5) Bob’s mail server
message to her mail places the message in
server; message Bob’s mailbox
placed in message 6) Bob invokes his user
queue agent to read message
3) client side of SMTP
opens TCP connection
with Bob’s mail server
1 user mail user
mail agent
agent server server
2 3 6
4
5
Alice’s mail server Bob’s mail server
Application Layer 2-49
Sample SMTP interaction
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <[email protected]>
S: 250 [email protected]... Sender ok
C: RCPT TO: <[email protected]>
S: 250 [email protected] ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection

Application Layer 2-50

Try SMTP interaction for yourself:
 telnet servername 25
 see 220 reply from server
 enter HELO, MAIL FROM, RCPT TO, DATA, QUIT
commands

above lets you send email without using email

client (reader)

Application Layer 2-51

SMTP: final words
 SMTP uses persistent comparison with HTTP:
connections  HTTP: pull
 SMTP requires
 SMTP: push
message (header &
body) to be in 7-bit  both have ASCII
ASCII command/response
 SMTP server uses interaction, status codes
CRLF.CRLF to
determine end of  HTTP: each object
message encapsulated in its own
response message
 SMTP: multiple objects
sent in multipart
message

Application Layer 2-52

 Mail message format
SMTP: protocol for
exchanging email header
messages blank
line
RFC 822: standard for text
message format:
 header lines, e.g.,
body
• To:
• From:
• Subject:
different from SMTP MAIL
FROM, RCPT TO:
commands!
 Body: the “message”
• ASCII characters only
Application Layer 2-53
 Mail access protocols
user
mail user
SMTP SMTP access
agent agent
protocol
(e.g., POP,
IMAP)

sender’s mail receiver’s mail

server server

 SMTP: delivery/storage to receiver’s server

 mail access protocol: retrieval from server
• POP: Post Office Protocol [RFC 1939]: authorization,
download
• IMAP: Internet Mail Access Protocol [RFC 1730]: more
features, including manipulation of stored messages on
server
• HTTP: gmail, Hotmail, Yahoo! Mail, etc.
Application Layer 2-54
POP3 protocol
S: +OK POP3 server ready
C: user bob
authorization phase S: +OK
 client commands: C: pass hungry
S: +OK user successfully logged on
• user: declare username
• pass: password C: list
 server responses S: 1 498
• +OK S: 2 912
S: .
• -ERR
C: retr 1
transaction phase, client: S: <message 1 contents>
 list: list message numbers S: .
 retr: retrieve message by C: dele 1
number C: retr 2
 dele: delete S: <message 2 contents>
S: .
 quit
C: dele 2
C: quit
S: +OK POP3 server signing off
Application Layer 2-55
 POP3 (more) and IMAP
more about POP3 IMAP
 previous example uses  keeps all messages in
POP3 “download and one place: at server
delete” mode  allows user to organize
• Bob cannot re-read messages in folders
e-mail if he changes  keeps user state across
client sessions:
 POP3 “download-and- • names of folders and
keep” mode: mappings between
• copies of messages message IDs and
on different clients folder name
 POP3 is stateless
across sessions
(simple)
 POP3 cannot create
folders on the server Application Layer 2-56
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming with
UDP and TCP

Application Layer 2-57

DNS: domain name system
people: many identifiers: Domain Name System:
• SSN, name,  distributed database
passport # implemented in
Internet hosts, routers: hierarchy of many name
• IP address (32 bit) - servers
used for addressing  application-layer
datagrams protocol: hosts, name
• “name”, e.g., servers communicate to
www.yahoo.com - resolve names
used by humans (address/name
translation)
Q: how to map between
IP address and name, • note: core Internet
and vice versa ? function, implemented
as application-layer
protocol
• complexity atApplication Layer 2-58
 DNS: services, structure
DNS services why not centralize
 hostname to IP address DNS?
translation  single point of failure
 host aliasing  traffic volume
• canonical, alias names  distant centralized
• relay1.west- database
coast.hotmail.com VS
www.hotmail.com  maintenance
A: doesn‘t scale!
 mail server aliasing
 load distribution
• replicated Web
servers: many IP
addresses correspond
to one name

Application Layer 2-59

DNS: a distributed, hierarchical
database
Root DNS Servers Root DNS Server

… … Top-Level Domain, TLD

com DNS servers org DNS servers edu DNS servers

pbs.org poly.edu umass.edu

yahoo.com amazon.com
DNS servers DNS serversDNS servers
DNS servers DNS servers
Authoritative DNS Server

client wants IP for www.amazon.com; 1st

approximation:
 client queries root server to find com DNS server
 client queries .com DNS server to get amazon.com
DNS server
 client queries amazon.com DNS server to get IP
address for www.amazon.com
Application Layer 2-60
 DNS: root name servers
 contacted by local name server that can not resolve
name
 root name server:
• contacts authoritative name server if name mapping not
known
• gets mapping
• returns mapping
c. Cogent, Herndon, VA (5 otherto
sites) local name server
d. U Maryland College Park, MD k. RIPE London (17 other sites)
h. ARL Aberdeen, MD
j. Verisign, Dulles VA (69 other sites ) i. Netnod, Stockholm (37 other sites)

e. NASA Mt View, CA m. WIDE Tokyo

f. Internet Software C. (5 other sites)
Palo Alto, CA (and 48 other
sites)

a. Verisign, Los Angeles CA

13 logical root name
(5 other sites) “servers” worldwide
b. USC-ISI Marina del Rey, CA
l. ICANN Los Angeles, CA
（ 2012 ）
(41 other sites)
g. US DoD Columbus,
• each “server” replicated
OH (5 other sites) many times

Application Layer 2-61

TLD, authoritative servers
top-level domain (TLD) servers:
• responsible for com, org, net, edu, aero, jobs,
museums, and all top-level country domains, e.g.:
cn, uk, fr, ca, jp
• Network Solutions maintains servers for .com TLD
• Educause for .edu TLD
authoritative DNS servers:
• organization’s own DNS server(s), providing
authoritative hostname to IP mappings for
organization’s named hosts
• can be maintained by organization or service
provider

Application Layer 2-62

Local DNS name server
 does not strictly belong to hierarchy
 each ISP (residential ISP, company,
university) has one
• also called “default name server”
 when host makes DNS query, query is sent
to its local DNS server
• has local cache of recent name-to-address
translation pairs (but may be out of date!)
• acts as proxy, forwards query into hierarchy

Application Layer 2-63

DNS name
resolution root DNS server

example 2
 host at cis.poly.edu 3
TLD DNS server
wants IP address 4
for
gaia.cs.umass.edu 5

local DNS server

iterated query: dns.poly.edu
 contacted server 7 6
1 8
replies with name
of server to
authoritative DNS server
contact dns.cs.umass.edu
 “I don’t know this requesting host
name, but ask this cis.poly.edu
server”
gaia.cs.umass.edu

Application Layer 2-64

DNS name
resolution root DNS server

example 2 3
7
recursive query: 6
 puts burden of TLD DNS
server
name resolution on
contacted name local DNS server
server dns.poly.edu 5 4

 heavy load at upper 1 8

levels of hierarchy?
authoritative DNS server
dns.cs.umass.edu
requesting host
cis.poly.edu

gaia.cs.umass.edu

Application Layer 2-65

DNS: caching, updating
records
 once (any) name server learns mapping, it
caches mapping
• cache entries timeout (disappear) after some time
(TTL, often set to two days)
• TLD servers typically cached in local name
servers
• thus root name servers not often visited
 cached entries may be out-of-date (best
effort name-to-address translation!)
• if name host changes IP address, may not be
known Internet-wide until all TTLs expire
 update/notify mechanisms proposed IETF
standard
• RFC 2136
Application Layer 2-66
DNS records
DNS: distributed database storing resource records
(RR)
RR format: (name, value, type, ttl)

The meaning of name and value depend on Type:

type=A type=CNAME
 name is hostname  name is alias name for some
 value is IP address “canonical” (the real) name
 www.ibm.com is really
type=NS
• name is domain servereast.backup2.ibm.com
(e.g., foo.com)  value is canonical name
• value is hostname
of authoritative type=MX
name server for this  value is name of
domain
mailserver associated with
name Application Layer 2-67
DNS protocol, messages
 query and reply messages, both with
same message format2 bytes 2 bytes

message header identification flags

 identification: 16 bit # # questions # answer RRs

for query, reply to query
# authority RRs # additional RRs
uses same #
 flags: questions (variable # of questions)
 query (0) or reply (1)
 recursion desired (1) answers (variable # of RRs)
 recursion available
(1) authority (variable # of RRs)
 reply is authoritative
(1) additional info (variable # of RRs)

Application Layer 2-68

DNS protocol, messages

2 bytes 2 bytes

identification flags

# questions # answer RRs

# authority RRs # additional RRs

name, type fields

questions (variable # of questions)
for a query
RRs in
response answers (variable # of RRs)
to query
records for authority (variable # of RRs)
authoritative servers
additional “helpful” additional info (variable # of RRs)
info that may be used
Application Layer 2-69
Inserting records into DNS
 example: new startup “Network Utopia”
 register name networkutopia.com at DNS
registrar (e.g., Network Solutions)
• provide names, IP addresses of authoritative
name server (primary and secondary)
• registrar inserts two RRs into .com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
 create authoritative server type A record for
www.networkuptopia.com; type MX record
for networkutopia.com

Application Layer 2-70

Attacking DNS
DDoS attacks redirect attacks
 bombard root  man-in-middle
servers with traffic • Intercept queries
• not successful to date  DNS poisoning
• traffic filtering  Send bogus relies to
• local DNS servers DNS server, which
cache IPs of TLD caches
servers, allowing root exploit DNS for DDoS
server bypass
 bombard TLD  send queries with
servers spoofed source
address: target IP
• potentially more
dangerous  requires
amplification
Application Layer 2-71
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming
with UDP and TCP

Application Layer 2-72

Pure P2P architecture
 no always-on server
 arbitrary end systems
directly communicate
 peers are
intermittently
connected and
change IP addresses
examples:
• file distribution
(BitTorrent)
• Streaming (KanKan)
• VoIP (Skype)

Application Layer 2-73

File distribution: client-server vs P2P
Question: how much time to distribute file
(size F) from one server to N peers?
• peer upload/download capacity is limited
resource
us: server upload
capacity

u1 di: peer i download

file, size F us d1 u2 capacity
d2
server
di
uN network (with abundant
bandwidth) ui
dN
ui: peer i upload
capacity

Application Layer 2-74

File distribution time: client-server
 server transmission:
must sequentially send F
us
(upload) N file copies:
di
• time to send one copy:
network
F/us ui
• time to
 client: sendclient
each N copies:
must
NF/us
download file copy
• dmin = min client download
rate
• min client download time:
F/dmin time to distribute F
to N clients using
Dc-s > max{NF/us,,F/dmin}
client-server approach

increases linearly in N
Application Layer 2-75
File distribution time: P2P
 server transmission: must
upload at least one copy F
us
• time to send one copy: F/us
di
 client: each client must
network
download file copy ui
• min client download time:
F/dmin as aggregate must download NF bits
 clients:
• max upload rate (limiting max download rate) is us +
Sui
time to distribute F
DP2P > max{F/us,,F/dmin,,NF/(us + Sui)}
to N clients using
P2P approach

increases linearly in N …
… but so does this, as each peer brings service capacity
Application Layer 2-76
 Client-server vs. P2P: example
client upload rate = u, F/u = 1 hour, us = 10u, dmin ≥ us

3.5
P2P
Minimum Distribution Time

3
Client-Server
2.5

1.5

0.5

0
0 5 10 15 20 25 30 35

N
Application Layer 2-77
 P2P file distribution:
BitTorrent
 file divided into 256Kb chunks
 peers in torrent send/receive file chunks

tracker: tracks peers torrent: group of peers

participating in torrent exchanging chunks of a
file

Alice arrives …
… obtains list
of peers from tracker
… and begins exchanging
file chunks with peers in torrent

Application Layer 2-78

P2P file distribution:
BitTorrent
 peer joining torrent:
• has no chunks, but will
accumulate them over
time from other peers
• registers with tracker to
get list of peers, connects
to subset of peers
(“neighbors”)
 while downloading, peer uploads chunks to other
peers
 peer may change peers with whom it exchanges
chunks
 churn: peers may come and go
 once peer has entire file, it may (selfishly) leave or
(altruistically) remain in torrent Application Layer 2-79
BitTorrent: requesting, sending file
chunks
requesting chunks: sending chunks: tit-for-
 at any given time, tat
different peers have  Alice sends chunks to those
different subsets of file four peers currently
chunks sending her chunks at
 periodically, Alice asks highest rate
each peer for list of • other peers are choked by
chunks that they have Alice (do not receive chunks
 Alice requests missing from her)
chunks from peers, • re-evaluate top 4 every10
secs
rarest first
 every 30 secs: randomly
select another peer, starts
sending chunks
• “optimistically unchoke” this
peer
Application Layer 2-80
• newly chosen peer may join
 BitTorrent: tit-for-tat
(1) Alice “optimistically unchokes” Bob
(2) Alice becomes one of Bob’s top-four providers; Bob reciprocates
(3) Bob becomes one of Alice’s top-four providers

higher upload rate: find

better trading partners, get
file faster !
Application Layer 2-81
Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks (CDNs)
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming
with UDP and TCP

Application Layer 2-82

Video Streaming and CDNs:
context
 video traffic: major consumer of Internet bandwidth
• Netflix, YouTube: 37%, 16% of downstream
residential ISP traffic
• ~1B YouTube users, ~75M Netflix users
 challenge: scale - how to reach ~1B users?
• single mega-video server won’t work (why?)
 challenge: heterogeneity
 different users have different capabilities (e.g.,
wired versus mobile; bandwidth rich versus
bandwidth poor)
 solution: distributed, application-level
infrastructure

Application Layer 2-83

 Multimedia: spatial coding example: instead
of sending N values of same

video
color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 video: sequence of
images displayed at ……………………..
……………….…….
constant rate
• e.g., 24 or 30
images/sec
 digital image: array of
pixels
• each pixel represented frame i
by bits
 coding: use redundancy
within and between
images to decrease # bitstemporal coding example:
instead of sending
used to encode image complete frame at i+1,
send only differences from
• spatial (within image) frame i frame i+1
• temporal (from one
image to next) Application Layer 2-84
 Multimedia: spatial coding example: instead
of sending N values of same

video
color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 CBR: (constant bit rate):
video encoding rate fixed ……………………..
……………….…….
 VBR: (variable bit rate):
video encoding rate
changes as amount of
spatial, temporal coding
changes
 examples: frame i
• MPEG 1 (CD-ROM) 1.411
Mbps
• MPEG1 Layer 3- MP3 temporal coding example:
128kbps instead of sending
complete frame at i+1,
• MPEG2 (DVD) 3-6 Mbps send only differences from
• MPEG4 (often used in frame i
frame i+1
Internet, < 1 Mbps)

Application Layer 2-85

Streaming stored video:
simple scenario:

Internet

video server client

(stored video)

流式存储视频三个特色：
 流：客户下载一部分文件后即可播放 - 要求将文件分片； 客户缓存
 相互作用：暂停、前进、后退等操作 - 如 TCP 缓存机制、 HTTP 字节范围首部实现前进后退并防止浪费）
 连续播放：自动连续接收数据
UDP 流、 HTTP 流、适应性流 DASH
Application Layer 2-86
 2.6.2 Streaming multimedia:
DASH
 DASH: Dynamic, Adaptive Streaming over
HTTP
 server:
• divides video file into multiple chunks (e.g.
4sec/chunk in Netflix)
• each chunk stored, encoded at different rates
• manifest file: provides URLs for different chunks
 client:
• periodically measures server-to-client bandwidth
• consulting manifest, requests one chunk at a time
• chooses max coding rate sustainable given
current bandwidth
• can choose different coding rates at different
points in time (depending on availableApplication
bandwidth Layer 2-87
2.6.2 Streaming multimedia:
DASH
 DASH: Dynamic, Adaptive Streaming over
HTTP
 “intelligence” at client: client determines
• when to request chunk (so that buffer
starvation, or overflow does not occur)
• what encoding rate to request (higher quality
when more bandwidth available)
• where to request chunk (can request from URL
server that is “close” to client or has high
available bandwidth)

Application Layer 2-88

2.6.3 Content distribution
networks
 challenge: how to stream content (selected
from millions of videos) to hundreds of
thousands of simultaneous users?

 option 1: single, large “mega-server”

• single point of failure
• point of network congestion
• long path to distant clients
• multiple copies of video sent over outgoing link

….quite simply: this solution doesn’t scale

Application Layer 2-89

2.6.3 Content distribution
networks
 challenge: how to stream content (selected
from millions of videos) to hundreds of
thousands of simultaneous users?

 option 2: store/serve multiple copies of

videos at multiple geographically
distributed sites (CDN)
• enter deep: push CDN servers deep into many
access networks
• close to users
• used by Akamai, 1700 locations
• bring home: smaller number (10’s) of larger
clusters in POPs near (but not within) access
networks
• used by Limelight Application Layer 2-90
 Content Distribution Networks
(CDNs)
 CDN: stores copies of content at CDN
nodes
 subscriber
• e.g. Netflixrequests
stores copies of MadMen
content from CDN
• directed to nearby copy, retrieves content
• may choose different copy if network path
congested

… …

…
manifest file
…

where’s Madmen?

… …
Application Layer 2-91
Content Distribution Networks
(CDNs)

“over the top”

… …
Internet host-host communication as a service

…
OTT challenges: coping with a congested
…

Internet
 from which CDN node to retrieve content?
 viewer behavior in presence of congestion?
… …
 what content to place in which CDN node?
more .. in chapter 7
 CDN content access: a closer look
Bob (client) requests video http://netcinema.com/6Y7B23V
 video stored in CDN at http://KingCDN.com/NetC6y&B23V

1. Bob gets URL for video

http://netcinema.com/6Y7B23V
from netcinema.com web page 2. resolve http://netcinema.com/6Y7B23V
2 via Bob’s local DNS
1
6. request video from 5 Bob’s
KINGCDN server, local DNS
streamed via HTTP server
3. netcinema’s DNS returns URL 4&5. Resolve
netcinema.com 4 http://KingCDN.com/NetC6y&B23
http://KingCDN.com/NetC6y&B23V
via KingCDN’s authoritative DNS,
3 which returns IP address of KingCDN
server with video
netcinema’s
authoratative DNS KingCDN.com KingCDN
authoritative DNS Application Layer 2-93
 Case study: Netflix
Amazon cloud upload copies of
multiple versions of
video to CDN servers
CDN
server
Netflix registration,
accounting servers
3. Manifest file
2. Bob browses returned for
CDN
Netflix video 2 requested video server
3
1

1. Bob manages
Netflix account CDN
server

4. DASH streaming

Application Layer 2-94

Chapter 2: outline
2.1 principles of 2.5 P2P applications
network 2.6 video streaming
applications and content
2.2 Web and HTTP distribution
2.3 electronic mail networks
• SMTP, POP3, IMAP 2.7 socket
2.4 DNS programming with
UDP and TCP

Application Layer 2-95

Socket programming
goal: learn how to build client/server
applications that communicate using
sockets
socket: door between application process
and end-end-transport protocol
application application
socket controlled by
process process app developer

transport transport
network network controlled
link
by OS
link Internet
physical physical

Application Layer 2-96

 Socket programming
Two socket types for two transport
services:
• UDP: unreliable datagram
• TCP: reliable, byte stream-oriented
Application Example:
1. client reads a line of characters (data)
from its keyboard and sends data to
server
2. server receives the data and converts
characters to uppercase
3. server sends modified data to client
4. client receives modified data and
displays line on its screen Application Layer 2-97
Socket programming with
UDP
UDP: no “connection” between client & server
 no handshaking before sending data
 sender explicitly attaches IP destination address
and port # to each packet
 receiver extracts sender IP address and port# from
received packet
UDP: transmitted data may be lost or received
out-of-order
Application viewpoint:
 UDP provides unreliable transfer of groups of bytes
(“datagrams”) between client and server

Application Layer 2-98

Client/server socket interaction:
UDP
server (running on serverIP) client
create socket:
create socket, port= x: clientSocket =
serverSocket = socket(AF_INET,SOCK_DGRAM)
socket(AF_INET,SOCK_DGRAM)
Create datagram with server IP and
port=x; send datagram via
read datagram from clientSocket
serverSocket

write reply to
serverSocket read datagram from
specifying clientSocket
client address,
port number close
clientSocket

Application 2-99
 Example app: UDP client
Python UDPClient
include Python’s socket
library from socket import *
serverName = ‘hostname’
serverPort = 12000
create UDP socket for clientSocket = socket(AF_INET,
server
SOCK_DGRAM)
get user keyboard
input message = raw_input(’Input lowercase sentence:’)
Attach server name, port to clientSocket.sendto(message.encode(),
message; send into socket
(serverName, serverPort))
read reply characters from modifiedMessage, serverAddress =
socket into string
clientSocket.recvfrom(2048)
print out received string print modifiedMessage.decode()
and close socket
clientSocket.close()
Application Layer 2-100
 Example app: UDP server
Python UDPServer
from socket import *
serverPort = 12000
create UDP socket serverSocket = socket(AF_INET, SOCK_DGRAM)
bind socket to local port
number 12000
serverSocket.bind(('', serverPort))
print (“The server is ready to receive”)
loop forever while True:
Read from UDP socket into message, clientAddress = serverSocket.recvfrom(2048)
message, getting client’s
address (client IP and port) modifiedMessage = message.decode().upper()
send upper case string serverSocket.sendto(modifiedMessage.encode(),
back to this client clientAddress)

Application Layer 2-101

Socket programming with
TCP
client must contact  when contacted by
server client, server TCP
 server process must creates new socket for
first be running server process to
 server must have communicate with that
created socket (door) particular client
that welcomes client’s • allows server to talk
contact with multiple clients
• source port numbers
client contacts server used to distinguish
by: clients (more in Chap
 Creating TCP socket, application
3) viewpoint:
specifying IP address, TCP provides reliable, in-order
port number of server byte-stream transfer (“pipe”)
process between client and server
 when client creates
socket: client TCP
establishes connection Application Layer 2-102
Client/server socket interaction:
TCP
server (running on hostid) client
create socket,
port=x, for incoming
request:
serverSocket = socket()

wait for incoming create socket,

connection request
TCP connect to hostid, port=x
connectionSocket = connection setup clientSocket = socket()
serverSocket.accept()

send request using

read request from clientSocket
connectionSocket

write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket

Application Layer 2-103

 Example app: TCP client
Python TCPClient
from socket import *
serverName = ’servername’
create TCP socket for serverPort = 12000
server, remote port 12000
clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect((serverName,serverPort))
sentence = raw_input(‘Input lowercase sentence:’)
No need to attach server clientSocket.send(sentence.encode())
name, port
modifiedSentence = clientSocket.recv(1024)
print (‘From Server:’, modifiedSentence.decode())
clientSocket.close()

Application Layer 2-104

 Example app: TCP server
Python TCPServer
from socket import *
create TCP welcoming
serverPort = 12000
socket serverSocket = socket(AF_INET,SOCK_STREAM)
serverSocket.bind((‘’,serverPort))
server begins listening for
incoming TCP requests
serverSocket.listen(1)
print ‘The server is ready to receive’
loop forever while True:
server waits on accept() connectionSocket, addr = serverSocket.accept()
for incoming requests, new
socket created on return
sentence = connectionSocket.recv(1024).decode()
read bytes from socket (but
not address as in UDP) capitalizedSentence = sentence.upper()
close connection to this
connectionSocket.send(capitalizedSentence.
client (but not welcoming encode())
socket)
connectionSocket.close()
Application Layer 2-105
 Chapter 2:
summary
our study of network apps now
complete!
 application architectures  specific protocols:
• client-server • HTTP
• P2P • SMTP, POP, IMAP
 application service
requirements: • DNS
• reliability, bandwidth, • P2P: BitTorrent
delay  video streaming,
 Internet transport service CDNs
model  socket programming:
• connection-oriented,
reliable: TCP TCP, UDP sockets
• unreliable, datagrams:
UDP
Application Layer 2-106
Chapter 2:
summary
most importantly: learned about
protocols!
 typical request/reply important themes:
message exchange:  control vs. messages
• client requests info
or service • in-band, out-of-band
• server responds  centralized vs.
with data, status decentralized
code  stateless vs. stateful
 message formats:
 reliable vs. unreliable
• headers: fields
giving info about message transfer
data  “complexity at network
• data: info(payload) edge”
being
communicated Application Layer 2-107