Lecture 2

Application Layer

Computer Networks
The slides are made by J.F Kurose and K.W. Ross,
adapted by Phuong Vo and Tan Le

Instructor: Le Duy Tan, Ph.D

Email: [email protected]
Application Layer 2-1
Lecture 2: application layer
our goals:  learn about protocols by
 conceptual, examining popular
implementation aspects application-level
of network application protocols
protocols  HTTP
 transport-layer  SMTP / POP3 / IMAP
service models  DNS
 client-server
paradigm

Application Layer 2-2

Lecture 2: outline
2.1 Principles of network applications
2.2 Web and HTTP
2.3 electronic mail
 SMTP, POP3, IMAP
2.4 DNS
2.5 Socket programming

Application Layer 2-3

Some network apps
 e-mail  voice over IP (e.g., Skype)
 web  real-time video
 text messaging conferencing
 remote login  social networking
 P2P file sharing  search
 multi-user network games  …
 streaming stored video  …
(YouTube, Hulu, Netflix)

Application Layer 2-4

Creating a network app application
transport
network
data link

write programs that: physical

 run on (different) end systems

 communicate over network
 e.g., web server software
communicates with browser
software
no need to write software for application
network-core devices transport
network
 network-core devices do not data link application
transport
physical

run user applications network

data link
physical
 applications on end systems
allows for rapid app
development, propagation

Application Layer 2-5

Application architectures
possible structure of applications:
 client-server
 peer-to-peer (P2P)

Application Layer 2-6

 Client-server architecture
server:
 always-on host
 permanent IP address
 data centers for scaling

clients:
 communicate with server
client/server
 may be intermittently
connected
 may have dynamic IP
addresses
 do not communicate directly
with each other

Application Layer 2-7

P2P architecture
 no always-on server peer-peer
 arbitrary end systems
directly communicate
 peers request service from
other peers, provide service
in return to other peers
 self scalability – new
peers bring new service
capacity, as well as new
service demands
 peers are intermittently
connected and change IP
addresses
 complex management

Application Layer 2-8

Processes communicating
process: program running clients, servers
within a host client process: process that
 within same host, two initiates communication
processes communicate server process: process
using inter-process that waits to be contacted
communication (defined by
OS)
 processes in different hosts
communicate by
exchanging messages

Application Layer 2-9

Sockets
 process sends/receives messages to/from its socket
 socket analogous to door
 sending process shoves message out door
 sending process relies on transport infrastructure on other
side of door to deliver message to socket at receiving
process

application application
socket controlled by
process process app developer

transport transport
network network controlled
link
by OS
link Internet
physical physical

Application Layer 2-10

Addressing processes
 to receive messages, process  identifier includes both IP
must have identifier address and port numbers
 host device has unique 32- associated with process on
bit IP address host.
 Q: is IP address of host  example port numbers:
associated with one  HTTP server: 80
process?  mail server: 25
 to send HTTP message to
 A: no, many processes gaia.cs.umass.edu web
can be running on same server:
host  IP address: 128.119.245.12
 port number: 80

Application Layer 2-11

What transport service does an app need?
data integrity throughput
 some apps (e.g., file transfer,  some apps (e.g.,
web transactions) require multimedia) require
100% reliable data transfer minimum amount of
 other apps (e.g., audio) can throughput to be
tolerate some loss “effective”
 other apps (“elastic apps”)
make use of whatever
timing
throughput they get
 some apps (e.g., Internet
telephony, interactive security
games) require low delay
 encryption, data integrity,
to be “effective”
…

Application Layer 2-12

 Transport service requirements: common apps

application data loss throughput time sensitive

file transfer no loss elastic no

e-mail no loss elastic no
Web documents no loss elastic no
real-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec
video:10kbps-5Mbps
stored audio/video loss-tolerant same as above yes, few secs
interactive games loss-tolerant few kbps up yes, 100’s msec
text messaging no loss elastic yes and no

Application Layer 2-13

Internet transport protocols services

TCP service: UDP service:

 reliable transport between  unreliable data transfer
sending and receiving between sending and
process receiving process
 flow control: sender won’t  does not provide:
overwhelm receiver
reliability, flow control,
 congestion control: throttle congestion control,
sender when network
overloaded timing, throughput
 does not provide: timing, guarantee, security, or
minimum throughput connection setup,
guarantee, security
 connection-oriented: setup Q: why bother? Why is
required between client and there a UDP?
server processes

Application Layer 2-14

Internet apps: application, transport protocols

application underlying
application layer protocol transport protocol

e-mail SMTP [RFC 2821] TCP

remote terminal access Telnet [RFC 854] TCP
Web HTTP [RFC 2616] TCP
file transfer FTP [RFC 959] TCP
streaming multimedia HTTP (e.g., YouTube), TCP or UDP
RTP [RFC 1889]
Internet telephony SIP, RTP, proprietary
(e.g., Skype) TCP or UDP

Application Layer 2-15

Securing TCP

TCP & UDP SSL is at app layer

 no encryption  apps use SSL libraries,
 cleartext passwds sent into that “talk” to TCP
socket traverse Internet in SSL socket API
cleartext  cleartext passwords sent
SSL into socket traverse
 provides encrypted TCP Internet encrypted
connection  see Chapter 8
 data integrity
 end-point authentication

Application Layer 2-16

App-layer protocol defines
 types of messages open protocols:
exchanged,  defined in RFCs
 e.g., request, response  allows for interoperability
 message syntax:  e.g., HTTP, SMTP
 what fields in messages proprietary protocols:
& how fields are defined  e.g., Skype
 message semantics
 meaning of information
in fields
 rules for when and how
processes send & respond
to messages

Application Layer 2-17

Lecture 2: outline
2.1 principles of network applications
 app architectures
 app requirements
2.2 Web and HTTP
2.3 electronic mail
 SMTP, POP3, IMAP
2.4 DNS
2.5 Socket programming

Application Layer 2-18

Web and HTTP
First, a review…
 web page consists of objects
 object can be HTML file, JPEG image, Java
applet, audio file,…
 web page consists of base HTML-file which
includes several referenced objects
 each object is addressable by a URL, e.g.,
www.someschool.edu/someDept/pic.gif

host name path name

Application Layer 2-19

HTTP overview
HTTP: hypertext transfer
protocol HT
 Web’s application layer TP
req
protocol PC running ues
HT t
Firefox browser TPr
 client/server model esp
ons
 client: browser that e
requests, receives, t
(using HTTP protocol) u es
req server
and “displays” Web T P nse
HT po running
objects P res
Apache Web
T
 server: Web server HT server
sends (using HTTP
protocol) objects in iphone running
response to requests Safari browser

Application Layer 2-20

HTTP overview (continued)
uses TCP:
 client initiates TCP connection (creates socket) to server,
port 80
 server accepts TCP connection from client
 HTTP messages (application-layer protocol messages)
exchanged between browser (HTTP client) and Web server
(HTTP server)
 TCP connection closed

Application Layer 2-21

HTTP connections
non-persistent HTTP persistent HTTP
 at most one object sent over
 multiple
TCP connection
objects can
 connection then closed be sent over single
 downloading multiple objectsTCP connection
required multiple
connections between client, server

Application Layer 2-22

 Non-persistent HTTP
suppose user enters URL: (contains text,
www.someSchool.edu/someDepartment/home.index references to 10
jpeg images)
1a. HTTP client initiates TCP
connection to HTTP server
(process) at 1b. HTTP server at host
www.someSchool.edu on port 80 www.someSchool.edu waiting
for TCP connection at port 80.
“accepts” connection, notifying
2. HTTP client sends HTTP client
request message (containing
URL) into TCP connection 3. HTTP server receives request
socket. Message indicates that message, forms response
client wants object message containing requested
someDepartment/home.index object, and sends message into
its socket
time
Application Layer 2-23
 Non-persistent HTTP (cont.)
4. HTTP server closes TCP
connection.
5. HTTP client receives response
message containing html file,
displays html. Parsing html file,
finds 10 referenced jpeg objects

time
6. Steps 1-5 repeated for each of 10
jpeg objects

Application Layer 2-24

Non-persistent HTTP: response time
RTT (Round-Trip Time): time
for a small packet to travel
from client to server and
back initiate TCP
HTTP response time: connection
 one RTT to initiate TCP RTT
connection request
file
 one RTT for HTTP request
time to
RTT
and first few bytes of HTTP transmit
file
response to return file
 file transmission time received
 non-persistent HTTP
response time = time time
2RTT+ file transmission
time
Application Layer 2-25
Persistent HTTP

non-persistent HTTP persistent HTTP:

issues:  server leaves connection
 requires 2 RTTs per object open after sending
 OS overhead for each TCP response
connection  subsequent HTTP
 browsers often open messages between same
parallel TCP connections to client/server sent over
fetch referenced objects open connection
 client sends requests as
soon as it encounters a
referenced object
 as little as one RTT for all
the referenced objects

2-26
Example 1 (Problem 8)
Suppose that the Web page associated with the link
contains exactly one object, consisting of a small
amount of HTML text which references to eight very
small objects on the same server. Let RTT0 denote
the RTT between the local host and server.
Neglecting transmission times, how much time
elapses with
a. Non-persistent HTTP with no parallel TCP
connections?
b. Non-persistent HTTP with the browser configured
for 5 parallel connections?
c. Persistent HTTP?
Application Layer 2-27
Example 1 - answer
a) 2RTT0 + 8*2RTT0
= 18RTT0
b) 2RTT0 + 2* (RTT0 + RTT0)
= 6RTT0
c) Persistent connection with pipelining. This is the
default mode of HTTP
2RTT0 + RTT0 = 3RTT0
Persistent connection without pipelining, without
parallel connections.
2RTT0 + 8RTT0 = 10RTT0
Application Layer 2-28
Example 2 (Problem 10)
Consider a short, 10-meter link, over which a sender can
transmit at a rate of 150 bits/sec in both directions.
Suppose that packets containing data (HTML and 10
objects) are 100,000 bits long each, and packets containing
only control (e.g., ACK or handshaking) are 200 bits long.

1) How much time elapses with non-persistent HTTP and parallel

downloads? Draw the figure before the calculation.

2) How about persistent HTTP. Do you expect significant gains over

the non-persistent case? Draw the figure before the calculation.

Application Layer 2-29

Example 2 - answer
1) Non-persistent with parallel download:
(200/150 + Tp + 200/150 + Tp +
200/150+Tp + 100,000/150+ Tp )
+
(200/(150/10)+Tp + 200/(150/10) + Tp +
200/(150/10)+Tp + 100,000/(150/10) + Tp )
= 7377 + 8 * Tp (seconds)

2)Persistent HTTP:
(200/150+Tp + 200/150 + Tp + 200/150 +
Tp + 100,000/150 + Tp )
+
10 * (200/(150/10) + Tp +
100,000/(150/10)+ Tp )

=7351 + 24 * Tp (seconds)

Application Layer 2-30

 HTTP request message
 two types of HTTP messages: request, response
 HTTP request message:
 ASCII (human-readable format)
carriage return character
line-feed character
request line
(GET, POST, GET /index.html HTTP/1.1\r\n
HEAD commands) Host: www-net.cs.umass.edu\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
headerAccept-Language: en-us,en;q=0.5\r\n
linesAccept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n
carriage return, Keep-Alive: 115\r\n
line feed at start Connection: keep-alive\r\n
\r\n
of line indicates
end of header lines
Application Layer 2-31
Uploading form input
POST method:
 web page often includes
form input
 input is uploaded to server
in entity body

URL method:
 uses GET method
 input is uploaded in URL
field of request line:
https://google.com/search?q=monkeys+and+banana

Application Layer 2-32

Method types
HTTP/1.0: HTTP/1.1:
 GET  GET, POST, HEAD
 POST  PUT
 HEAD  uploads file in entity
 asks server to leave body to path specified
requested object out of in URL field
response  DELETE
 deletes file specified in
the URL field

Application Layer 2-33

 HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK\r\n
status phrase) Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n
Server: Apache/2.0.52 (CentOS)\r\n
Last-Modified: Tue, 30 Oct 2007 17:00:02
GMT\r\n
header ETag: "17dc6-a5c-bf716880"\r\n
Accept-Ranges: bytes\r\n
lines Content-Length: 2652\r\n
Keep-Alive: timeout=10, max=100\r\n
Connection: Keep-Alive\r\n
Content-Type: text/html; charset=ISO-8859-1\
r\n
\r\n
data, e.g., data data data data data ...
requested
HTML file
Application Layer 2-34
HTTP response status codes
 status code appears in 1st line in server-to-
client response message.
 some sample codes:
200 OK
 request succeeded, requested object later in this msg
301 Moved Permanently
 requested object moved, new location specified later in this msg
(Location: header of the response message. The client software will
automatically retrieve the new URL)
400 Bad Request
 request msg not understood by server
404 Not Found
 requested document not found on this server
505 HTTP Version Not Supported
Application Layer 2-35
Quiz - Problems 4&5
 Answer the questions in Problems 4 and 5, pages
171 - 172.

Application Layer 2-36

User-server state: cookies
example:
many Web sites use cookies  Susan always access
four components: Internet from PC
1) cookie header line of  visits specific e-commerce
HTTP response site for first time
message  when initial HTTP requests
2) cookie header line in arrives at site, site creates:
next HTTP request  unique ID
message  entry in backend
3) cookie file kept on database for ID
user’s host, managed
by user’s browser
4) back-end database at
Web site
Application Layer 2-37
 Cookies: keeping “state” (cont.)
client server

Shopee 8734
usual http request msg Tiki server
cookie file creates ID
usual http response
1678 for user create backend
Shopee 8734
set-cookie: 1678 entry database
Tiki 1678
usual http request msg
cookie: 1678 cookie- access
specific
usual http response msg action

one week later:

access
Shopee 8734 usual http request msg
Tiki 1678 cookie: 1678 cookie-
specific
usual http response msg action
Application Layer 2-38
Cookies (continued)
aside
what cookies can be used cookies and privacy:
for:  cookies permit sites to
 authorization learn a lot about you
 shopping carts  you may supply name and
 recommendations
e-mail to sites
 user session state (Web e-
mail)

how to keep “state”:

 protocol endpoints: maintain state at
sender/receiver over multiple transactions
 cookies: http messages carry state

Application Layer 2-39

Web caches (proxy server)
goal: satisfy client request without involving origin server
 user sets browser: Web
accesses via cache
 browser sends all HTTP proxy
requests to cache HT st
TP u e
req server req
 object in cache: cache HT ues P se
client TP t H TT po n
res res origin
returns object pon
se HT
T P server
t
 else cache requests ues
req e
object from origin TT P o ns
p
H res
server, then returns T TP
H
object to client
client origin
server

Application Layer 2-40

More about Web caching
 cache acts as both why Web caching?
client and server  reduce response time for
 server for original client request
requesting client
 client to origin server  reduce traffic on an
 typically cache is institution’s access link
installed by ISP  Internet dense with
(university, company, caches: enables “poor”
residential ISP) content providers to
effectively deliver
content (so too does P2P
file sharing)

Application Layer 2-41

Caching example:
assumptions:
 avg object size (L): 100K bits
origin
 avg request rate from browsers to origin
servers
servers (a):15 requests/sec public
 avg data rate to browsers (R): 1.50 Mbps Internet
 RTT from institutional router to any
origin server: 2 sec
 access link rate: 1.54 Mbps
problem: large 1.54 Mbps
consequences: queueing delays access link
 LAN utilization: 0.0015at high utilization institutional
 access link utilization = 97% network
1 Gbps LAN
 total delay = Internet delay + access
delay + LAN delay
= 2 sec + minutes + usecs
traffic intensity = La/R
L: packet length (bits)
a: average packet arrival rate. R: link bandwidth (bps)
Application Layer 2-42
 Caching example: fatter access link
assumptions:
 avg object size: 100K bits origin
 avg request rate from browsers to servers
origin servers:15/sec public
 avg data rate to browsers: 1.50 Mbps Internet
 RTT from institutional router to any
origin server: 2 sec
 access link rate: 1.54 Mbps
154 1.54 Mbps
154 Mbps
consequences: Mbps access link
 LAN utilization: 0.0015 institutional
 access link utilization = 99% network
9.9% 1 Gbps LAN
 total delay = Internet delay + access
delay + LAN delay
= 2 sec + minutes + usecs
msecs

Cost: increased access link speed (not cheap!)

Application Layer 2-43
 Caching example: install local cache
assumptions:
 avg object size: 100K bits origin
 avg request rate from browsers to servers
origin servers:15/sec public
 avg data rate to browsers: 1.50 Mbps Internet
 RTT from institutional router to any
origin server: 2 sec
 access link rate: 1.54 Mbps
1.54 Mbps
consequences: access link
 LAN utilization: 0.0015 institutional
 access link utilization = 100% network
? 1 Gbps LAN
 total delay = Internet delay + access
?
delay + LAN delay local web
= 2 How to compute
sec + minutes + usecs link cache
utilization, delay?
Cost: web cache (cheap!)
Application Layer 2-44
Caching example: install local cache
Calculating access link
utilization, delay with
origin
cache: servers
 suppose cache hit rate is 0.4 public
 40% requests satisfied at cache, Internet
60% requests satisfied at origin
 access link utilization:
 60% of requests use access link
 data rate to browsers over access link 1.54 Mbps
access link
= 0.6*1.50 Mbps = .9 Mbps
 utilization = 0.9/1.54 = .58 institutional
or La/R = (15 * 60%) * 0.1 /1.54 = 0.58 network
1 Gbps LAN
 total delay local web
 = 0.6 * (delay from origin servers) +0.4 cache
* (delay when satisfied at cache)
 = 0.6 (2.01) + 0.4 (~msecs) = ~ 1.2 secs
 less than with 154 Mbps link (and
cheaper too!) Application Layer 2-45
Conditional GET
client server
 Goal: don’t send object if
cache has up-to-date
cached version HTTP request msg
object
If-modified-since: <date>
 no object transmission not
delay modified
 lower link utilization HTTP response
before
HTTP/1.0
 cache: specify date of 304 Not Modified <date>
cached copy in HTTP
request
If-modified-since:
<date> HTTP request msg
 server: response contains If-modified-since: <date> object
no object if cached copy modified
HTTP response after
is up-to-date:
HTTP/1.0 200 OK <date>
HTTP/1.0 304 Not
Modified <data>
Application Layer 2-46
Lecture 2: outline
2.1 principles of network applications
 app architectures
 app requirements
2.2 Web and HTTP
2.3 electronic mail
 SMTP, POP3, IMAP
2.4 DNS
2.5 Socket programming

Application Layer 2-47

Electronic mail outgoing
message queue
user mailbox
Three major components: user
agent
 user agents
 mail servers mail user
server agent
 simple mail transfer
protocol: SMTP SMTP mail user
server agent
User Agent SMTP
 a.k.a. “mail reader” SMTP user
 composing, editing, mail
agent
reading mail messages server
user
 e.g., Gmail, Outlook, agent
Thunderbird, iPhone mail user
client agent
 outgoing, incoming
messages stored on server Application Layer 2-48
Electronic mail: mail servers
mail servers: user
agent
 mailbox contains incoming
messages for user mail user
server
 message queue of outgoing agent
(to be sent) mail messages SMTP mail user
 SMTP protocol between server agent
mail servers to send email SMTP
messages
SMTP user
 client: sending mail agent
mail
server server
 “server”: receiving mail user
agent
server
user
agent

Application Layer 2-49

Electronic Mail: SMTP [RFC 2821]
 uses TCP to reliably transfer email message from
client to server, port 25
 direct transfer: sending server to receiving server
 three phases of transfer
 handshaking (greeting)
 transfer of messages
 closure
 command/response interaction (like HTTP, FTP)
 commands: ASCII text
 response: status code and phrase
 messages must be in 7-bit ASCII

Application Layer 2-50

Scenario: Alice sends message to Bob
1) Alice uses UA to compose 4) SMTP client sends Alice’s
message “to” message over the TCP
[email protected] connection
2) Alice’s UA sends message to 5) Bob’s mail server places the
her mail server; message message in Bob’s mailbox
placed in message queue 6) Bob invokes his user agent to
3) client side of SMTP opens read message
TCP connection with Bob’s
mail server

1 user mail user

mail agent
agent server server
2 3 6
4
5
Alice’s mail server Bob’s mail server
Application Layer 2-51
Sample SMTP interaction
C: telnet hamburger.edu 25
S: 220 hamburger.edu
C: HELO crepes.fr
S: 250 Hello crepes.fr, pleased to meet you
C: MAIL FROM: <[email protected]>
S: 250 [email protected]... Sender ok
C: RCPT TO: <[email protected]>
S: 250 [email protected] ... Recipient ok
C: DATA
S: 354 Enter mail, end with "." on a line by itself
C: Do you like ketchup?
C: How about pickles?
C: .
S: 250 Message accepted for delivery
C: QUIT
S: 221 hamburger.edu closing connection

Application Layer 2-52

SMTP: final words
 SMTP uses persistent comparison with HTTP:
connections  HTTP: pull
 SMTP requires message
(header & body) to be in
 SMTP: push
7-bit ASCII  both have ASCII
command/response
interaction, status codes
 HTTP: each object
encapsulated in its own
response msg
 SMTP: multiple objects
sent in multipart msg

Application Layer 2-53

Mail message format
SMTP: protocol for
exchanging email msgs header
blank
RFC 822: standard for text line
message format:
 header lines, e.g.,
 To: body
 From:
 Subject:
 Body: the “message”
 ASCII characters only

Application Layer 2-54

Mail access protocols
user
mail user
SMTP SMTP access
agent agent
protocol
(e.g., POP,
IMAP)

sender’s mail receiver’s mail

server server

 SMTP: delivery/storage to receiver’s server

 mail access protocol: retrieval from server
 POP: Post Office Protocol [RFC 1939]: authorization,
download
 IMAP: Internet Mail Access Protocol [RFC 1730]: more
features, including manipulation of stored msgs on server
 HTTP: gmail, Hotmail, Yahoo! Mail, etc.

Application Layer 2-55

POP3 protocol
S: +OK POP3 server ready
C: user bob
authorization phase S: +OK
C: pass hungry
 client commands:
S: +OK user successfully logged on
 user: declare username
 pass: password C: list
 server responses S: 1 498
 +OK S: 2 912
S: .
 -ERR
C: retr 1
transaction phase, client: S: <message 1 contents>
 list: list message numbers S: .
 retr: retrieve message by C: dele 1
number C: retr 2
S: <message 1 contents>
 dele: delete
S: .
 quit C: dele 2
C: quit
S: +OK POP3 server signing off
Application Layer 2-56
POP3 (more) and IMAP
more about POP3 IMAP
 previous example uses  keeps all messages in one
POP3 “download and place: at server
delete” mode  allows user to organize
 Bob cannot re-read e- messages in folders
mail if he changes  keeps user state across
client sessions:
 POP3 “download-and-  names of folders and
keep”: copies of messages mappings between
on different clients message IDs and folder
 POP3 is stateless across name
sessions

Application Layer 2-57

Lecture 2: outline
2.1 principles of network applications
 app architectures
 app requirements
2.2 Web and HTTP
2.3 electronic mail
 SMTP, POP3, IMAP
2.4 DNS
2.5 Socket programming

Application Layer 2-58

DNS: domain name system
people: many identifiers: Domain Name System:
 SSN, name, passport #  distributed database
Internet hosts, routers: implemented in hierarchy of
 IP address (32 bit) - many name servers
used for addressing  application-layer protocol:
datagrams hosts, name servers
 “name”, e.g., communicate to resolve
www.yahoo.com - used names (address/name
by humans translation)
Q: how to map between IP  note: core Internet function,
address and name, and implemented as application-
vice versa ? layer protocol
 complexity at network’s
“edge”

Application Layer 2-59

DNS: services
 hostname to IP address translation
 host aliasing
 canonical, alias names
 mail server aliasing
 load distribution

 replicated Web servers: many IP

addresses correspond to one name

Application Layer 2-60

DNS: a distributed, hierarchical database
Root DNS Servers

… …

com DNS servers org DNS servers edu DNS servers

pbs.org poly.edu umass.edu

yahoo.com amazon.com
DNS servers DNS serversDNS servers
DNS servers DNS servers

why not centralize DNS?

 single point of failure
 traffic volume
 distant centralized database
 maintenance

Application Layer 2-61

 DNS: root name servers
 contacted by local name server that can not resolve name
 root name server:
 contacts authoritative name server if name mapping not known
 gets mapping
 returns mapping to local name server

c. Cogent, Herndon, VA (5 other sites)

d. U Maryland College Park, MD k. RIPE London (17 other sites)
h. ARL Aberdeen, MD
j. Verisign, Dulles VA (69 other sites ) i. Netnod, Stockholm (37 other sites)

e. NASA Mt View, CA m. WIDE Tokyo

f. Internet Software C. (5 other sites)
Palo Alto, CA (and 48 other
sites)

a. Verisign, Los Angeles CA 13 root name

(5 other sites)
b. USC-ISI Marina del Rey, CA
“servers”
l. ICANN Los Angeles, CA worldwide
(41 other sites)
g. US DoD Columbus,
OH (5 other sites)

Application Layer 2-62

TLD, authoritative servers
top-level domain (TLD) servers:
 responsible for com, org, net, edu, aero, jobs, museums,
and all top-level country domains, e.g.: uk, fr, ca, jp
 Network Solutions maintains servers for .com TLD
 Educause for .edu TLD
authoritative DNS servers:
 organization’s own DNS server(s), providing authoritative
hostname to IP mappings for organization’s named hosts
 can be maintained by organization or service provider

Application Layer 2-63

Local DNS name server
 does not strictly belong to hierarchy
 each ISP (residential ISP, company, university) has
one
 also called “default name server”
 when host makes DNS query, query is sent to its
local DNS server
 has local cache of recent name-to-address translation
pairs (but may be out of date!)
 acts as proxy, forwards query into hierarchy

Application Layer 2-64

DNS name root DNS server
resolution example
2
 host at cis.poly.edu 3
TLD DNS server
wants IP address for 4
gaia.cs.umass.edu
5

local DNS server

iterated query: dns.poly.edu
 contacted server replies 7 6
1 8
with name of server to
contact
authoritative DNS server
 “I don’t know this dns.cs.umass.edu
name, but ask this requesting host
server” cis.poly.edu

gaia.cs.umass.edu

Application Layer 2-65

DNS name root DNS server
resolution example
2 3
recursive query: 7
6
 puts burden of name TLD DNS
server
resolution on
contacted name local DNS server
server dns.poly.edu 5 4

 heavy load at upper 1 8

levels of hierarchy?
authoritative DNS server
dns.cs.umass.edu
requesting host
cis.poly.edu

gaia.cs.umass.edu

Application Layer 2-66

Problem 7
Suppose within your Web browser you click on a link to
obtain a Web page. The IP address for the associated URL is
not cached in your local host, so a DNS lookup is necessary
to obtain the IP address. Suppose that n DNS servers are
visited before your host receives the IP address from DNS;
the successive visits incur an RTT of RTT1,…, RTTn. Further
suppose that the Web page associated with the link contains
exactly one object, consisting of a small amount of HTML
text. Let RTT0 denote the RTT between the local host and the
server containing the object. Assuming zero transmission
time of the object, how much time elapses from when the
client clicks on the link until the client receives the object?

Application Layer 2-67

Problem 7 - answer
 The total amount of time to get the IP address is
RTT1 + RTT2 + … + RTTn
 Once the IP address is known, elapses to set up the
TCP connection and another RTT0 elapses to
request and receive the small object. The total
response time is
2RTT0 + RTT1 + RTT2 + ... + RTTn

Application Layer 2-68

DNS: caching, updating records
 once (any) name server learns mapping, it caches
mapping
 cache entries timeout (disappear) after some time (TTL)
 TLD servers typically cached in local name servers
• thus root name servers not often visited
 cached entries may be out-of-date (best effort
name-to-address translation!)
 if name host changes IP address, may not be known
Internet-wide until all TTLs expire
 update/notify mechanisms proposed IETF standard
 RFC 2136

Application Layer 2-69

DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)

type=A type=CNAME
 name is hostname  name is alias name for some
 value is IP address “canonical” (the real) name
 www.ibm.com is really
type=NS
 name is domain (e.g., servereast.backup2.ibm.com
foo.com)  value is canonical name
 value is hostname of
authoritative name type=MX
server for this domain
 value is name of mailserver
associated with name

Application Layer 2-70

DNS protocol, messages
 query and reply messages, both with same message
format 2 bytes 2 bytes

msg header identification flags

 identification: 16 bit # for # questions # answer RRs

query, reply to query uses
# authority RRs # additional RRs
same #
 flags: questions (variable # of questions)
 query or reply
 recursion desired answers (variable # of RRs)
 recursion available
 reply is authoritative authority (variable # of RRs)

additional info (variable # of RRs)

Application Layer 2-71

DNS protocol, messages

2 bytes 2 bytes

identification flags

# questions # answer RRs

# authority RRs # additional RRs

name, type fields

questions (variable # of questions)
for a query
RRs in response answers (variable # of RRs)
to query
records for authority (variable # of RRs)
authoritative servers
additional “helpful” additional info (variable # of RRs)
info that may be used
Application Layer 2-72
Inserting records into DNS
 example: new startup “Network Utopia”
 register name networkuptopia.com at DNS registrar
(e.g., Network Solutions)
 provide names, IP addresses of authoritative name server
(primary and secondary)
 registrar inserts two RRs into .com TLD server:
(networkutopia.com, dns1.networkutopia.com, NS)
(dns1.networkutopia.com, 212.212.212.1, A)
 create authoritative server type A record for
www.networkuptopia.com; type MX record for
networkutopia.com

Application Layer 2-73

Chapter 2: outline
2.1 principles of network applications
2.2 Web and HTTP
2.3 electronic mail
• SMTP, POP3, IMAP
2.4 DNS
2.5 socket programming with UDP and TCP

Application Layer 2-74

Socket programming
goal: learn how to build client/server applications that
communicate using sockets
socket: door between application process and end-end-
transport protocol

application application
socket controlled by
process process app developer

transport transport
network network controlled
link
by OS
link Internet
physical physical

Application Layer 2-75

 Socket programming
Two socket types for two transport services:
 UDP: unreliable datagram
 TCP: reliable, byte stream-oriented

Application Example:
1. client reads a line of characters (data) from its
keyboard and sends data to server
2. server receives the data and converts characters to
uppercase
3. server sends modified data to client
4. client receives modified data and displays line on
its screen
Application Layer 2-76
Socket programming with UDP
UDP: no “connection” between client & server
 no handshaking before sending data
 sender explicitly attaches IP destination address and
port # to each packet
 receiver extracts sender IP address and port# from
received packet
UDP: transmitted data may be lost or received
out-of-order
Application viewpoint:
 UDP provides unreliable transfer of groups of bytes
(“datagrams”) between client and server

Application Layer 2-77

Client/server socket interaction: UDP

server (running on serverIP) client

create socket:
create socket, port= x: clientSocket =
serverSocket = socket(AF_INET,SOCK_DGRAM)
socket(AF_INET,SOCK_DGRAM)
Create datagram with server IP and
port=x; send datagram via
read datagram from clientSocket
serverSocket

write reply to
serverSocket read datagram from
specifying clientSocket
client address,
port number close
clientSocket

Application 2-78
 Example app: UDP client
Python UDPClient
include Python’s socket
library from socket import *
serverName = ‘hostname’
serverPort = 12000
create UDP socket for clientSocket = socket(AF_INET,
server
SOCK_DGRAM)
get user keyboard
input message = raw_input(’Input lowercase sentence:’)
Attach server name, port to clientSocket.sendto(message.encode(),
message; send into socket
(serverName, serverPort))
read reply characters from modifiedMessage, serverAddress =
socket into string
clientSocket.recvfrom(2048)
print out received string print modifiedMessage.decode()
and close socket
clientSocket.close()
Application Layer 2-79
 Example app: UDP server
Python UDPServer
from socket import *
serverPort = 12000
create UDP socket serverSocket = socket(AF_INET, SOCK_DGRAM)
bind socket to local port
number 12000
serverSocket.bind(('', serverPort))
print (“The server is ready to receive”)
loop forever while True:
Read from UDP socket into message, clientAddress = serverSocket.recvfrom(2048)
message, getting client’s
address (client IP and port) modifiedMessage = message.decode().upper()
send upper case string serverSocket.sendto(modifiedMessage.encode(),
back to this client clientAddress)

Application Layer 2-80

Socket programming with TCP
client must contact server  when contacted by client,
 server process must first be server TCP creates new socket
running for server process to
 server must have created communicate with that
socket (door) that welcomes particular client
client’s contact  allows server to talk with
multiple clients
client contacts server by:  source port numbers used
 Creating TCP socket, to distinguish clients
specifying IP address, port (more in Chap 3)
number of server process
 when client creates socket: application viewpoint:
client TCP establishes TCP provides reliable, in-order
connection to server TCP byte-stream transfer (“pipe”)
between client and server

Application Layer 2-81

Client/server socket interaction: TCP
server (running on hostid) client
create socket,
port=x, for incoming
request:
serverSocket = socket()

wait for incoming create socket,

connection request
TCP connect to hostid, port=x
connectionSocket = connection setup clientSocket = socket()
serverSocket.accept()

send request using

read request from clientSocket
connectionSocket

write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket

Application Layer 2-82

 Example app: TCP client
Python TCPClient
from socket import *
serverName = ’servername’
create TCP socket for serverPort = 12000
server, remote port 12000
clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect((serverName,serverPort))
sentence = raw_input(‘Input lowercase sentence:’)
No need to attach server clientSocket.send(sentence.encode())
name, port
modifiedSentence = clientSocket.recv(1024)
print (‘From Server:’, modifiedSentence.decode())
clientSocket.close()

Application Layer 2-83

 Example app: TCP server
Python TCPServer
from socket import *
create TCP welcoming
serverPort = 12000
socket serverSocket = socket(AF_INET,SOCK_STREAM)
serverSocket.bind((‘’,serverPort))
server begins listening for
incoming TCP requests
serverSocket.listen(1)
print ‘The server is ready to receive’
loop forever while True:
server waits on accept() connectionSocket, addr = serverSocket.accept()
for incoming requests, new
socket created on return
sentence = connectionSocket.recv(1024).decode()
read bytes from socket (but
not address as in UDP) capitalizedSentence = sentence.upper()
close connection to this
connectionSocket.send(capitalizedSentence.
client (but not welcoming encode())
socket)
connectionSocket.close()
Application Layer 2-84
Video streaming and CDN

Application Layer 2-85

Video Streaming and CDNs: context
 video traffic: major consumer of Internet bandwidth
• Netflix, YouTube: 37%, 16% of downstream
residential ISP traffic
• ~1B YouTube users, ~75M Netflix users
 challenge: scale - how to reach ~1B
users?
• single mega-video server won’t work (why?)
 challenge: heterogeneity
 different users have different capabilities (e.g.,
wired versus mobile; bandwidth rich versus
bandwidth poor)
 solution: distributed, application-level
infrastructure

Application Layer 2-86

 spatial coding example: instead

Multimedia: video of sending N values of same

color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 video: sequence of images
displayed at constant rate ……………………..
……………….…….
 e.g., 24 images/sec
 digital image: array of
pixels
 each pixel represented
by bits
frame i
 coding: use redundancy
within and between images
to decrease # bits used to
encode image temporal coding example:
 spatial (within image) instead of sending
complete frame at i+1,
 temporal (from one send only differences from
frame i+1
image to next) frame i

Application Layer 2-87

 spatial coding example: instead

Multimedia: video of sending N values of same

color (all purple), send only two
values: color value (purple) and
number of repeated values (N)
 CBR: (constant bit rate):
video encoding rate fixed ……………………..
……………….…….
 VBR: (variable bit rate):
video encoding rate changes
as amount of spatial,
temporal coding changes
 examples:
• MPEG 1 (CD-ROM) 1.5 frame i
Mbps
• MPEG2 (DVD) 3-6 Mbps
• MPEG4 (often used in temporal coding example:
instead of sending
Internet, < 1 Mbps) complete frame at i+1,
send only differences from
frame i frame i+1

Application Layer 2-88

Dynamic Adaptive Streaming
over HTTP (DASH)

Application Layer 2-89

Streaming multimedia: DASH
 DASH: Dynamic, Adaptive Streaming over HTTP
 server:
 divides video file into multiple chunks
 each chunk stored, encoded at different rates
 manifest file: provides URLs for different chunks
 client:
 periodically measures server-to-client bandwidth
 consulting manifest, requests one chunk at a time
• chooses maximum coding rate sustainable given
current bandwidth
• can choose different coding rates at different points
in time (depending on available bandwidth at time)

Application Layer 2-90

 Dynamic Adaptive Streaming
over HTTP (DASH)

O. Oyman and S. Singh. “Quality of experience for HTTP adaptive streaming

services" IEEE Communications Magazine, vol. 50, no. 4 (2012): 20-27. 91
Media Presentation Description
XML file

Application Layer 2-92

Streaming multimedia: DASH
 DASH: Dynamic, Adaptive Streaming over HTTP
 “intelligence” at client: client determines
 when to request chunk (so that buffer starvation, or
overflow does not occur)
 what encoding rate to request (higher quality when
more bandwidth available)
 where to request chunk (can request from URL server
that is “close” to client or has high available
bandwidth)

Application Layer 2-93

Streaming stored video:
simple scenario:

Internet

video server client

(stored video)

Application Layer 2-94

Content distribution networks
 challenge: how to stream content (selected from
millions of videos) to hundreds of thousands of
simultaneous users?

 option 1: single, large “mega-server”

 single point of failure
 point of network congestion
 long path to distant clients
 multiple copies of video sent over outgoing link

….quite simply: this solution doesn’t scale

Application Layer 2-95

Content distribution networks
 challenge: how to stream content (selected from
millions of videos) to hundreds of thousands of
simultaneous users?

 option 2: store/serve multiple copies of videos at

multiple geographically distributed sites (CDN)
 enter deep: push CDN servers deep into many access
networks
• close to users
• used by Akamai, 1700 locations
 bring home: smaller number (10’s) of larger clusters in
POPs near (but not within) access networks
• used by Limelight

Application Layer 2-96

 Content Distribution Networks (CDNs)
 CDN: stores copies of content at CDN nodes
• e.g. Netflix stores copies of MadMen
 subscriber requests content from CDN
• directed to nearby copy, retrieves content
• may choose different copy if network path
congested

… …

…
manifest file
…

where’s Madmen?

… …
Application Layer 2-97
 CDN content access: a closer look
Bob (client) requests video http://netcinema.com/6Y7B23V
 video stored in CDN at http://KingCDN.com/NetC6y&B23V

1. Bob gets URL for video

http://netcinema.com/6Y7B23V
from netcinema.com web page 2. resolve http://netcinema.com/6Y7B23V
2 via Bob’s local DNS
1
6. request video from 5 Bob’s
KINGCDN server, local DNS
streamed via HTTP server
3. netcinema’s DNS returns URL 4&5. Resolve
netcinema.com 4 http://KingCDN.com/NetC6y&B23
http://KingCDN.com/NetC6y&B23V
via KingCDN’s authoritative DNS,
3 which returns IP address of KingCDN
server with video
netcinema’s
authoratative DNS KingCDN.com KingCDN
authoritative DNS Application Layer 2-98
 Case study: Netflix
Amazon cloud upload copies of
multiple versions of
video to CDN servers
CDN
server
Netflix registration,
accounting servers
3. Manifest file
2. Bob browses returned for
CDN
Netflix video 2 requested video server
3
1

1. Bob manages
Netflix account CDN
server

4. DASH streaming

Application Layer 2-99

Lecture 2: summary
our study of network apps now complete!
 application architectures  specific protocols:
 client-server  HTTP
 P2P  SMTP, POP, IMAP
 application service
requirements:  DNS
 reliability, bandwidth,
delay
 Internet transport service
model
 connection-oriented,
reliable: TCP
 unreliable, datagrams:
UDP
Application Layer 2-100
Lecture 2: summary
most importantly: learned about protocols!
 typical request/reply important themes:
message exchange:
 client requests info or
 control vs. data msgs
service  in-band, out-of-band
 server responds with  centralized vs. decentralized
data, status code  stateless vs. stateful
 message formats:  reliable vs. unreliable msg
 headers: fields giving
info about data transfer
 data: info being  “complexity at network
communicated edge”

Application Layer 2-101