Graph Traversal: Dijkstra

Course Code: CSC 2211 Course Title: Algorithms

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: 12 Week No: 12 Semester: Spring 22-23

Lecturer: Md. Faruk Abdullah AL Sohan; [email protected]
 Single source shortest paths
• Done: BFS to find the minimum path length from v to u in
O(|E|+|V|)

• Actually, can find the minimum path length from v to every node
– Still O(|E|+|V|)
– No faster way for a “distinguished” destination in the worst-case

• Now: Weighted graphs

Given a weighted graph and node v,
find the minimum-cost path from v to every node
• As before, asymptotically no harder than for one destination
• Unlike before, BFS will not work -> only looks at path length.
2
 Shortest Path: Applications
• Driving directions

• Cheap flight itineraries

• Network routing

• Critical paths in project management

3
 Not as easy
10 5
100 100
100 100 A B
-11 7

500

Why BFS won’t work: Shortest path may not have the fewest edges
– Annoying when this happens with costs of flights

We will assume there are no negative weights

• Problem is ill-defined if there are negative-cost cycles
• Today’s algorithm is wrong if edges can be negative
– There are other, slower (but not terrible) algorithms

4
 Dijkstra: an important CS “founder”
• Algorithm named after its inventor Edsger
Dijkstra (1930-2002)
– A good Dijkstra quote: “computer science is no
more about computers than astronomy is
about telescopes”
– My favorite Dijkstra joke: “Well, obviously he
had to go into computer science, he has ijk in
his name! He’s basically built for writing loops”

5
 Dijkstra’s algorithm
• The idea: reminiscent of BFS, but adapted to
handle weights
– Grow the set of nodes whose shortest distance
has been computed
– Nodes not processed yet will have a “best distance
so far”
– A priority queue will turn out to be useful for
efficiency
– Non-negative edge weights

6
 Dijkstra’s Algorithm: Idea
0 2 4 
2 2
A B F 3 H
1
5 10 2 1
4 9 3 G 
2 C 1
11 1
4 D E
7 12

• Initially, start node has cost 0 and all other nodes have cost 
• At each step:
– Pick closest unknown vertex v
– Add it to the “cloud” of known vertices
– Update distances for nodes with edges from v

• That’s it! (But we need to prove it produces correct answers)

7
 Pseudocode
1. For each node v, set v.cost =  and v.known = false
2. Set source.cost = 0
3. While there are unknown nodes in the graph
a) Select the unknown node v with lowest cost
b) Mark v as known
c) For each edge (v,u) with weight w,
c1 = v.cost + w // cost of best path through v to u
c2 = u.cost // cost of best path to u previously known
if(c1 < c2){ // if the path through v is better
u.cost = c1
u.path = v // for computing actual paths
}

8
 Dijkstra’s Algorithm
• Input: Graph G, start vertex s
Dijkstra(G,s)
01 for each vertex u Î G.V
02 u.dist := ¥
03 u.pred := NIL
04 s.dist := 0
05 S := Æ // S is used to explain the algorithm
06 init(Q, G.V) // Q is a priority queue
07 while not isEmpty(Q)
08 u := extractMin(Q)
09 S := S È {u}
10 for each v Î u.adj do
11 Relax(u, v, G) relaxing
12 modifyKey(Q, v) edges

Shortest Path  9
 Relaxation
• Relaxing an edge (u,v) means testing whether we can
improve the shortest path to v found so far by going through u.

u v u v
2 2 Relax (u,v,G)
5 9 5 6
if v.dist > u.dist + w(u,v) then
Relax(u,v) Relax(u,v)
v.dist := u.dist + w(u,v)
v.pred := u
5 2 7 5 2 6
u v u v

Shortest Path  10
 Dijkstra’s Correctness
• We prove that whenever u is added to S, u.dist= d(s,u), i.e., dist is minimum, and that equality is
maintained thereafter.
• Proof (by contradiction)
– Initially "v: v.dist ³ d(s,v)
– Let u be the first vertex such that there is a shorter path than u.dist, i.e., u.dist > d(s,u)
– We will show that this assumption leads to a contradiction

Shortest Path  11
 ...Dijkstra Correctness
• Let y be the first vertex Î V–S on the actual shortest path from s
to u, then it must be that y.dist = d(s,y) because
– x.dist is set correctly for y's predecessor xÎS on the shortest path (by
choice of u as the first vertex for which dist is set incorrectly)
– when the algorithm inserted x into S, it relaxed the edge (x,y) , setting
y.dist to the correct value

Shortest Path  12
 ...Dijkstra Correctness
u.dist > d(s,u) initial assumption

= d(s,y) + d(y,u) optimal substructure

= y.dist + d(y,u) correctness of y.dist

≥ y.dist no negative weights

• But u.dist > y.dist Þ algorithm would have chosen y (from the
PQ) to process next, not u
Þ contradiction
• Thus, u.dist = d(s,u) at time of insertion of u into S, and
Dijkstra's algorithm is correct

Shortest Path  13
 Dijkstra’s Running Time
• Extract-Min executed |V| time
• Decrease-Key executed |E| time
• Time = |V| TExtract-Min + |E| TDecrease-Key
• T depends on different Q implementations

Q T(Extract-Min) T(Decrease-Key) Total

array O(V) O(1) O(V 2)
binary O(lg V) O(lg V) O(E lg V)
heap

Shortest Path  14
 Example #1
0   
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9

3 G 
2 C
11 1
 D E
7 
Visited Node A B C D E F G H
0       

15
 Example #1
0 2  
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 
2 C 1 A
11 1
4 D E
7 
Visited Node A B C D E F G H
A 0       
2 1 4    

16
 Example #1
0 2  
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 
2 C 1 A, C
11 1
4 D E
7 12
Visited Node A B C D E F G H
A 0       
C 2 1 4    
2 4 12   

17
 Example #1
0 2 4 
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 
2 C 1 A, C, B
11 1
4 D E
7 12
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
4 12 4  

18
 Example #1
0 2 4 
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 
2 C 1 A, C, B, D
11 1
4 D E
7 12
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
D 4 12 4  
12 4  

19
 Example #1
0 2 4 7
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 
2 C 1 A, C, B, D, F
11 1
4 D E
7 12
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
D 4 12 4  
F 12 4  
12  7

20
 Example #1
0 2 4 7
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 8
2 C 1 A, C, B, D, F, H
11 1
4 D E
7 12
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
D 4 12 4  
F 12 4  
H 12  7
12 8

21
 Example #1
0 2 4 7
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 8
2 C 1 A, C, B, D, F, H, G
11 1
4 D E
7 11
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
D 4 12 4  
F 12 4  
H 12  7
G 12 8
11
22
 Example #1
0 2 4 7
2 2
A B F 3 H
1 Order Added to Known Set:
5 10 2 1
4 9 3 G 8
2 C 1 A, C, B, D, F, H, G, E
11 1
4 D E
7 11
Visited Node A B C D E F G H
A 0       
C 2 1 4    
B 2 4 12   
D 4 12 4  
F 12 4  
H 12  7
G 12 8
E 11
23
 Features
• When a vertex is marked known,
the cost of the shortest path to that node is known
– The path is also known by following back-pointers

• While a vertex is still not known,

another shorter path to it might still be found

Note: The “Order Added to Known Set” is not important

– A detail about how the algorithm works (client doesn’t care)
– Not used by the algorithm (implementation doesn’t care)
– It is sorted by path-cost, resolving ties in some way
• Helps give intuition of why the algorithm works

24
 Interpreting the Results
• Now that we’re done, how do we get the path
from, say, A to E?
0 2 4 7
2 2
A B F 3 H
1 vertex known? cost path
5 10 2 1
4 9 3 A Y 0
G 8
2 C 1 B Y 2 A
11 1
4 D E C Y 1 A
7 11
D Y 4 A
Order Added to Known Set: E Y 11 G
F Y 4 B
A, C, B, D, F, H, G, E
G Y 8 H
H Y 7 F
25
 Interpreting the Results
• Now that we’re done, how do we get the path
from, say, A to E?
0 2 4 7
2 2
A B F 3 H
1 vertex known? cost path
5 10 2 1
4 9 3 A Y 0
G 8
2 C 1 B Y 2 A
11 1
4 D E C Y 1 A
7 11
D Y 4 A
Order Added to Known Set: E Y 11 G
F Y 4 B
A, C, B, D, F, H, G, E
G Y 8 H
H Y 7 F
26
 Stopping Short
• How would this have worked differently if we were only
interested in:
– The path from A to F?

0 2 4 7
2 2
A B F 3 H vertex known? cost path
1
5 10 2 1 A Y 0
4 9 3 G 8
C 1
B Y 2 A
2 11 1
4 D E
C Y 1 A
7 11 D Y 4 A
Order Added to Known Set: E Y 11 G
F Y 4 B
A, C, B, D, F, H, G, E
G Y 8 H
H Y 7 F
27
 Stopping Short
0 2 4 7
2 2
A B F 3 H
1
5 10 2 1
4 9 3 G 
2 C 1
11 1
4 D E
7 12 vertex known? cost path
A Y 0
B Y 2 A
C Y 1 A
D Y 4 A
Order Added to Known Set: E  12 C
F Y 4 B
A, C, B, D, F
G 
H 7 F
28
 Example #2
0 
2
A B 1
1 
2 5 E
 1
1 D 3
5
 C
6  vertex known? cost path
2 G
 10
A 0
F
B 
C 
D 
Order Added to Known Set: E 
F 
G 

29
 Example #2
0 
2
A B 1
1 
2 5 E
1 1
1 D 3
5
2 C
6  vertex known? cost path
2 G
 10
A Y 0
F
B 
C £2 A
D 1 A
Order Added to Known Set: E 

A
F 
G 

30
 Example #2
0 6
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
7 10
A Y 0
F
B 6 D
C £2 A
D Y 1 A
Order Added to Known Set: E 2 D

A, D
F 7 D
G 6 D

31
 Example #2
0 6
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
4 10
A Y 0
F
B 6 D
C Y 2 A
D Y 1 A
Order Added to Known Set: E 2 D

A, D, C
F 4 C
G 6 D

32
 Example #2
0 3
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
4 10
A Y 0
F
B 3 E
C Y 2 A
D Y 1 A
Order Added to Known Set: E Y 2 D

A, D, C, E
F 4 C
G 6 D

33
 Example #2
0 3
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
4 10
A Y 0
F
B Y 3 E
C Y 2 A
D Y 1 A
Order Added to Known Set: E Y 2 D

A, D, C, E, B
F 4 C
G 6 D

34
 Example #2
0 3
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
4 10
A Y 0
F
B Y 3 E
C Y 2 A
D Y 1 A
Order Added to Known Set: E Y 2 D
F Y 4 C
A, D, C, E, B, F
G 6 D

35
 Example #2
0 3
2
A B 1
1 2
2 5 E
1 1
1 D 3
5
2 C
6 6 vertex known? cost path
2 G
4 10
A Y 0
F
B Y 3 E
C Y 2 A
D Y 1 A
Order Added to Known Set: E Y 2 D
F Y 4 C
A, D, C, E, B, F, G
G Y 6 D

36
 Example #3
1 1 1 1
X

80 70 60 50 …
90

As the algorithm runs, how will the best-cost-so-far for Y change?

Is this expensive?

37
 Example #3
1 1 1 1
X

80 70 60 50 …
90

As the algorithm runs, how will the best-cost-so-far for Y change?

, 90, 81, 72, 63, 54, …

Is this expensive?
No, each edge is processed only once

38
 A Greedy Algorithm

• A greedy algorithm:
– At each step, irrevocably does what seems best at that
step
• A locally optimal step, not known to be globally optimal
– Once a vertex is known, it is not revisited
• Turns out to be globally optimal

• Example: Dijkstra’s algorithm

– For single-source shortest paths in a weighted graph
(directed or undirected) with no negative-weight edges
39