Graph theory

6 vertices (V), nodes, points

7 edges (E)

GV: T.S. Nguyễn Quốc Minh 1

• Unidirectional. each edge • Each edge is bidirectional
has a specific direction

GV: T.S. Nguyễn Quốc Minh 2

BFS

GV: T.S. Nguyễn Quốc Minh 3

• Breath First Search (BFS)
• Queue
• Enqueue()
• Dequeue()

Each node/vertex is either visited or

unvisited

Algorithm:
• Enqueue a starting node. Mark it as visited.
• While queue is NOT empty:
• Dequeue a node. Print it out.
• Enqueue ALL of ITS unvisited ADJACENT nodes.
Mark them as visited.

GV: T.S. Nguyễn Quốc Minh 4

BFS

GV: T.S. Nguyễn Quốc Minh 5

BFS

GV: T.S. Nguyễn Quốc Minh 6

BFS

GV: T.S. Nguyễn Quốc Minh 7

BFS

GV: T.S. Nguyễn Quốc Minh 8

DFS

GV: T.S. Nguyễn Quốc Minh 9

• Depth First Search (DFS)
• Stack
• Push()
• Pop()

Each node/vertex is either visited or

unvisited

Algorithm:
• Push a starting node. Mark it as visited.
• While stack is NOT empty:
• Pop the stack. Print it out.
• Push ALL of ITS unvisited ADJACENT nodes.
Mark them as visited.

GV: T.S. Nguyễn Quốc Minh 10

DFS

GV: T.S. Nguyễn Quốc Minh 11

DFS

GV: T.S. Nguyễn Quốc Minh 12

DFS

GV: T.S. Nguyễn Quốc Minh 13

DFS

GV: T.S. Nguyễn Quốc Minh 14

 Comparison
• Breath First Search (BFS) • Depth First Search (DFS)
• Queue • Stack
• Enqueue() • Push()
• Dequeue() • Pop()

Each node/vertex is either visited or unvisited Each node/vertex is either visited or unvisited

Algorithm: Algorithm:
• Enqueue a starting node. Mark it as visited. • Push a starting node. Mark it as visited.
• While queue is NOT empty: • While stack is NOT empty:
• Dequeue a node. Print it out. • Pop the stack. Print it out.
• Enqueue ALL of ITS unvisited ADJACENT nodes. • Push ALL of ITS unvisited ADJACENT nodes. Mark
Mark them as visited. them as visited.

GV: T.S. Nguyễn Quốc Minh 15

 BFS or DFS
It heavily depends on the structure of the search tree and the number and
location of solutions (aka searched-for items).
• If you know a solution is not far from the root of the tree, a breadth first
search (BFS) might be better.
• If the tree is very deep and solutions are rare, depth first search (DFS) might
take an extremely long time, but BFS could be faster.
• If the tree is very wide, a BFS might need too much memory, so it might be
completely impractical.
• If solutions are frequent but located deep in the tree, BFS could be
impractical.
• If the search tree is very deep you will need to restrict the search depth for
depth first search (DFS), anyway (for example with iterative deepening).

GV: T.S. Nguyễn Quốc Minh 16

Minimum Spanning Tree

GV: T.S. Nguyễn Quốc Minh 17

Problem: Laying Telephone Wire

Central office

18 GV: T.S. Nguyễn Quốc Minh

Wiring: Naive Approach

Central office

Expensive!

19 GV: T.S. Nguyễn Quốc Minh

Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

20 GV: T.S. Nguyễn Quốc Minh

 Weighted graph
• A weighted graph is a graph in which a number (weight) is
assigned to each edge. Such weights could represent costs, distances
or capacities, …

GV: T.S. Nguyễn Quốc Minh 21

 Prim's algorithm:

• Prim's algorithm:
• is agreedy algorithm
• finds a minimum spanning tree (MST) for
a weighted undirected graph

GV: T.S. Nguyễn Quốc Minh 22

 Prim's algorithm:

• Start with an empty spanning tree.

• The idea is to maintain two sets of vertices. (V)
• The first set contains the vertices already included in the MST (or visited),
• The second set contains the vertices not yet included. (or unvisited)
• At every step, it considers all the edges that connect the two sets
and picks the minimum weighted edge from these edges. Add
the chosen edge to the MST if it does not form any cycle.
• Repeat until all vertices are visited

GV: T.S. Nguyễn Quốc Minh 23

Example 1

GV: T.S. Nguyễn Quốc Minh 24

Graph Spanning Tree

MST = {}
Unvisited = { A, B, C, D, E}

GV: T.S. Nguyễn Quốc Minh 25

Graph Spanning Tree

MST = {B}
Unvisited = { A, C, D, E}

GV: T.S. Nguyễn Quốc Minh 26

Graph Spanning Tree

MST = {B, D}
Unvisited = { A, C, E}

GV: T.S. Nguyễn Quốc Minh 27

Graph Spanning Tree

MST = {B, D, E}
Unvisited = { A, C}

GV: T.S. Nguyễn Quốc Minh 28

Graph Spanning Tree

MST = {B, D, E, C}
Unvisited = { A}

GV: T.S. Nguyễn Quốc Minh 29

Graph Spanning Tree

MST = {B, D, E, C, A}
Unvisited = {}

GV: T.S. Nguyễn Quốc Minh 30

Example 2

GV: T.S. Nguyễn Quốc Minh 31

GV: T.S. Nguyễn Quốc Minh 32
GV: T.S. Nguyễn Quốc Minh 33
GV: T.S. Nguyễn Quốc Minh 34
GV: T.S. Nguyễn Quốc Minh 35
GV: T.S. Nguyễn Quốc Minh 36
GV: T.S. Nguyễn Quốc Minh 37
GV: T.S. Nguyễn Quốc Minh 38
GV: T.S. Nguyễn Quốc Minh 39
GV: T.S. Nguyễn Quốc Minh 40
Cost(MST) = (4 + 8 + 1 + 2 + 4 + 2 + 7 + 9) = 37

GV: T.S. Nguyễn Quốc Minh 41

Homework 2

• Write Prim’s MST algorithm in Python for the following graph

• Must NOT create a cycle
• Output: Edges of MST and Cost(MST)

GV: T.S. Nguyễn Quốc Minh 42

Kruskal’s Minimum Spanning Tree (MST) Algorithm
• Steps for finding MST using Kruskal’s algorithm:
1. Sort all the edges in non-decreasing order of their weight.
2. Pick the smallest edge. Add it if it doesn’t form a cycle. Check if it
forms a cycle with the spanning tree formed so far. If the cycle is not
formed, include this edge. Else, discard it.
3. Repeat step#2 until there are (V-1) edges in the spanning tree.

GV: T.S. Nguyễn Quốc Minh 43

Example 2

GV: T.S. Nguyễn Quốc Minh 44

 SORTING WEIGHTS
Edge Weight
Source Vertex Destination Vertex

E F 2
F D 2
B C 3
C F 3
C D 4
B F 5
B D 6
A B 7
A C 8

GV: T.S. Nguyễn Quốc Minh 45

GV: T.S. Nguyễn Quốc Minh 46
GV: T.S. Nguyễn Quốc Minh 47
GV: T.S. Nguyễn Quốc Minh 48
GV: T.S. Nguyễn Quốc Minh 49
GV: T.S. Nguyễn Quốc Minh 50
GV: T.S. Nguyễn Quốc Minh 51
GV: T.S. Nguyễn Quốc Minh 52
Example 2

GV: T.S. Nguyễn Quốc Minh 53

• The graph contains 9 vertices and 14 edges. So, the minimum
spanning tree formed will be having (9 – 1) = 8 edges.

GV: T.S. Nguyễn Quốc Minh 54

Sorting edges:
Weight Source Destination

1 7 6

2 8 2

2 6 5

4 0 1

4 2 5

6 8 6

7 2 3

7 7 8

8 0 7

8 1 2

9 3 4

10 5 4

11 1 7

14 3 5
GV: T.S. Nguyễn Quốc Minh 55
GV: T.S. Nguyễn Quốc Minh 56
GV: T.S. Nguyễn Quốc Minh 57
GV: T.S. Nguyễn Quốc Minh 58
GV: T.S. Nguyễn Quốc Minh 59
GV: T.S. Nguyễn Quốc Minh 60
GV: T.S. Nguyễn Quốc Minh 61
GV: T.S. Nguyễn Quốc Minh 62
Note: Since the number of edges included in the MST
equals to (V – 1), so the algorithm stops here

GV: T.S. Nguyễn Quốc Minh 63

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

64 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

65 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

66 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

67 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

68 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

69 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

70 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

71 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

72 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

73 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

74 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

75 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342

76 GV: T.S. Nguyễn Quốc Minh

Example 2704
867 BOS

849 PVD
ORD 187
740 144
1846 621 JFK
184 1258
802
SFO BWI
1391
1464
337 1090
DFW 946
LAX 1235
1121
MIA
2342
77 GV: T.S. Nguyễn Quốc Minh
 Prim or Kruskal
• Prim's algorithm is significantly faster in the limit when you've got a
really dense graph with many more edges than vertices.
• Kruskal performs better in typical situations (sparse graphs)
because it uses simpler data structures.
• We should use Kruskal when the graph is sparse, [Link] number of
edges,like E=O(V),when the edges are already sorted or if we can sort
them in linear time. We should use Prim when the graph is dense, i.e
number of edges is high ,like E=O(V²).

GV: T.S. Nguyễn Quốc Minh 78

 Dijkstra's algorithm
• Dijkstra's algorithm finds the shortest path from a given source
node to every other node
• Dijkstra's Algorithm can only work with graphs that
have positive weights.

GV: T.S. Nguyễn Quốc Minh 79

• Basics of Dijkstra's Algorithm
• Dijkstra's Algorithm basically starts at the node that you choose (the
source node) and it analyzes the graph to find the shortest path
between that node and all the other nodes in the graph.
• The algorithm keeps track of the currently known shortest distance
from each node to the source node and it updates these values if it
finds a shorter path.
• Once the algorithm has found the shortest path between the source
node and another node, that node is marked as "visited" and added
to the path.
• The process continues until all the nodes in the graph have been
added to the path. This way, we have a path that connects the source
node to all other nodes following the shortest path possible to reach
each node.
GV: T.S. Nguyễn Quốc Minh 80
Algorithm:

GV: T.S. Nguyễn Quốc Minh 81

GV: T.S. Nguyễn Quốc Minh 82
GV: T.S. Nguyễn Quốc Minh 83
Dijkstra’s Algorithm Steps:
• Set all vertices distances = infinity except for the source vertex, set the source
distance = 0.
• Push the source vertex in a min-priority queue in the form (distance , vertex), as
the comparison in the min-priority queue will be according to vertices distances.
• Pop the vertex with the minimum distance from the priority queue (at first the
popped vertex = source).
• Update the distances of the connected vertices to the popped vertex in case of
"current vertex distance + edge weight < next vertex distance", then push the
vertex
with the new distance to the priority queue.
• If the popped vertex is visited before, just continue without using it.
• Apply the same algorithm again until the priority queue is empty.

GV: T.S. Nguyễn Quốc Minh 84

GV: T.S. Nguyễn Quốc Minh 85
Priority Queue/Heap Queue
• Dijkstra used a priority queue to keep track of smallest cost

GV: T.S. Nguyễn Quốc Minh 86

 Time Complexity
• Dijkstra: O((V + E) log V)
• Prim : O((V +E) log V)
• Kruskal: O(ElogE)

GV: T.S. Nguyễn Quốc Minh 87

 Space Complexity
• Dijkstra: O(V +E)
• Prim : O(V +E)
• Kruskal: O(V+E)

GV: T.S. Nguyễn Quốc Minh 88

 Start at Node 0

SPT = {}

GV: T.S. Nguyễn Quốc Minh 89

SPT = {0, }

GV: T.S. Nguyễn Quốc Minh 90

SPT = {0, 1 }

GV: T.S. Nguyễn Quốc Minh 91

SPT = {0, 1, 2 }

GV: T.S. Nguyễn Quốc Minh 92

SPT = {0, 1, 2, 3 }

GV: T.S. Nguyễn Quốc Minh 93

SPT = {0, 1, 2, 3, 4}

GV: T.S. Nguyễn Quốc Minh 94

SPT = {0, 1, 2, 3, 4, 6}

GV: T.S. Nguyễn Quốc Minh 95

SPT = {0, 1, 2, 3, 4, 6, 5}
PrevNode 0 0 1 3 4 3

GV: T.S. Nguyễn Quốc Minh 96

GV: T.S. Nguyễn Quốc Minh 97
GV: T.S. Nguyễn Quốc Minh 98
GV: T.S. Nguyễn Quốc Minh 99
GV: T.S. Nguyễn Quốc Minh 100
 HEURISTIC
• A heuristic[1] or heuristic technique (problem solving, mental shortcut, rule of
thumb)[2][3][4][5] is any approach to problem solving that employs
a pragmatic method that is not fully optimized, perfected, or rationalized, but is
nevertheless "good enough" as an approximation or attribute substitution

• Heuristic: proceeding to a solution by trial and error or by rules that are

only loosely defined

• In mathematical optimization and computer science, heuristic (from Greek

εὑρίσκω "I find, discover") is a technique designed for problem solving more
quickly when classic methods are too slow for finding an exact or approximate
solution, or when classic methods fail to find any exact solution in a search
space. This is achieved by trading optimality, completeness, accuracy, or
precision for speed. In a way, it can be considered a shortcut.

GV: T.S. Nguyễn Quốc Minh 101

 A* Search Algorithm
• Why A* search?
• A* Search Algorithm is similar to Dijkstra’s in that it explores nodes in
a graph to find the lowest-cost path.
• A* uses an additional heuristic function in the cost calculation to
inform its search. The heuristic function acts as a hint to guide the
algorithm towards the destination, and hopefully reducing the search
space.

GV: T.S. Nguyễn Quốc Minh 102

• Cost to visit a node:

• - n is any node in the graph.

- f(n) is the total cost to reach node n from the starting node.
- g(n) is the exact cost to reach node n from the starting node. g(u) does not include the
heuristic in the cost.
- h(n) is the heuristic function, an estimate to reach the destination node from node n.

• The heuristic function h(n) needs to be admissible; it should never overestimate the optimal
cost of reaching the destination node from the current node. If the heuristic is not
admissible, A* is not guaranteed to find the lowest-cost path.

• Dijkstra is a special case of A* Search Algorithm, where h = 0 for all

nodes (only the edge weights factor into the final cost)

GV: T.S. Nguyễn Quốc Minh 103

Diịkstra A* search

GV: T.S. Nguyễn Quốc Minh 104

Heuristic function

• Manhattan Distance

•When to use this heuristic? – When we are allowed to

move only in four directions only (right, left, top, bottom)

GV: T.S. Nguyễn Quốc Minh 105

• Diagonal Distance
It is nothing but the maximum of absolute values of differences in the goal’s x and y coordinates
and the current cell’s x and y coordinates respectively, i.e.,

•When to use this heuristic? – When we are allowed to

move in eight directions only (similar to a move of a King
in Chess) GV: T.S. Nguyễn Quốc Minh 106
• Euclidean Distance

GV: T.S. Nguyễn Quốc Minh 107