CSE 215: Data Structures & Algorithms II

Week 1
Heaps and Priority Queue

Lec Sumaiya Nuha Mustafina

Dept Of CSE
[email protected]
1
Google Classroom (CSE 215+ CSE 216)

o4d3khk

2
Course Outline

Link to the outline

3
Heap?

4
Is it the same as heap memory ?

5
Is it the same as heap memory ?

6
Heap Data Structure

● The (binary) heap data structure is a complete binary tree that

satisfies the heap property.

● An array object that can be viewed as a complete binary tree.

● Each node of the tree corresponds to an element of the array.

● The tree is completely filled on all levels except possibly the

lowest, which is filled from the left up to a point.

● Application: Priority Queue, Heapsort etc.

7
Heap Data Structure View as a Binary Tree

16

14 15

6 7 9 5

2 4 3

8
Heap Data Structure View as an Array

0
16

1 2
14 15

3 4 5 6
6 7 9 5

7
8 9
2 4 3

Level Order Traversal

0 1 2 3 4 5 6 7 8 9

16 14 15 6 7 9 5 2 4 3
9
Heap Data Structure

● Two kinds of binary heaps:

○ max-heaps
○ min-heaps.

● In both kinds, the values in the nodes satisfy a heap property

10
Heap Properties

In a max-heap, the max-heap property is that for every node i other

than the root,

A[PARENT( i )] >= A[ i ] ;

that is, the value of a node is at most the value of its parent.

Thus, the largest element in a max-heap is stored at the root, and

the subtree rooted at a node contains values no larger than that
contained at the node itself.

11
Heap Properties

In a min-heap; the min-heap property is that for every node i other

than the root,

A[PARENT( i )] <= A[ i ] ;

The smallest element in a min-heap is at the root.

12
Heap

Q: Is the following array a max-heap?

23 17 14 6 13 10 1 5 7 12

13
Heap Data Structure

● Since a heap of n elements is based on a complete binary tree, its

height is 𝚹(log n).

● The total number of comparisons required in heap is according

to the height of the tree.

● Thus the time complexity of basic operation would also be

O(logn).

14
Heap: Some Basic Procedures

● The MAX-HEAPIFY procedure, which runs in O(log n) time, is the

key to maintaining the max-heap property.

● The BUILD-MAX-HEAP procedure, which runs in linear time,

produces a max-heap from an unordered input array.

● The HEAPSORT procedure, which runs in O(n.log n) time, sorts an

array in place.

● The MAX-HEAP-INSERT ,HEAP-EXTRACT-MAX , HEAP-INCREASE-

KEY, and HEAP-MAXIMUM procedures, which run in O(log n) time,
allow the heap data structure to implement a priority queue.

15
Heap represented as a Binary tree and an array
0

16

1 2
14 15

6 7 9 5

2 4 3

0 1 2 3 4 5 6 7 8 9

16 14 15 6 7 9 5 2 4 3
16
Heap
For 0-based indexing of node numbers.

PARENT(i)
return ⌊(i-1)/2⌋

LEFT(i)
return 2i+1

RIGHT(i)
return 2i+2

For leave nodes: indexed by ⌊(n-1)/2⌋, ⌊(n-1)/2⌋+1, ….. n-1.

n: number of elements in heap.

17
Heap
For 1-based indexing of node numbers.

PARENT(i)
return ⌊i/2⌋

LEFT(i)
return 2i

RIGHT(i)
return 2i+1

For leave nodes: indexed by ⌊n/2⌋+1, ⌊n/2⌋+2, ….. n.

**All the algorithms here are written considering 1-based indexing.

18
Maintaining the heap property

19
MAX-HEAP Property!!

16

2 3
4 15

4 5 6
7
13 7 9 5

8 9 10
2 6 3

20
MAX-HEAP Property!!

16

2 3
4 15

4 5 6
7
13 7 9 5

8 9 10
2 6 3

21
MAX-HEAPIFY(A,2)

l = 4; r = 5
largest = 4 1

16

2 3
4 15

4 5 6
7
13 7 9 5

8 9 10
2 6 3

22
MAX-HEAPIFY(A,2)

l = 4; r = 5
largest = 4 1

16

2 3
13 15

4 5 6
7
4 7 9 5

8 9 10
2 6 3

23
MAX-HEAPIFY(A,4)

l = 8; r = 9
largest = 9 1

16

2 3
13 15

4 5 6
7
4 7 9 5

8 9 10
2 6 3

24
MAX-HEAPIFY(A,4)

l = 8; r = 9
largest = 9 1

16

2 3
13 15

4 5 6
7
6 7 9 5

8 9 10
2 4 3

25
MAX HEAP:

16

2 3
13 15

4 5 6
7
6 7 9 5

8 9 10
2 4 3

26
Building a Heap from a given array

● Array A[1..n]
, where n= A:length

● Use MAX-HEAPIFY in a bottom-up manner to convert into a max-

heap.

● The procedure BUILD-MAX-HEAP goes through the non-leaf

nodes of the tree and runs MAX-HEAPIFY on each one.

27
Building a Heap from a given array
0 1 2 3 4 5 6 7 8 9 10

4 1 3 2 16 9 10 14 8 7

2 3
1 3

4 5 6
7
2 16 9 10

8 9 10
14 8 7

28
Building a Heap from a given array

29
Building a Heap from a given array

30
Building a Heap from a given array

31
Building a Heap from a given array

32
Building a Heap from a given array

33
Building a Heap from a given array

34
Building a Heap from a given array

10 5 25 40 30 35 20

35
Time Complexity of BUILD-MAX-HEAP

Simple Upper Bound:

● The time required by MAX -HEAPIFY when called on a node of height h is

O(h).

● Each call to MAX-HEAPIFY costs O(log n) time,

● BUILD-MAX-HEAP makes O(n) such calls. Thus, the running time is

O(nlogn).

● This upper bound, though correct, is not asymptotically tight. Why??

36
Time Complexity of BUILD-MAX-HEAP
Why is the upper bound not asymptotically tight?

37
Time Complexity of BUILD-MAX-HEAP
Why is the upper bound not asymptotically tight?

The time for MAX-HEAPIFY to run at a node varies with the height of the
node in the tree.

38
Time Complexity of BUILD-MAX-HEAP

There are at most

Substituting x by 1/2 in nodes of height h in

any n-element heap.

we get,

39
Time Complexity of BUILD-MAX-HEAP
So the tighter upper bound
time complexity :

To build a max-heap from an unordered array, it takes linear

time

40
HEAP: Insertion Deletion

Insertion: (O(nlogn)or O(n) )

1. Insert in the leaf node.
2. Increase heap size
3. Heapify the whole tree (BUILD-
MAX-HEAP).

Deletion: (O(nlogn) or O(n))

1. Swap the node to be deleted with

the last leaf node.
2. Remove the last leaf node.
3. Heapify the whole tree (BUILD-
MAX-HEAP).

41
HEAP: Insertion

Insertion (O(logn)):
1. Insert in the leaf node and
Increase heap size
2. Maintain heap property by
exchanging with the parent
nodes(ancestors).

42
HEAP: Insertion (logn)
INSERT : 14

16

2 3
13 15

4 5 6
7
6 7 9 5

8 9 10
2 4 3

43
HEAP: Insertion (logn)
INSERT : 14

16

2 3
13 15

4 5 6
7
6 7 9 5

8 9 10
2 4 3 14 11

44
HEAP: Insertion (logn)
INSERT : 14

16

2 3
13 15

4 5 6
7
6 14 9 5

8 9 10
2 4 3 7 11

45
HEAP: Insertion (logn)
INSERT : 14

16

2 3
14 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 7 11

46
HEAP: Insertion (logn)
After INSERTION

16

2 3
14 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 7 11

47
HEAP: Deletion

Deletion (O(logn)):
1. Swap the node to be deleted
with the last leaf node.
2. Remove the last leaf node.
3. Maintain heap property

48
HEAP: Deletion

Deletion (O(logn)):
3. Maintain heap property

HEAP-DELETE(A, i)
if A[i] > A[A.heap-size]
swap(A[i], A[A.heap-size])
A.heap-size = A.heap-size - 1
MAX-HEAPIFY(A, i)
else
swap(A[i], A[A.heap-size])
A.heap-size = A.heap-size - 1

49
HEAP: Deletion (logn)
Delete 14 ,i=2
HEAP-DELETE(A, 2)
1
Here,
A[i] > A[Heapsize] 16

2 3
14 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 7 11

50
HEAP: Deletion (logn)
Delete 14 ,i=2 1. Swap the node to be deleted
with the last leaf node.
HEAP-DELETE(A, 2)
2. Remove the last leaf node.
1 3. Maintain heap property

16

2 3
14 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 7 11

51
HEAP: Deletion (logn)
Delete 14 ,i=2 1. Swap the node to be deleted
with the last leaf node.
HEAP-DELETE(A, 2)
2. Remove the last leaf node.
1 3. Maintain heap property

16

2 3
7 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 14 11

52
HEAP: Deletion (logn)
Delete 14 ,i=2 1. Swap the node to be deleted
with the last leaf node.
HEAP-DELETE(A, 2)
2. Remove the last leaf node.
1 3. Maintain heap property

16

2 3
7 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 14 11

53
HEAP: Deletion (logn)
Delete 14 ,i=2 1. Swap the node to be deleted
with the last leaf node.
HEAP-DELETE(A, 2)
2. Remove the last leaf node.
1 3. Maintain heap property

16

2 3
7 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 14

54
HEAP: Deletion (logn)
HEAP-DELETE(A, 2) 1. Swap the node to be deleted
with the last leaf node.
2. Remove the last leaf node.
As A[i] > A[Heapsize]
1 3. Maintain heap property
MAX-HEAPIFY(A, 2)
16

2 3
7 15

4 5 6
7
6 13 9 5

8 9 10
2 4 3 14

55
HEAP: Deletion (logn)
After Deleting 14

16

2 3
13 15

4 5 6
7
6 7 9 5

8 9 10
2 4 3

56
HEAP SORT

Time Complexity? O(n logn)

57
HEAP SORT

Let’s take an array and apply heap sort to it.

10 5 25 40 30 35 20

58
Priority Queue

Priority queues come in two forms:

max-priority queues and min-priority queues.

A priority queue is a data structure for maintaining a set S of

elements, each with an associated value called a key.

Key = priority value

Elements with higher priority values are retrieved before elements

with lower priority values. (max-priority queue)

59
Priority Queue

Operations of Max Priority Queue:

INSERT(S, x): inserts the element x into the set S.

MAXIMUM(S): returns the element of S with the largest key.

EXTRACT-MAX(S): removes and returns the element of S with the

largest key

INCREASE-KEY (S, x, k) increases the value of element x’s key to the

new value k, which is assumed to be at least as large as x’s current
key value.

60
Priority Queue

Operations of Min Priority Queue:

INSERT(S, x): inserts the element x into the set S.

MINIMUM(S): returns the element of S with the smallest key.

EXTRACT-MIN(S): removes and returns the element of S with the

smallest key

DECREASE-KEY (S, x, k): decreases the value of element x’s key to the
new value k.

61
Priority Queue

Implementation of Max Priority Queue Using Max Heap:

HEAP-MAXIMUM(A): Implements MAXIMUM(S) operation in 𝚹(1) time.

HEAP-EXTRACT-MAX: implements EXTRACT-MAX(S) and running time

is O(logn).

HEAP-INCREASE-KEY: implements the INCREASE-KEY operation and

running time is O(logn).

MAX-HEAP-INSERT : implements the INSERT operation and running

time for a n element heap is O(logn).

62
Priority Queue Operations

Implementation of Max Priority Queue Using Max Heap:

HEAP-MAXIMUM (A)
return A[1]

63
Priority Queue Operations

Implementation of Max Priority Queue Using Max Heap:

64
Priority Queue Operations

Implementation of Max Priority Queue Using Max Heap:

65
Priority Queue Operations

Implementation of Max Priority Queue Using Max Heap:

66
Priority Queue Operations

The operation of HEAP-INCREASE-KEY(A,i,k) :

67
Priority Queue Operations

The operation of HEAP-INCREASE-KEY:

68
Priority Queue Operations

The operation of HEAP-INCREASE-KEY:

69
Priority Queue Operations

The operation of HEAP-INCREASE-KEY:

70
Priority Queue Operations

Illustrate the operation of HEAP-EXTRACT-MAX:

A={15, 13, 9, 5, 12, 8, 7, 4, 0, 6, 2, 1}

71
Heap Exercises

● What are the minimum and maximum numbers of elements in a

heap of height h?
● Show that an n-element heap has height ⌊log n⌋.
● Where in a max-heap might the smallest element reside,
assuming that all elements are distinct?
● Show that there are at most Гn/2^(h+1)ㄱ nodes of height h in
any n-element heap.
● What is the running time of HEAPSORT on an array A of length n
that is already sorted in increasing order? What about decreasing
order?

72
73