Faculty of Science

Department of Computing Science

CMPUT 204
ALGORITHMS I
Winter 2024

Instructors: Jia You (B1), Martin Müller (B2)

Topic 2
Loop invariants
 Recap: Recursion & Induction

Its correctness can be

k
proved by induction Induction hypothesis : 3 − 1 is an even number for k ≥ 1

Base step Induction step

1
For n = 1 : 3 − 1 = 2 For n = k + 1 : 3 k+1 k k k
− 1 = 3*3 − 1 = 2*3 + 3 − 1
3
Recap: Insertion Sort and Runtime Analysis

Worst-case
Average-case
Best-case

4
Loop Invariants
Concepts
Example #1: FindSum
Is Insertion Sort Correct?

e c t? Works on tests?
c orr
t his
Is Not a proof!

Rigorous
in theory

6
Correctness For Simple Programs:
Using Logic

a b

x y

temp
7
 Correctness For Recursive Programs:
Induction

Its correctness can be

k
proved by induction Induction hypothesis : 3 − 1 is an even number for k ≥ 1

Base step Induction step

1
For n = 1 : 3 − 1 = 2 For n = k + 1 : 3 k+1 k k k
− 1 = 3*3 − 1 = 2*3 + 3 − 1
8
 Correctness For Programs with Loops:
Loop Invariants
Very important method
Similar to induction

Its correctness can be proved by the Loop Invariant method

You need to practice, by doing lots of exercises!!

9
 About Loop Invariants
Design and prove Loop invariant (LI):
formal method of arguing “why and how a loop works”

How to find a Loop Invariant?

- What happens in each round in the loop?
- How do you make progress towards the final result?
- Which parts of the problem are solved now?
- Coding tip: add a comment that explains your LI

10
 Formal Methods to Prove
the Correctness of (almost) any Program

Induction: prove the correctness of recursive programs

Loop invariants: prove the correctness of programs with loops

Helps you write code Write correct code

11
 Assertions and Invariants

Assertion: true-false statement

placed in a program to assert what is true at that point

Invariant: an assertion, shouls always be true

A statement about your code that remains true even while
your program is changing its data

12
Definition: Loop Invariant

Assertion #1: sublist A[1⋯j − 1] is sorted

Assertion #2: key is inserted to the sublist A[1⋯j − 1]
“in the right place” (how to make this formal?)

13
Steps to Identify and Prove Loop Invariant

Prove the
correctness
of a program
with loops

14
Example #1: FindSum

15
 Step #1: Identify Loop Invariant
In each iteration j will have a di erent
value, j=2, j=3,…

Informal idea for loop invariant:

variable sum holds the sum of “all
previous values”

Loop invariant for FindSum:

The Iteration for j

16
ff
An Invariant can have Many Equivalent Forms

Loop invariant for FindSum:

Note: j changes for each loop. The invariant refers to the current j value
17
 Some Failed Attempts at Loop Invariant

This is ambiguous
Loop invariant for FindSum:

18
Loop Invariant: Two Steps

19
 Step #2: Prove Loop Invariant
We need to prove 4 steps

Loop invariant for FindSum:

20
Step #2: Prove Loop Invariant

after
j =j+1
21
 Compare Loop Invariant with Induction
Step #1: Step #2: Prove loop invariants
Identify loop
invariants

Given the
claim
22
 Compare Loop Invariant with Induction
Base case

Induction hypothesis

Induction step

Induction does not

have these two steps

23
 Step #2:
Loop Invariant and
Prove loop invariants
Program Flow
Termination #1
Maintenance Termination #2
Initialization

Loop Invariant

Step #1: Next: more examples

Identify loop invariants
24
More Examples
Example #2, FindMax
Example #3, GetFact
Example #4, GetProduct
Example #5, Collatz
 Example #1 revisited: FindSum
Question 2: Do I understand the
At the beginning of: cumulative effect of the loop?
iteration j = 2, sum = A[1]
iteration j = 3, sum = A[1] + A[2]
iteration j = 4, sum = A[1] + A[2] + A[3]

⋮
iteration j = n, sum = A[1] + A[2] + ⋯ + A[n − 1]
After termination:
First iteration: j = 2 j = n + 1, sum = A[1] + A[2] + ⋯ + A[n]
Last iteration: j = n
Question 1: Do I understand what the code does?
Question 3: Can I word exactly the cumulative effect of the loop?
26
 FindSum: Termination
At the beginning of:
iteration j = 2, sum = A[1]
iteration j = 3, sum = A[1] + A[2]
iteration j = 4, sum = A[1] + A[2] + A[3]

⋮
iteration j = n, sum = A[1] + A[2] + ⋯ + A[n − 1]
After termination:
j = n + 1, sum = A[1] + A[2] + ⋯ + A[n]
At the beginning of
“iteration j = n + 1”

27
 FindSum: Cumulative Effect?
At the beginning of:
iteration j = 2, sum = A[1]
iteration j = 3, sum = A[1] + A[2]
iteration j = 4, sum = A[1] + A[2] + A[3]

⋮
iteration j = n, sum = A[1] + A[2] + ⋯ + A[n − 1]
iteration j =n+1
sum = A[1] + A[2] + ⋯ + A[n − 1] + A[n]
Do we really go through Yes: j is set to n+1
But: the while loop stops, nothing is done
iteration j = n + 1? Computation was done in previous loop, after invariant for j=n was
checked
28
Example #2: FindMax
Qu 2: Do I understand the
At the beginning of: cumulative effect of the loop?
iteration i = 2, m = A[1]
iteration i = 3, m = max{A[1], A[2]}
iteration i = 4, m = max{A[1], A[2], A[3]}

⋮
iteration i = n, m = max{A[1], ⋯, A[n − 1]}
After termination:
i = n + 1, m = max{A[1], A[2], ⋯, A[n]}

Qu 1: Do I understand what the code does?

Qu 3: Can I word exactly the cumulative effect of the loop?
29
 FindMax Step 1: Identify Loop Invariant
Practice at home: do Step #2,
At the beginning of: prove the loop invariant
iteration i = 2, m = A[1]
iteration i = 3, m = max{A[1], A[2]}
iteration i = 4, m = max{A[1], A[2], A[3]}

⋮
iteration i= i, m = max{A[1], A[2], ⋯, A[i − 1]}
iteration i = n, m = max{A[1], ⋯, A[n − 1]}
Remember, we do not
run iteration i = n + 1 iteration i = n + 1, m = max{A[1], A[2], ⋯, A[n]}

Loop invariant for FindMax

Formulate the pattern for “at the
beginning of iteration i”
30
 Example #3: GetFact Computes factorial, n!

At the beginning of:

iteration j = 2, s = 1 rst
j =2
iteration j = 3, s = 1 * 2 = 2! last
j =n
iteration j = 4, s = 1 * 2 * 3 = 3!

⋮
iteration j = j, s = 1 * 2 * ⋯ * ( j − 1) = ( j − 1)!
iteration j = n, s = 1 * 2 * ⋯ * (n − 1) = (n − 1)!
Remember, we do not
iteration j = n + 1, s = 1 * 2 * ⋯ * (n − 1) * n = n!
run iteration j = n + 1

Loop invariant for GetFact

Formulate the pattern for “at the
beginning of iteration j”
31
fi
 Interesting algorithm for
multiplying numbers,
Example #4: GetProduct based on binary
representation of x
Example for a x y product
more dif cult 1st iteration 20 4 0
loop invariant
2nd iteration 10 8 0
3rd iteration 5 16 0
4th iteration 2 32 16
5th iteration 1 64 16
6th iteration 0 128 80
Loop invariant for GetProduct
At the beginning of iteration x,
product = a * b − x * y Formulate the pattern for “at the
beginning of iteration x”
32
fi
 Example #5: Collatz 3n+1 Problem
current
iteration 1 24
iteration 2 12 iteration 8 8
iteration 3 6 iteration 9 4
iteration 4 3 iteration 10 2
iteration 5 10 iteration 11 1
iteration 6 5
Do you see what this program iteration 7 16
is doing?

33
 Example #5: Collatz (Cont’d)
current
iteration 1 25
iteration 2 82 iteration 8 47
iteration 3 41 iteration 9 142
iteration 4 124 iteration 10 71
iteration 5 62 iteration 11 214

Collatz conjecture: nobody knows whether this

iteration 6

iteration 7
31
94
⋮
loop terminates for all positive integer inputs
34
 Thought Process for Loop Invariants
Second: reason about the cumulative Including the
effect at the beginning of each iteration termination iteration

rst
j=j +1
j=j
rst
j=j
last
j=j +1

Third: formulate the pattern for j = j last

“at the beginning of iteration j” First: check if the loop This is “usually”
The loop invariant you are looking for terminates or not simple
rst
You need to be rigorous & correct Get the value of j and j last
fi
fi
fi
 Step #2: Maintenance
Proof Step #2: Termination #2
process Second: reason about the cumulative Including the
effect at the beginning of each iteration termination iteration

Step #2: Initialization

rst
j=j +1
j=j
rst
j=j
last
j=j +1

Third: formulate the pattern for j = j last

“at the beginning of iteration j” First: check if the loop
Step #2: Termination #1
The loop invariant you are looking for terminates or not
rst
Step #1: Identify the loop invariant Get the value of j and j last
fi
fi
fi
 Loop Invariants for
Insertion Sort
Correctness Proof using Loop Invariant
Dealing with nested loops
Insertion Sort: Two Loop Invariants
for loop:
First, selects next key to insert
(Then runs the while loop)
At end, inserts it in the right place

while loop:
Moves elements larger than the key
Computes the right place to insert key

Formulate the pattern for “at the

beginning of for-loop iteration j”

Formulate the pattern for “at the

beginning of while-loop iteration i”

38
 Loop Invariant - For Loop
Before for loop (BFL):
iteration j =2
iteration j =3
iteration j =4 rst
j =2
Get the value of key
⋮ last
j =n
iteration j =n
iteration j =n+1
There is no change to j

Do the insertion Remember, we do not

run iteration j = n + 1

39
fi
 6 5 3 7 1 4 2
Before for-loop (BFL):

iteration j =2 6 5 3 7 1 4 2 A[1]

iteration j =3 5 6 3 7 1 4 2 A[1,2]

iteration j =4 3 5 6 7 1 4 2 A[1,2,3]

iteration j =5 3 5 6 7 1 4 2 A[1,2,3,4]

iteration j =6 1 3 5 6 7 4 2 A[1,2,3,4,5]
⋮
Formulate the pattern for “at the beginning of iteration j”

40
Before for-loop (BFL):

iteration j =2 6 5 3 7 1 4 2 A[1]

iteration j =3 5 6 3 7 1 4 2 A[1,2]

iteration j =4 3 5 6 7 1 4 2 A[1,2,3]
iteration j =j ⋯ A[1,⋯, j − 1]
iteration j = 5 3
A[1] 5
A[2] 6 A[ j 7
− 1] A[1j] 4 2 A[1,2,3,4]

iteration j =6 1 3 5 6 7 4 2

⋮
Formulate the pattern for “at the beginning of iteration j”

41
 6 5 3 7 1 4 2
Before for-loop (BFL):

iteration j =2 6 5 3 7 1 4 2 A[1]

iteration j =3 5 6 3 7 1 4 2 A[1,2]

iteration j =4 3 5 6 7 1 4 2 A[1,2,3]
iteration j =j ⋯ A[1,⋯, j − 1]
iteration j = 5 3
A[1] 5
A[2] 6 A[ j 7
− 1] A[1j] 4 2 A[1,2,3,4]
Loop invariant for the for-loop
iteration j = 6 1 3 5 6 7 4 2

⋮
42
Loop Invariant - While Loop A before

BWL iteration i = 5 1 3 5 6 7 4 2 i=5

BWL iteration i = 4 1 3 5 6 7 7 2 i=4

BWL iteration i = 3 1 3 5 6 6 7 2 i=3

BWL iteration i = 2 1 3 5 5 6 7 2 i=2

before
At the beginning of iteration i, A[1..i + 1] = A [1..i + 1]
Loop invariant for the while-loop (part 1)
43
 before
A

BWL iteration i = 5 1 3 5 6 7 4 2 i=5

BWL iteration i = 4 1 3 5 6 7 7 2 i=4

BWL iteration i = 3 1 3 5 6 6 7 2 i=3

BWL iteration i = 2 1 3 5 5 6 7 2 i=2

Formulate the pattern for “at the beginning of iteration i”

44
 j=6
before before
A [i + 1] A [ j − 1] A before

BWL iteration i = 5 1 3 5 6 7 4 2 i=5

BWL iteration i = 4 1 3 5 6 7 7 2 i=4 A[6]

BWL iteration i = 3 1 3 5 6 6 7 2 i=3 A[5,6]

BWL iteration i = 2 1 3 5 5 6 7 2 i=2 A[4,5,6]

A[i + 2] A[ j]
before
At the beginning of iteration i, A[i + 2..j] = A [i + 1..j − 1]
Loop invariant for the while-loop (part 2)
45
BFL iteration j = 6 before
1 3 5 6 7 4 2 A

BWL iteration i = 5 1 3 5 6 7 4 2 i=5 j=6

BWL iteration i = 4 1 3 5 6 7 7 2 i=4 j=6

BWL iteration i = 3 1 3 5 6 6 7 2 i=3 j=6

BWL iteration i = 2 1 3 5 5 6 7 2 i=2 j=6

At the beginning of iteration i

Loop invariant for before
the while-loop
A[1..i + 1] = A [1..i + 1]
before
A[i + 2..j] = A [i + 1..j − 1]
46
 Loop Invariant (While-Loop)
Before for-loop (BFL) iteration j

Before while-loop (BWL)

iteration i =j−1
iteration i =j−2
iteration i =j−3
⋮
last
iteration i =i
rst
i =j−1 iteration i =i last
− 1 Termination

{i* otherwise
last 1 if A[1] > key
i =
A[i*] > key and A[i* − 1] < key
47
fi
 Insertion Sort Summary: Two Loop Invariants
Loop invariant for the while-loop

Any new notation in

your loop invariant must be
well de ned
Loop invariant for the for-loop
A[1,⋯, j − 1] is an
ordered permutation of
corresponding elements
in the initial array
This is important: be rigorous and correct 48
fi
 Next Topic
Asymptotic Growth of Functions