Randomized Algorithms
Algorithms that you have seen so far are all deterministic
Always follows the same execution path for the same input
Randomized algorithms are non-deterministic
Makes some random choices in some steps of the
algorithms
Output and/or running time of the algorithm may depend
on the random choices made
If you run the algorithm more than once on the same input
data, the results may differ depending on the random
choice
A Simple Example: Choosing a Large
Number
Given n numbers, find a number that is ≥ median
Simple deterministic algorithm takes O( n) time
A simple randomized algorithm:
Choose k numbers randomly, k < n, and output the
maximum
Runs faster
But this may not give the correct result always
But the probability that an incorrect result is output is less
1
than
2𝑘𝑘
Somewhat trivial example, but highlights the essence of a
large class of randomized algorithms
Types of Randomized Algorithms
Monte Carlo Algorithms
Randomized algorithms that always has the same time
complexity, but may sometimes produce incorrect outputs
depending on random choices made
Time complexity is deterministic, correctness is probabilistic
Las Vegas Algorithm
Randomized algorithms that never produce incorrect
output, but may have different time complexity depending
on random choices made (including sometimes not
terminating at all)
Time complexity is probabilistic, correctness is
deterministic
Monte Carlo Algorithms
Monte Carlo Algorithms
May sometimes produce incorrect output
But will always terminate with same time complexity
(usually polynomial to be of interest)
Let p = probability that it will produce incorrect output
Idea:
1
Design the algorithm such that p ≤ ½
2
Run the algorithm k times on the same input
Probability that it will produce incorrect output on all k
1
trials ≤
2𝑘𝑘
Can make the probability of error arbitrarily small by
making k sufficiently large
Example: Primality Testing
A decision problem: decide whether a given positive
integer n is prime or not
Without loss of generality, we will assume n is odd
Very important problem in practice
Known polynomial time algorithm has high complexity,
takes too much time in practice
Many good randomized algorithms designed
Basic Idea of randomized algorithms for primality testing
Try to find “witness of compositeness” randomly
A number such that know number-theoretic tests applied on
it can prove that n is composite
If no witness can be found in large enough number of tries,
declare n as prime
Problem: A composite number may be declared as prime
if the random choices fail to choose a witness in every try
So the Yes answer may be wrong
But a prime is never declared as composite
So the No answer is always correct
Monte Carlo algorithms like this for decision problems
are said to have one-sided error
One of the Yes/No answers may be wrong, but not both
How can we make the probability of declaring a
composite as prime (incorrect output) small?
Choose the witness type that you want to use such that
there is an “abundance of witnesses” in the space from
which the witness is chosen
If you choose from a space of size N, and you know a lower
bound m on the number of witnesses of compositeness in
that space, then the probability of incorrect output in each
𝑚𝑚
try is upper bounded by 1 − 𝑁𝑁
Choose witness type so that m is proven to be large
Simplest witness of compositeness: factor of n
Algorithm:
Pick an integer x randomly from 3 to 𝑛𝑛
Check if x is a factor of n
If no, output Yes, else output No
Problem: factors are not abundant
33 has only 2 factors, 3 and 11
81 has only 3 factors, 3, 9, and 37
In general, m is very small compared to N
So this is not a good type of witness to use
Witness: Fermat Witness
Fermat’s Theorem: If n is prime and p is an integer co-prime
to n, then 𝑎𝑎𝑛𝑛−1 ≡ 1 (𝑚𝑚𝑚𝑚𝑚𝑚 𝑛𝑛)
But the inverse is not true, so this is a necessary condition for
n to be prime, but not sufficient
There exists composite numbers that satisfy this for some a
Example: n = 15 and a = 4
n is said to be a pseudoprime to base a
Fermat witness of compositeness: An integer a ∈ ℤ+ 𝑛𝑛 such
that
𝑔𝑔𝑔𝑔𝑚𝑚 𝑎𝑎, 𝑛𝑛 ≠ 1 or 𝑎𝑎 𝑛𝑛−1 ≢ 1 (𝑚𝑚𝑚𝑚𝑚𝑚 𝑛𝑛)
Algorithm:
Pick an integer a at random from ℤ+ 𝑛𝑛
Check if 𝑔𝑔𝑔𝑔𝑚𝑚 𝑎𝑎, 𝑛𝑛 = 1 and 𝑎𝑎 𝑛𝑛−1 ≡ 1 (𝑚𝑚𝑚𝑚𝑚𝑚 𝑛𝑛)
If yes, output Yes, else output No
How many such a’s can be there in ℤ+
𝑛𝑛 ?
