Randomized Algorithms

CS648

Lecture 27
• Random bit complexity
• Derandomization

1
 Random bit complexity

Input

A Randomized Algorithm
(for Min-Cut, QuickSort, RIC,…)

Random Bit generator

Definition : The total number of random bits taken from the Random Bit
Generator by the algorithm is called its Random bit complexity.

2
RECALL THE NOTION OF
INDEPENDENCE

3
 Types of independences

Definition: are said to be mutually independent if

Definition: are said to be pairwise independent if

for every

4
 Types of independences

Definition: are said to be mutually independent random variables if

for any

Definition: are said to be pairwise independent random variables if

for every and every

5
 Important facts

A randomized algorithm typically require random bits/numbers that have

• a uniform distribution
• pairwise independence

 Random bit complexity can be reduced.

Theorem:
We can generate pairwise independent random bits using
only mutually independent random bits.

We shall now prove this theorem.

6
 pairwise independent
random variables

tosses of a fair coin

7
 GENERATING
UNIFORMLY RANDOM AND PAIRWISE INDEPENDENT BITS

using few truly random bits

8
 Generating
Uniformly Random and pairwise independent Bits
Let be mutually independent random bits.
Aim: To generate pairwise independent random bits
Key idea: Generate all non-empty subsets of {}
Ex: Why the XOR operation ? You should
2 1 0
get its answer yourself after a few
0 0 0 slides…
0 0 1 { } 𝑋0
0 1 0 { } 𝑋1
0 1 1 { } 𝑋 1⊕ 𝑋 0
1 0 0 { } 𝑋2
1 0 1 {, } 𝑋 2⊕ 𝑋 0
1 1 0 𝑋 2⊕ 𝑋 1
{, }
1 1 1 {,, } 𝑋 2⊕ 𝑋 1 ⊕ 𝑋 0
9
 Generating
Uniformly Random and pairwise independent Bits
Let be mutually independent random bits.
Aim: To generate pairwise independent random bits

Algorithm:
For to
{
Consider binary representation of ;

 ∙∙∙
Let the bits at places only are ;

10
 Generating
Uniformly Random and pairwise independent Bits
,
Lemma: Each is a uniformly random bit.
Proof: Let = ∙∙∙

𝑋 𝑗 ⊕ 𝑎 2 ⊕ ∙∙∙ ⊕ 𝑎𝑘=1
1

𝟏
¿
𝟐
11
 Generating
Uniformly Random and pairwise independent Bits
,
Lemma: ’s are pairwise independent.
Proof: Let = ∙∙∙ and = ∙∙∙
{, ,…, } ≠ {, ,…, }
Without loss of generality, let {, ,…, }
Let .

𝑷 ( 𝑌 𝑞 =1 ∩𝑌 𝑖=0 ) =¿

𝟏 𝟏
𝑷 (𝐶
𝟐 ∩𝑷
𝐷 ( 𝐶∨𝐷
∨∩𝐵𝐵) ) 𝑷
=?( 𝐷∨𝐵
𝟐 ⋅) ?
𝟏
¿
𝟒
12
 DERANDOMIZATION

a randomized algorithm  a deterministic algorithm

13
 Large cut in a graph

Theorem: (Probabilistic Methods - I)

Let be an undirected graph on vertices and edges.
There exists a cut of size at least .

14
 Large cut in a graph
A randomized algorithm:
∅;
Add each vertex from to randomly independently with probability .
Return the cut defined by .

: size of cut () returned by the randomized algorithm.

E[] =

 There exists an such that

Question: What is the underlying sample space ?

Answer: Depends upon the “random bits” used by the algorithm.

15
 Large cut in a graph
Question: How to de-randomize the algorithm ?
Answer: Compute cut associated with each and return the largest.
Question: How many random bits does the algorithm require ?
Answer:

Question: If we use mutually independent bits for all vertices, what is the size of ?
Answer: .

Question: Do we really need mutually independent bits for all vertices ?

Answer: NO 
IDEA : Use only pairwise independent random bits.
But will it still ensure E[] = ? Let us see …

16
 Large cut in a graph
: the pairwise independent random variable for each vertex.
: size of cut () returned by the randomized algorithm.
E[] = ??


 E[]

𝟏 𝟏
𝟒 𝟒
𝒎
¿
𝟐
17
 Large cut in a graph
Lemma: If we use only pairwise independent random bits,
the expected size of cut will be at least .

Question: How many truly random bits does the algorithm require now ?
Answer:

Question: What is the size of now ?

Answer: O().

Deterministic algorithm:
Just enumerate cuts associated with each and report the largest one.
Running time: O()

18
 Large cut in a graph

Theroem: There is an O() time deterministic algorithm

that computes a cut of size at least
in a graph having edges and vertices.

19
 DERANDOMIZATION

using conditional expectation

20
 Problem 1: Large cut in a graph
Problem: Let be an undirected graph on vertices and edges.
Compute a cut of size at least .

A randomized algorithm:
∅; ∅;
For each vertex
Add to or randomly with probability independent of other vertices
return the cut defined by .

: size of cut () returned by the randomized algorithm.

E[] =
Question: How to deterministically compute a cut of size in () time?

A simple application of conditional expectation 

21
Problem 2: Approximate Distance Oracles
Problem: Let be an undirected graph on vertices and edges.
Compute a 3-approximate distance oracle of size .

A randomized algorithm:
∅;
Add each vertex from to randomly independently with probability .
for each , compute Ball(, )
for each , compute distance to all vertices.

: returned by the randomized algorithm.

E[] =

Question: How to deterministically compute a 3-approximate distance oracle of size

O() ?

A non-trivial application of conditional expectation (published in ICALP 2005)

22
 Problem 3: Min-Cut
Problem: Let be an undirected graph on vertices and edges.
Compute minimum cut of .

Randomized algorithmMin-cut():
{ Repeat times
{ Let ;
 Contract(). }
return the edges of multi-graph ;
}

Theorem: The algorithm computes a min-cut with probability at least .

Question: How to deterministically compute a min-cut in time ?

No idea whether we can use conditional expectation ?

23
 Large cut in a graph

A randomized algorithm:
∅; ∅;
For each vertex
Add to or randomly with probability independent of other vertices
return the cut defined by .

𝒗 𝟏 𝒗𝟐 𝒗𝟑 … 𝒗𝒊 𝒗 𝒊+𝟏 𝒗𝒏

24
 Notations:
For a given graph , and ,

:
set of all edges from that have as one of the endpoint.

:
set of all edges from that have at least one end point in .

:
set of all edges from with one endpoint in and another in .

:
set of all edges from with one endpoint and another endpoint in .

25
 Notations:

: random variable denoting the number of edges in a cut output by the algorithm.

: random variable taking value 1 if and 0 otherwise

: {, , …, }

: {, , …, } where for .

means “”

26
 CONDITIONAL EXPECTATION

Make sure you understand “Conditional expectation” before using it.

So try to focus on the following slide.

27
 𝑽𝒊

𝒗 𝟏 𝒗𝟐 𝒗𝟑 … 𝒗𝒊 𝒗 𝒊+𝟏 𝒗𝒏

𝑨 𝑩
28
 𝑽𝒊

𝒗 𝟏 𝒗𝟐 𝒗𝟑 … 𝒗𝒊 𝒗 𝒊+𝟏 𝒗𝒏

|| +
𝑩
| 𝑽
𝑨
𝒊
𝑽𝒊

…
…

𝑨 𝑩
29
DERANDOMIZATION USING
CONDITIONAL EXPECTATION

30
 The Binary tree associated with the
Randomized algorithm
𝐄[ 𝒁 ]
= 𝟏 𝒙 =
𝒙𝟏 𝟏 𝟎
𝐄 [ 𝒁|𝒙 𝟏=𝟏 ] 𝐄𝒙[ 𝒁|𝒙 𝟏=𝟎 ]
𝒙
𝟏 𝟐= 𝟏 𝟐 =𝟎
= 𝟎 =
𝐄 [ 𝒁|𝒙 𝟐=𝟏 , 𝒙𝟏=𝟏𝒙 ]𝟐 𝒙 𝟐

𝒏
…

A cut of value
31
 Role of conditional expectation

= +

Either
or

In general,


Either
or

32
 Using Conditional expectation

We wish to make choices for ’s such that

…
IDEA:
Given that , choose that value for such that

33
= || + |

= ??
|| + || + |
= ??
|| + || + |

Question: Should we assign to or to ?

 Assign to if || ||

34
 Making Choice for
𝑽𝒊

𝒗 𝟏 𝒗𝟐 𝒗𝟑 … 𝒗𝒊 𝒗 𝒊+𝟏 𝒗𝒏

𝑩
𝑽
𝑨
𝒊
𝑽𝒊

…
…

𝒗 𝒊+𝟏 𝒗 𝒊+𝟏

𝑨 𝑩
35
 Deterministic algorithm for Large cut
Input: = ()
∅; ∅;
For each vertex • This was a simple example of using
{ if ||> || conditional expectation to
derandomize a randomized
Add to ; algorithm. But it conveys the crux
else of this powerful method. In order to
use it to derandomize any other
Add to ; algorithm, all you might need is
} creative and analytical skills.
• Also remember, we can not hope to
return the cut defined by . derandomize every randomized
algorithm. But if it is possible to
derandomize an algorithm,
Time Complexity: O(). conditional expectation may prove
to be a very useful tool.
Theorem: There is a deterministic O() time algorithm to compute a cut of size at least
in any given undirected graph.

36