1.

The study of the fundamental limits to computation

↑
assuming unbounded but finite consumpution of resources
— time , space and processing power
2. Various models of computation

λ − calculus Register machines P ost system

church − T uring thesis

T uring machines recursive f unctions

3. Abstraction : Generative model vs machine models

regular context free context− unrestricted

languages languages sensitive languages
languages
&
regular expressions & & &
&
pushdown linear bounded turing
finite state automata automata automata machines

STRING & SETS

Decision problems vs functions
A decision problem is a boolean-valued function. Set theorectically a decision problem is specified by a
set A of all possible inputs and the subset B ⊆ A for which the function is true.

Strings. The set of all possible inputs to a decision problem may be encoded as the set of finite-length
strings over some fixed finite alphabet. All data types – numbers , graphs , trees , programs etc — can
all be encoded naturally as strings. So by using the string data type we need to consider only one data
type with a few basic operations.

Definition & Notations

• An alphabet is any finite set .(suitably chosen).

• The elements of an alphabet are called letters .
• A string over an alphabet Σ is any finite-length sequence of letters from Σ typically u, v, w, x, y, z
are string over Σ. |x| denotes the length of string x.
• There is a unique 0-length string over Σ called the null string or the empty string denoted by ε.
|ε| = 0.

Further notations for any a ∈ Σ

1
 df df
• a0 = ε an+1 = aan = an a
• Σ⋆ is the set of all (finite - length) string over Σ.
• If Σ = φ then Σ⋆ = {ε} by [Link] Σ 6== φ then Σ⋆ is an infinite set of strings.

Operations on strings over Σ

• For x, y ∈ Σ⋆ , x.y is the string obtained by juxtaposing y to the right of x. This is the operations
of (con)catenation. Often the ′ .′ is omitted
• (Con)catenation is
(i) associative : x(y z) = (x y)z
(ii) ε is the identity : εx = xε = x.
(iii) |xy| = |x| + |y|. for x = am and y = an we have xy = am an = am+n . for all m, n ≥ 0.
• h Σ⋆ , . , ε i is a monoid.
• For any x ∈ Σ⋆ ,
df df
x0 = ε xn+1 = xn . x = [Link]
• For a ∈ Σ and a ∈ Σ⋆ ,
#a(x) is the number of occurrences of ′ a′ in x.

The prefix ordering and prefixes

Definition 1. x ∈ Σ⋆ is a prefix of y ∈ Σ⋆ if there exists a string u ∈ Σ⋆ such that x.u = y. x 4 y
denotes that x is a prefix of y.

Facts about prefixes

• ε 4 x for every x ∈ Σ⋆
• x 4 x for every x ∈ Σ⋆
• x 4 y and y 4 x ⇒ x = y for all x, y ∈ Σ⋆
• x 4 y and y 4 z ⇒ x4z
• 4 is a partial order on Σ⋆ .

Definition 2. x is a proper prefix of y (denoted x ≺ y) if x 4 y and x 6= y.

• x≺y iff ∃u 6= ε : x.u = y.

Problems on strings
1. Consider the set T of trees whose nodes are labelled by letters ′ a′ to ′ z ′ such that more than one
node in a tree may have the same letter labelling it. In particular , consider the following tree. It is
necessary to represent this tree as a string. Now answer the following.

(a) What is Σ ? Give an intuitive expalanation of the letters in the alphabet Σ that you will use
to represent trees such as the above.

2
 λ − calculus Register machines P ost system

church − T uring thesis

T uring machines recursive f unctions

(b) What is the representation of the above tree as a string in Σ⋆ ?

(c) Can you give reasons or show that

(i) Every tree in T has a unique string representation i.e no two distinct trees have the same string
representation.
(ii) There are strings in Σ⋆ which do not represent a tree in T .

2. A string x ∈ Σ⋆ is called a substring of another string y ∈ Σ⋆ if for some u, v ∈ Σ⋆ , y = u.x.v

(a) Let x ≤ y if x is a substring of y. Prove that ≤ is a partial order on Σ⋆ .
(b) Show that x 4 y implies x ≤ y.

3. Let L be a set of labels on nodes of a graph. Give a scheme to represent both directed and undi-
rected graphs as strings. Is it possible for the same graph to have different strings representations
?

4. If a structure h A , . i with an associative binary product operations . and closed under . has a left
identity and a right identity then it has a unique element which is both the left identity and the
left right identity.
P ⋆
Sets of string on — The structure h 2Σ , ∪, ∩, • φ, Σ⋆ , {ε} i

• A, B, C denote subsets of Σ⋆ and are called languages.

• Besides the usual set operations like ∪, ∩, \ and ∼ (complementation) we may extend the operation
of concatenation to sets of strings
• A.B = {x.y | x ∈ A, y ∈ B}
df df
• A0 = {ε} An+1 = An .A = [Link]
↑
This ensures that Am .An = Am+n
df [
A⋆ = An
n≥0

Clearly since A0 ⊆ A⋆ we have ε ∈ A⋆ always.

df [
A+ = An = A.A⋆

3
 P
Note : For any a ∈ , a⋆ and a+ will be used as short hand for {a}⋆ and {a}+ respectively.
Similarly for any x ∈ Σ⋆ x⋆ and x+

Properties of set operations on languages

• ∪, ∩, • are all associative operations on languages.

• ∪, ∩ are also commutative (through • is not)
• φ is the identity for ∪
Σ⋆ is the identity for ∩
{ε} is the identity for •
Note P
: φ is the empty language
over whereas {ε} the (nonempty)
language containing the empty string
• φ is the zero of •
i.e i φ.A = φ = A.φ
• ∪ and ∩ distribute over each other
• Catenation distributes over union
A.(B ∪ C) = (A.B) ∪ (A.C)
(A ∪ B)C = (A.C) ∪ (B.C)
• For any indexed family ( possible infinite ) concatenation distrubutes over arbitrary unions.
[ [
A.( Bj ) = [Link]
j∈J j∈J
[ [
( Ai ).B = Ai .B
i∈I i∈I

• The de Morgan laws hold

∼ (A ∪ B) =∼ A∩ ∼ B
∼ (A ∩ B) =∼ A∪ ∼ B
• ⋆ satisfies the following properties
A⋆ .A⋆ = A⋆
(A⋆ )⋆ = A⋆
A⋆ = {ε} ∪ A.A⋆ = {ε} ∪ A+
φ⋆ = {ε}
• Why catenation does not distribute over ∩
Consider languages A, B, C and let
x ≺ x′ for x, x′ ∈ A
and let y ∈ B − C and z ∈ C − B
such that u = x.y = x′ .z
Then clearly u ∈ A.B ∩ A.C
whereas u 6∈ A.(B ∩ C).

4
 FINITE AUTOMATA & REGULAR SETS

Introduction
Consider a basic model of computation which we know machines could perform even the early years of
the 20th century

• vending machines which “accepted ” sequence of coins of appropriate

• looms which could “read ” patterns from a finite paper tape and could replicate the patterns.
• even more recently , tape recorders and gramophone players which could “read” a finite input and
convert them into sounds.
• even more recently programs that can “read” an input file containing a finite sequence of characters
and take appropriate action.

All the above are concrete examples or instances of the following abstraction.

• a finite sequence of discrete inputs

• that is “read” from left to right and
• accepted by a machine with a finite number of possible states.

Diagrammatically
finite length
input sequence

Finite state
control

More abstractly we have the “finite automation” abstraction which is a structure

Definition 3. M = h Q, Σ, δ, q0 , F i a finite-state

automation is a structure where

P
• is a finite alphabet
• Q is a finite set of “states” of the machine
• q0 ∈ Q is the initial state.

• F ⊆ Q is the set of final states

• δ : Q × Σ −→ Q is the transition function which specifies for a given state and input symbol what
the next state is

5
Simple examples

1. Consider a vending machine which accepts only 5-rupee coins and delivers coffee for each such coin.
We forget about the coffee and specify it as

δ Σ 5
Q

q0 q0

Σ = {5}
Q = {q0 }
F = {q0 }
δ(q0 , 5) = q0
2. Consider a machine that does not accept any other coins (say Re1 and Re2).
Σ = {1, 2, 5}
Q = {q0 , q1 }
F = {q0 }

δ Σ 1 2 5
Q
q0 q1 q1 q0
q1 q1 q1 q1

3. Consider a machine that accepts a nonempty sequence of Re1 coins followed by a nonempty sequence
of Re2 coins but does not accept “anything else”
We abstract out the “anything else” by a new symbol ⊥. So we then have

δ Σ
Q
1 2 ⊥

q0 q1 q⊥ q⊥

q1 q1 q2 q⊥

q2 q⊥ q2 q⊥

q⊥ q⊥ q⊥ q⊥

Σ = {1, 2, ⊥}
Q = {q0 , q1 , q2 , q⊥ }
F = {q2 }

6
 Automata may also be represented graphically

2
q1 2
1
q2
⊥
q0
⊥ 1,⊥
final state
q⊥
2
initial state

1,2,⊥

4. Consider a machine that only accepts any sequence conisting of at least 3 one-rupee coins and
assume Σ = {1, 2, 5}

2,5 2,5 2,5 1,2,5

1 1 1
q0 q11 q12 q13

5. Consider a machine that accepts a sequence of coins in which there is at least one sub-sequence of
there consecutive 1s.
2,5
1,2,5

1 1 1
q0 q11 q12 q13

2,5 2,5

6. Consider a machine that accepts a sequence of coins in which there is exactly one sub-sequence of
three consecutive 1s

7
 2,5 2,5

1 1 1
q0 q11 q12 q13

2,5 2,5 1
2,5
2,5
1 1 q21
q23 q22

7. Consider a vending machine which accepts exactly coins of denominations {1, 2, 3} which add
up to 4.
Clearly Σ = {1, 2, 5} and the language accepted by the machine is {14 , 121, 112, 211, 22} let the states
of the machine by numbered/named 0, 1, 2, 3, 4, ⊥ where ⊥ stands for any number > 4

2 2,5
1 3
1

1,2,5
5
⊥

1
1 ,5
2 1,2
1
2 4
2

The number on each state indicates the total value of the sequence of coins input so far.

Given a machine M = h Q, Σ, δ, q0 , F i define δ ⋆ : Q × Σ⋆ −→ Q as the function defined by induction on

the length of the input string

δ ⋆ (q, ε) = q
δ ⋆ (q, xa) = δ(δ ⋆ (q, x), a)

δ ⋆ is the multi-step version of δ.

Definition 4. An automation accepts a string x ∈ Σ⋆ if δ ⋆ (q0 , x) ∈ F and rejects it otherwise. The
language accepted by it is L(M ) = {x ∈ Σ⋆ | δ ⋆ (q0 , x) ∈ F }.
Definition 5. A ⊆ Σ⋆ is regular if A = L(M ) for some DF A M .

FACTS

1. φ and {ε} are regular.

2. Σ⋆ is also regular.

8
 3. The various languages we have defined through the various automata in the preeding examples are
all regular.
⋆
4. If L ⊆ Σ⋆ is regular and Σ′ ⊇ Σ, then L ⊆ Σ′ is still regular.

Number-theoretic examples. Consider an automation that accepts all bit strings that are multiples
of 2. Assume ε represents the number 0.

Clearly here Σ = {0, 1} and the automation accepts the language {ε} ∪ {x 0 | x ∈ Σ⋆ }

δ
Σ
0 1
1 Q
02 12
02 02 12

0 12 02 12
0

Consider a machine that accepts only multiples of 3. In designing this automation we consider the fol-
lowing. For any x ∈ {0, 1}⋆ let (x)2 denote the integer it represents. Then we have

(ε)2 = 0 (0)2 = 0 (1)2 = 1.

and for any x ∈ Σ⋆ and b ∈ Σ

(x b)2 = 2(x)2 + b

The states of the automation are numbered i ∈ {0, 1, 2} and satisfy the preoperty that for any x ∈ Σ⋆
the automation is in state (x)2 mod 3 after consuming x. Then inductively the transition function
δ : {0, 1, 2} × {0, 1} −→ {0, 1, 2} satisfies the property that

if q = (x)2 mod 3 then δ(q, b) = (xb)2 mod 3

= (2(x)2 + b) mod 3
= (2[(x)2 mod 3] + b) mod 3
= (2q + b) mod 3.

Clearly q0 = 0 and F = {0}.

which gives us the following automation

0 δ Σ 0 1
Q
1
1 0 03 03 13
03 13 23 13 23 03

1 0 23 13 23

Question ? How does one design a modulo 6 counter ? there are two possible answers to this question.

9
 1. Follow the usual steps and design a 6-state automation.
2. Do a “product construction” to create the 6-state automation from the modulo 2 and modulo 3
counters.

The Product Construction

Let
M1 = h Q1 , Σ, δ1 , q1 , F1 i
and M2 = h Q1 , Σ, δ2 , q2 , F2 i
be automation defined on a comman alphabet Σ. Their product M3 = M1 × M2 = h Q3 , Σ, δ3 , q3 , F3 i is
the automation defined by

Q3 = Q1 × Q2 = {q1 ′ , q2 ′ | q1 ′ ∈ Q1 , q2 ′ ∈ Q2 }
F3 = F1 × F2
q3 = (q1 , q2 )
δ3 : Q3 × Σ −→ Q3 such that
δ((q1 ′ , q2 ′ ), a) = (δ1 (q1 ′ , a), δ2 (q2 ′ , a))

Example . The product of the automation for the modulo 2 counter and that of the modulo 3 counter
yield an automation that accepts only bit strings that are binary representations of multiples of 6.
Lemma 1. If M3 = M1 × M2 then
δ3⋆ ((q1 ′ , q2 ′ ), ε) = (q1 ′ , q2 ′ )
δ3⋆ ((q1 ′ , q2 ′ ), x a) = δ3 (δ3⋆ ((q1 ′ , q2 ′ ), x), a)
Theorem 1. L(M3 ) = L(M1 ) ∩ L(M2 )
Proof. for all x ∈ Σ⋆
x ∈ L(M3 )
⇔ δ3⋆ ((q1 , q2 ), x) ∈ F3
⇔ (δ1⋆ (q1 , x), (δ2⋆ (q2 , x)) ∈ F1 × F2
⇔ (δ1⋆ (q1 , x) ∈ F1 ∧ (δ2⋆ (q2 , x)) ∈ F2
⇔ x ∈ L(M1 ) ∧ x ∈ L(M2 )
⇔ x ∈ L(M1 ) ∩ L(M2 )

10
Example : A modulo 6 counter .
A direct construction yields.

0
δ 0 1
Σ
0 Q
1
16
06 06 16
06 26 0
0
16 26 36
1
1
0 26 46 56
0 46
1
1
36 06 16
36 56
46 26 36

56 46 56
1

A product construction yields.

0
δ Σ 0 1
1 0 Q
12 13
02 03 02 23 0 02 03 02 03 12 13
0
1
12 13 02 23 12 03
1
0
0 02 13 02 23 02 13 12 23
1
1
12 03 02 03 12 13
12 03 12 23

02 13 02 23 12 03

1 12 23 02 13 12 23

The following bijection holds between the two automata states

1−1
i6 ((i ↔ mod 2)2 , (i mod 3)3 )

Closure Properties of Regular languages

Let R = {A ⊆ Σ⋆ | L is regular}.
Theorem 2. 1. R is closed under set intersection.
— follows from the previous theorem
2. R is closed under complementation
i.e A ∈ R ⇒ ∼ A ∈ R

11
 ∼
Proof. If A ∈ R , then for some DF AM A = L(M ). Consider the complement DF A M = h Q, Σ, δ, q0 , Q − F i
∼
in which every non-accepting state becomes final and vice-versa. Clearly L(M ) = Σ⋆ − L(M ) =∼ L(M ) =∼ A

3. R is closed under union

Proof. Since R is closed under both ∩ and ∼ it follows that for any A, B ∈ R A ∪ B =∼ (∼ A∩ ∼ B)
and hence A ∪ B is regular.

4. R is closed under concatenation

Proof. Let A, B ∈ R be accepted respectively by automata

M = h P, Σ, p0 , δA , F i
N = h Q, Σ, q0 , δB , G i

Tutorial 2 (Warm up)

0) Prove that if Σ 6= φ is finite Σ⋆ is countable.

Answer : Let Σ = {a1 , . . . , an }. Consider the mapping (1-1 and onto) of string from Σ⋆ to numbers in
base n ( without 0) such that ai denotes the digit i in base n. The clearly with ε being mapped to 0
we have that the string in Σ⋆ uniquely represent numbers in base n. Hence since N is countable and
Σ⋆ −−−→ N we have that Σ⋆ is countably infinite.
1−1
onto

Tutorial 2

1. Given Σ is there a language that is not regular ?

Answer : Consider Σ = {a, b} then the following languages are not regular
a) {an bn | n > 0}
b) {x ∈ Σ⋆ | ∀u 4 x : #a(u) ≥ #b(u) ∧
#a(x) = #b(x)}
c) If Σ = 0, 1 and P is the set of strings (without leading zeroes ) representing primes then P is not
regular

P = {x ∈ 1.Σ⋆ | (x)2 is a prime}.

2. Given a regular language A ⊆ Σ⋆ and two machines M1 and M2 such that L (M1 ) = L (M2 ) = A.
Then is there an isomorphism between them ?
Answer : Not necessary as we shall in the minimization of DF As. But a concrete example is the
following. Consider a different machine to recognize multiples of 2.
Clearly this machine has 3 states and is not isomorphic to the machine

12
 0 1
1
0 0 1

0 0

0 1
1
0 1

3. Prove without using the product construction or any of its consequences that every finite language
on Σ is regular.

Answer : Consider a finite alphabet Σ = {a1 , . . . , am } if n = 0 then clearly every language on Σ is

φ since Σ = φ
1) Even if Σ 6= φ the language A could be empty. The empty language is accepted by any automation
which has no final states
Σ

A=φ q0

If A is not empty then consider the following case A = {ε}. This language is accepted by the
automation.

13
 Σ
Σ
q1
q0

A 6= {ε} Intially we consider the case when A 6= φ and ε 6∈ A. If ε ∈ A we make the initial state
final. Define for each ai ∈ Σ , Ai = {x ∈ Σ⋆ | ai x ∈ A}. Also define M AX(A) = max{|x|x ∈ A}.
Basis M AX(A) = 0. Then A = {ε} and we have already constructed the required automation.
IH : Assume for all B M AX(B) < n for some n > 0 it is true that B is regular

Induction step. Let M AX(A) = n > 0.

[
Clearly A = ai .Ai . In certain cases it is possible that Ai = φ. (when ai is not the prefix of
1≤i≤m
any string in A.). Also in certain cases when ai ∈ A , Ai = {ε}.
But since A is finite it is clear that each Ai is also finite and further M AX(Ai ) < n for each Ai .
By the induction hypothesis there exists a machine Mi which accepts Ai .
Let Mi = h Qi , Σ, δi , qi0 , Fi i for each 1 ≤ i ≤ m.
Assume for all i 6= j Qi ∩ Qj = φ.
Construct the machine

M = h Q, Σ, δ, q0 , F i as follows :
[
Q= Qi ∪ {q0 } ∪ {qai | Ai = φ}
1≤i<m
S
where q0 , qa1 , . . . , qam are all mutually distinct and also different from each state in Qi
[
F = Fi and δ is defined as follows :
1≤i≤m

for every qi ∈ Qi , δ(qi , a) = δi (qi , a) for each a ∈ Σ

(
q0 if Ai 6= φ
δ(q0 , ai ) =
qai if Ai = φ
and δ(qai , a) = qai for all a ∈ Σ.

NONDETERMINISTIC FINITE AUTOMATA

Introduction And Motivation

Intuitively it seems obvious that a good way to prove that the union of two regular languages is regular is
to define a DF A that is obtained by combining the capbilities of the individual DF As of the respective
languages.

Example. In particular , we would like to “connect (the two DF As) in parallel ” to obtain an automa-
tion which can simulate either of the two behaviours.

14
i) Consider an automation that accepts bit strings which are either multiples of 2 or multiples of 3. The
resulting language is the union of the two languages.

ii) Consider a vending machine that

a) accepts a single 5-rupee coin and delivers coffee or
b) accepts any sequence of 1-rupee and 2-rupee coins that add up to 4 and delivers tea.

It should be possible to combine the two vending machine to get a single vending machine which delivers
coffee or tea according to the money put in. The only thing required is a mechanism of combining the
two vending machine and deciding as soons as a coin is put it whether to trigger the first machine or the
second one.

Deciding
mechanism

p0 q0
Coffee
machine Tea
pc machine q4

Coffee Tea

A simple way to combine the two machines is as follows. Define the machine P Q with initial state pq0

δP Q (pq0 , 5) = pc for every state in P − {p0 }

δP Q (pq0 , 1) = q1 ∪ Q − {q0 } define
(
δP Q (pq0 , 2) = a2 δP (p, a) if pq ∈ P
δP Q (pq, a) =
δQ (pq, a) if pq ∈ Q
In general combining two machines M1 and M2 which accept languages A, B ⊆ Σ⋆ bring in the union of
the following disjoint sets

1) A − B
2) B − A
3) A ∩ B

In the case if 1 and 2 the decision as to which machine to start is [Link] the question of which
machine to run in the case of x ∈ A ∩ B requires certain external decision-making which we abstract out
as non-determinism.

Having abstracted this out , it also requires that we relax the condition of acceptance or rejection as
follows :
x is accepted by the combined machine if it is accepted by at least one of them
x is rejected by the combined machine only if it is rejected by both of them

Hence there should be a simple way of combining machines to produce machines that accepts unions of
languages such that the decisions that accept unions of languages such that the decisions mechananism
is abstracted away and left to the implementor. To this end we define a “nondeterministic” automation
as follows :
Definition 6. A nonderministic finite automation (NFA) is a structure N = h Q, Σ, △, S, F i where

15
φ 6= S ⊆ Q △ : Q × Σ → 2Q , △ is the transition relation. The function △⋆ : 2Q × Σ⋆ → 2Q is the
natural extension of the function △ and is defined by the rules.
df
△⋆ (T, ε) = T
df
△⋆ (T, ax) = △⋆ (△(T, a), x)
where for any set S ⊆ Q [
△(T, a) = △(q, a)
q∈T

An NFA N accepts x ∈ Σ⋆ if △⋆ (q0 , x) ∩ F 6= φ and it accepts a language A ⊆ Σ⋆ if it can accepts every

string in A.

Intuitively a NFA accepts a string if there exists a path from one of the start states to one of the final
states and it rejects a string only if there is no path from any start state to any final state. As usual
L (N ) denotes the language accepted by N.

Facts 1. Every DFA M = h Q, Σ, δ, q0 , F i is equivalent to the NFA N = h Q, Σ, δ, {q0 }, F i with

df
△(q, a) = {δ(q, a)}.
Lemma 2. For any x, y, ∈ Σ⋆ and T ⊆ Q
△⋆ (T, x.y) = △⋆ (△⋆ (T, x), y)
Proof. By induction on |x|.
Basis |x| = 0 i.e. x = ε. Then
△⋆ (T, εy) = △⋆ (T, y) = △⋆ (△⋆ (T, ε), y).
Induction step : Assume

IH. for all |x′ | ≤ n all T ⊆ Q for some n > 0 ,

△⋆ (T, x′ .y) = △⋆ (△⋆ (T, x), y)

Consider x = ax′ with |x| = n + 1.

Then
△⋆ (T, ax′ y) = △⋆ (△(T, a), x′ y)
′
let T = △(T, a) = △⋆ (T ′ , x′ y)
By IH we have = △⋆ (△⋆ (T ′ , x′ ), y)
= △⋆ (△⋆ (△(T, a), x′ ), y)
⋆
By def of △ = △⋆ (△⋆ (T, x), y).
Lemma 3. The function △⋆ commutes with set union i.e. for any family {Ti ⊆ Q | i ∈ I} indexed by
the set I.
[ [
△⋆ ( Ti , x = △⋆ (Ti , x)
1∈I i∈I

Proof. Again by induction

[ on |x|. [
Basis x ∈ ε. Then △⋆ ( Ti , ε) = △⋆ (Ti , ε)
i∈I i∈I
′
Induction step Let x [
= ax with |x| > 0
△⋆ ( Ti , ax′ )
i∈I [
= △⋆ (△( Ti , a), x′ )
i∈I

16
 [
let Ti = T = △⋆ (△(T, a), x′ )
i∈I [
= △⋆ ( △(q, a), x′ )
q∈T
[ [
= △⋆ ( △(qi , a), x′ )
q∈I qi ∈Ti
[ [
By IH = △ (⋆
△(qi , a), x′ )
i∈I qi ∈Ti
[
= △⋆ (△(Ti , a), x′ )
i∈I
[
= △⋆ (Ti , ax′ )
i∈I
[
= △⋆ (Ti , x)
i∈I

The Subset Construction

Theorem 3. A NFA accepts a language A ⊆ Σ⋆ iif A is regular. Equivalently a N F A accepts A ⊆ Σ⋆
iff there exists a DFA that accepts A.
Proof. (⇐) Assume A ⊆ Σ⋆ is regular. Then there exists a DFA D = h QD , Σ, δD , qDo , FD i that accepts
A. Define N = h QD , Σ, △, {qDo }, FD i with △({qD }, a) = δ(qD , a).

(⇒) The proof of this part requires the construction of a DF A that accepts the same language.

Let N = h QN , Σ, △, SN , FN i be a N F A.
Now construct a DF A
D = h QD , Σ, δ, qDo , FD i as follows .
df
QD = 2QN i.e. each state of the DF A represents a set of N F A state.
δD (qD , a) = △N (qD , a)
[
= △N {qN }, a)
qN ∈qD

qDo = SN
FD = {T ⊆ QN | T ∩ FN 6= φ}

Lemma 4. For every T ⊆ QN and x ∈ Σ⋆

⋆
δD (T, x) = △⋆N (T, x)
Proof. By induction on |x|.
for x = ε we have
⋆
δD (T, ε) = T = △⋆N (T, ε).
for x = ax′ we have
⋆
δD ⋆
(T, x) = δD (T, ax′ )
= δD (δD (T, a), x′ )
⋆
⋆
= δD (△⋆N (T, a), x′ )
By IH = △N (△N (T, a), x′ )
⋆

= △⋆N (T, ax′ )

= △⋆N (T, x)

17
Claim 1. D and N accepts the same language.
Proof. For any x ∈ Σ⋆
x ∈ L (D)
⋆
⇔ δD (qDo , x) ∈ FD
⇔ △⋆N (SN , x) ∈ FD
⇔ △⋆N (SN , x) ∩ FN 6= φ
⇔ x ∈ L (N )
End of proof of the subset construction theorem

ε-Moves : A Natural Extension of N F A

Clearly for any state q ∈ Q of a N F A and letter a ∈ Σ , we have allowed △(q, a) to be either empty , a
singleton or a set of many states. A natural extension as in the case of the vending machines is to use a
non-deterministic mechansim to choose the correct vending machine which will accepts the sequence of
coins. This extension is called a N F A with ε-moves.
Definition 7. A N F A with ε-moves is an N F A
N = h QN , Σ, △N , SN , FN i
with all components as before excepts that
△N : QN × (Σ ∪ {ε}) → 2Q
The intention is that now it is possible for a state transition to occur without the consumption of an input.
In particular now △⋆ (T, ε) = T does not necessarily hold since there may be several states in T which
may allow silent moves.

Fact
Any N F A is also an N F A with ε-moves. Typically △N (T, ε) = φ in the case of a N F A without ε-moves.
Definition 8. Let q ∈ QN be a state of a N F A with ε-moves. The ε − closure of q (denoted ε⋆ (q)) is
the set of states reachable from q only by ε-moves and is defined inductively as follows :
ε0 (q) = q
′′ ′ ′′ ′
εn+1 (q) = n n
[ε (q) ∪ {q ∈ QN | ∃q ∈ ε (q) : q ∈ △n (q , ε)}
⋆ n
ε (q) = ε (q)
n≥0
We denote the fact that q ′ ∈ ε⋆ (q) alternately by q ⇒ q ′ to mean that q ′ is reachable from q by only a
sequence of ε−moves. Note : q ⇒ q holds for any state
For any set T ⊆ QN we exented this by
[
ε⋆ (T ) = ε⋆ (q).
q∈T

In other words q ′ ∈ ε⋆ (T ) iff ∃q ∈ T : q ⇒ q ′

⋆
Definition 9. Let △εN : 2QN × Σ⋆ → 2QN be defined as
⋆
△εN (T, ε) = ε⋆ (T ) = {q ′ | ∃q ∈ T : q ⇒ q ′ }.
⋆ a ⋆
△εN (T, ax) = {q ∈ QN | ∃q ∈ T : ∃q1 , q2 ∈ QN : q0 ⇒ q1 → q2 ⇒ q2 ∧ q ∈ △εN (q2 , x)}.
⋆
Note. △⋆N (q, ε) = q whereas △εN (q, ε) = ε⋆ (q)

• Regular Expressions : Representation of regular languages.

• Building N F Aε from regular expression.

18
Theorem 4. Every regular expression represents a regular language.
Proof. By induction on the structure of regular expressions.
Basis. φ denotes the empty language and is accepted by a DF A/N F A with no final states.

ε is accepted by q0 which is a N F A.

a ∈ Σ is accepted by q0 a qf

? is matched by any symbol of Σ obtained by a finite union of the N F A′ s for each a ∈ Σ

@ is any string in Σ⋆ is accepted by q0

19
Induction Step

α∪β

ε
ε Nα
q0 qf
ε
ε
Nβ

αβ

ε
Nα Nβ

α⋆
ε
ε ε

Theorem 5. For every N F Aε there exists a N F A without ε-moves which accepts the same language.
Proof. Let Nε = h Qε , Σ, δε , Sε , Fε i be a N F A with ε-moves Construct the N F A N = h Qε , Σ, δ, S, F i
without ε-moves as follows.
For every q ∈ Qε and a ∈ Σ
a a
q −→ q ′ δ iff ∃q1 , q2 : q ⇒ q1 −→ε q2 ⇒ q ′ in Nε .
Claim 2. N and Nε accept the same language.
Proof. x ∈ L (Nε ) with x = a1 . . . an .
a a an
⇔ ∃q0 ∈ S ∃qf ∈ F : q0 ⇒1 ε q1 ⇒2 ε . . . ⇒ ε qf in Nε
a1 a2 an
⇔ ∃q0 ∈ S ∃qf ∈ F : q0 −→ q
ε 1 −→ ε . . . −→ ε qf in Nε

Theorem 6. Every regular language has a regular expression representation. We have already proved
the following viz :

1. Every regular expression represents a regular language where the regular expressions were defined
by the following
0) For every a ∈ Σ , a is a RE and L (a) = {a}.
1) ε and φ are regular expression and L (ε) = {ε}
L (φ) = φ
2) For regular expressions α, β.
α.β is a regular expression representing L (α).L β
α ∪ β is a regular expression representing L (α) ∪ L β

20
 3) For any regular expression α,
(α) is a regular expression representing L (α)
α⋆ is a regular expression representing (L (α))⋆
∼ α is a regular expression representing ∼ L (α)

Precedence of Operators

⋆ is the highest ∪ is the lowest

• precedes ∪ but is lower than any unary operator
Theorem 7. For every regular language A ⊆ Σ⋆ , there exists a regular expression 0000 with L (α) = A.
Proof. Consider a regular language A and same N F A N = h Q, Σ, △, S, F i which accepts A.
1
Construction of a regular expression αX
p Q for any X ⊆ Q and states p, q ∈ Q such that
X ⋆ x
L (αpq ) = {x ∈ Σ | p −→ q with all intermediate states in X}
↑

If x = a1 a2 . . . am then ∃r1 , r2 , . . . , rm−1 ∈ X :

a1 a2 am−1 am
p −→ r1 −→ r2 −→ . . . −→ rm −→ q.

1 p, q may or amy not be in X

21
By induction on the size of X
a
Basis X = φ. Let a1 , a2 , . . . , ak ∈ Σ be all the letters such that p ⇒i q for 1 ≤ i ≤ k. Then
(
φ df a1 ∪ a2 ∪ · · · ∪ ak . if k>0
if p 6= q αpq =
φ if k = 0.

(
df a1 ∪ a2 ∪ · · · ∪ ak ∪ ε if k>0
else if p = q, αφpq =
ε if k = 0.

Induction step X 6= φ

Choose any r ∈ X . Let Y = X − {r}

df
Then αX Y Y Y ⋆ Y
pq = αpq ∪ αpr (α⇒ ) αrq

Hence by this induction process for each start state sıS and final state f ∈ F we have an expression αQ sf
[ Q
which represents the set of all paths from s to f in N . Hence αsf is the regular expression denoting
s∈S
the language accepted by N .
Note. This theorem essentially tells us that the language REΣ of regular expressions over Σ is complete
and that it is always possible to convert a DF A or a N F A into a regular expression.

Kleene Algebra of Regular Expressions

+ is associative & commuatative
φ is the identity element for +
+ is idempotent

• is associative
ε is the identity for •
φ is the zero for •
• distributes over + (both left & right)

α⋆ = ε + αα⋆ = ε + α⋆ α

The equation x = αx + β has the least solution α⋆ β

Better still define ≤ on regular expression such that

α ≤ β iff α + β = β

and have the two influence rules

α γ + β ≤ γ ⇒ α⋆ β ≤ γ
and γ α + β ≤ γ ⇒ αα β ≤ γ
The story so far
From regular expressions to regular grammars

22
 ⋆
2Σ
RLGΣ

RΣ
L

L
REΣ

N F AΣε
EXPRESSION
N F AΣ
(REPRESENTATION)

DF AΣ

MACHINES
(ACCEPTANCE)

Consider the regular expression equation

X = aX + b
Whose solution is the regular expression
X = a⋆ b

which may be obtained as the maximal fixpoint under ⊆ of the composition of monotonic functions • an
⋆
d + on the lattice 2Σ

String Generation through Grammars

The generative process may intuitively be looked upon as follows. Consider a typical solution of the
equation

X = A.X ∪ B

on language over Σ⋆ i.e. A, B ⊆ Σ⋆ abd we need to find the unknown X. We may construct an increasing
sequence of sets Xi , i ≥ 0

φ = X0 ⊆ X1 ⊆ . . . Xi ⊆ Xi+q ⊆ . . .

such that

23
 Xi+1 = [Link] ∪ B
[
The maximum solution is the set Xi
i≥0

Essentially we are unfolding the recursion in the eqaution inductively to obtain.

X1 = A.φ ∪ B = B
X2 = A.X1 ∪ B = B ∪ A.B
X3 = A.X2 ∪ B = B ∪ A.B ∪ A.A.B.
..
.
This allows us to “generate” a larger and larger collection of strings from the empty sets. We use this
essentially to define the notion of generation by “orienting” the ‘=’ in a left-to-right fashion.

Grammers & Generation

Definition 10. A right-linear grammar over an alphabet Σ is a structure G = h V, Σ, S, P i where
• V 6= φ is a finite set of variables or nondeterminals
• Σ is finite alphabet with V ∩ Σ = φ.
• S ∈ V is the start symbol of the grammar or rules
• P ⊆f (V ∪ Σ)⋆ × (V ∪ Σ)⋆ is a finite collection of productions or rules such that each production is
of only one of the following forms.
)
(i)A −→ xB
where x ∈ Σ⋆ and A, B ∈ V
(ii)A −→ x

i.e p ⊆f V × Σ⋆ • (V ∪ {ε}

A derivation of a (right linear) grammar is a sequence of strings in (Σ ∪ V )⋆ starting from S. such that

Example. G = h {S}, {a, b}, S, P i where P consists of the

productions S −→ abS and S −→ ε.
Its derivations are of the form S ⇒⋆ (ab)n S ⇒ (ab)n
S ⇒ abS ⇒ ababS ⇒ abababS ⇒ ababab
for any string xA ⇒ yB iff for some z ∈ Σ⋆ , y = xz and A −→ zB ∈ P .
The reflexive - transitive closure of ⇒ is denoted ⇒⋆ and the language generated by a grammar G with
start symbol S is the set

L (G) = { x ∈ Σ⋆ | S ⇒⋆ x }.

i.e x ∈ L (G) iff there exists a derivation of x from S.

In the example above

ε, ab, abab, . . . , (ab)n , · · · ∈ L (G)
In fact L (G) = { (ab)n | n ≥ 0 }
There also exists infinite derivations which do not contribute to the language generated. An infinite
derivation is obtained by never applying the production S −→ ε.
Lemma 5. Any derivation of a right linear grammar is either of the form
S0 ⇒ x1 S1 ⇒ . . . ⇒ x1 x2 . . . xi Si ⇒ . . .
or S0 ⇒ x1 S1 ⇒ . . . ⇒ x1 x2 . . . xi Si ⇒ . . . ⇒ x1 x2 . . . xn−1 Sn−1 ⇒ x1 x2 . . . xn−1 xn
where S0 , S1 , . . . , ∈ V and x1 , x2 , · · · ∈ Σ⋆ .

24
Theorem 8. If G is a right linear grammer then L (G) is a regular language.
Proof. Let G = hV, Σ, S0 , P i be a right linear grammar with V = {S0 , S1 , . . . , Sm } with a finite collection
of productions

P = { Si −→ xk Sj | Sj ∈ V ∪ {ε} , Si ∈ V , xk ∈ Σ⋆ , 1 ≤ k ≤ n }

If y ∈ L (G) then there exists a derivation S0 ⇒⋆ y of the form

S0 ⇒ xi1 Si1 ⇒ xi1 xi2 Si2 ⇒ . . . ⇒ xi1 xi2 . . . xip = y

with {Si1 , Si2 , . . . , Sip } ⊆ V.

We construct an automation (N F A) with a start state labelled S0 . For each production

Si −→ a1 . . . al Sj where Sj ∈ V ∪ {ε}

Construct states Sij , Si2 , . . . , Sil−1 , Sil such that

aj
Sij −→ Sij+1 for 1 ≤ r ≤ l

is a transition in △. If Sj 6= ε then Sij = Sj otherwise Sil is a new final state. Clearly for each such
production.
We have the following claim

Si −→ a1 . . . al Sil for Sil ∈ V ∪ {ε} iff △⋆ (Si , a1 . . . al ) = Sil

The following claim may then be proven.

S0 ⇒0 iff △⋆ (S0 , y) ∈ F

where F is the collection of final states of the automation.

It follows therefore that the language generated by a right linear grammar is also the language accepted
by some N F A and hence must be regular.
Theorem 9. Every regular language over Σ may be generated by a right linear grammar.
Proof. Let A ∈ RΣ . Then there exists a DF A

D = h Q, Σ, δ, q0 , F i

which accepts A. Construct a grammar

G = h V, Σ, q0 , P i where V = Q

with
qi −→ aqj ∈ iff δ(qi , a) = qj

and qi −→ ε ∈ P iff qi ∈ F

Claim 3. A is the langauge generated by G.

25
Proof. Every derivation in G is of the form
q0 ⇒ a1 q1 ⇒ a1 a2 q2 ⇒ . . . ⇒ a1 a2 . . . an qn ⇒ a1 . . . an where qn −→ ε ∈ P . By the construction of the
grammar it follows that δ ⋆ (q0 , a1 . . . an ) = qn ∈ F
Similarly for every string a1 . . . an ∈ L (D) we have a collection of states q0 , q1 , . . . , qn ∈ Q with qn ∈ F
such that δ ⋆ (q0 , a1 . . . an ) = qn . Correspondingly we have the derivation

q0 ⇒ a1 q1 ⇒ a1 a2 q2 ⇒ . . . ⇒ a1 a2 . . . an qn ⇒ a1 a2 . . . an .
The two theorem proved above give us the following characterization
A language L ⊆ Σ⋆ is regular iff
there exists a right-linear grammar
which generates L

Analogous to the notation of a right - linear grammar is the notion of a left-linear grammar in which
all the productions are of the form S −→ T x where T ∈ V ∪ {ε} and x ∈ Σ⋆ . Both right-linear and
left-linear grammars have the same power of generation. But to prove that left-linear grammars generate
only regular languages we need the following lemma.
Lemma 6. If A ⊆ Σ⋆ is regular them so is AR where AR = {xR |x ∈ Σ⋆ } and for any string x = a1 . . . am .
xR = am . . . a1 is the reverse of x.
Proof. Since A is regular there exists a N F A

N = h Q, Σ, △, S, F i with L (N ) = A.

Consider the N F A

N R = h Q, Σ, △R , F, S i

constructed from N by
(i) making S the set of final states
(ii) making F the set of start states
(iii) taking the universe of the relation △ as the
transition relation.
a a
i.e q −→ q ′ ∈ △ iff q ′ −→ q ∈ △R
The result then follows from the following claim
Claim 4. For all q0 ∈ S and qf ∈ F and x ∈ Σ⋆

qf ∈ △⋆ (q0 , x) iff q0 ∈ △⋆ (qf , xR )

Proof. By induction on the length of x.
Corollary 1. A language A ⊆ Σ⋆ is regular iff AR is regular iff AR is regular
Proof. Follows from the fact that A = (AR )R .
We are now ready to prove our main theorem
Theorem 10. A language is regular iff there exists a left-linear grammar that accepts it.
Proof. We prove that for every left-linear grammar there exists a corresponding right linear grammar
which generates the reverse language and vice-versa.
Let GL be the class of left-linear grammars over Σ and GR the class of right-linear grammars. Define the
following 1 − 1 correspondence.

26
 f : GL −→ GR

such that

GL = h VL , Σ, SL , PL i h VR , Σ, SR , PR i

f (GL ) = GR iff
VL = VR , SL = SR and for all T ∈ VL = VR
and ∪ ∈ VL ∪ {ε} = VR ∪ {ε} and x ∈ Σ⋆
T −→ x ∪ ∈ PR iff T −→ ∪xR ∈ PR .

Claim 5. x ∈ L (GL ) iff xR ∈ L (GL ) for all x ∈ Σ⋆ and GR , GL such that GR = f (GL )
Proof. By induction on the length of x.

Main proof

(⇒) If A ⊆ Σ⋆ is regular then AR is also regular and there exists GR which generates AR . From the
claim above f −1 (GR ) = GL generates (AR ) = A. Hence every regular language may be generated by a
left-linear grammar.
(⇐) Let GL be any left-linear grammar. Then GR = f (GL ) is a right-linear grammar that generates
(L (GL ))R . But then (L (GL ))R must be regular which implies L (GL ) is also regular.
However we need to emphasize that a grammar in which there are both left-linear as well as right linear
productions may generate a language that is not regular as the following example shows

Example G = h {S, A, B}, {a, b}, S, P i where S −→ A , A −→ aB , A −→ ε and B −→ Ab are the

productions generates the language
m m
S ⇒ A ⇒ ε{a b | m≥0}
S ⇒ A . . ⇒ am Abm ⇒ am bm}
| ⇒ aB ⇒ aAb ⇒ .{z
2m

The Pumping Lemma

Theorem 11. Let A ⊆ Σ⋆ be a regular language. Then there exists m > 0 such that for any x ∈ A with
|x| ≥ m x may be decomposed into x = uvw with |uv| ≤ m and |v| > 0 such that xi = uv i w ∈ A for every
i ≥ 0.

A ⊆ Σ⋆ is regular
⇒ ∃m > 0 : ∀x ∈ A : |x| ≥ m ⇒
∃ u, v, w : x = uvw ∧ |uv| ≤ m ∧ |u| > 0
∧ ∀k ≥ 0 : xk = uv k w ∈ A.

Proof. If A is finite then choose m > max{|x| |x ∈ A} and the theorem holds vacuously. So assume A is
infinite.
Claim 6. If A is infinite then for each n ≥ 0 , there exists x ∈ A such that |x| > n.
Proof. Suppose not. Then for some value of n ≥ 0. every
[ string in x ∈ A has a length ≤ n. But for any
k ≥ 0 and finite Σ , Σk is a finite set which implies Σk is also finite. In fact Σk has exactly |Σ|k
k≤n
[ X
k k
different strings and | Σ |= |Σ| .
k≤n k≤n

27
The contrapositive of the pumping lemma states that

∀m > 0 : ∃x ∈ A : |x| ≥ m ∧
∀u, v, w : x = u v w ∧ |uv| ≤ m ∧ |v| > 0
⇒ ∃ k ≥ 0 : xk = uv k w 6∈ A
⇒ A is not regular.

CHALLENGER’S STRATEGY
1. x = am bm
2. k = 0

Example A = {an bn |n ≥ 0} is not regular. Assume A is regular. Therefore the pumping lemma there
must hold and A is infinite. Since A is regular there must be a DF A D which accepts A. Suppose the
DF A has N states. Choose m = N + 1. Consider the string am bm and the sequence of states
a a a a b b b
q0 −→ q1 −→ . . . −→ qN −→ qm −→ qm+1 −→ . . . −→ q2m

Clearly for some i < j < N , qi = qj

Choose u = ai , v = aj−i , w = am−j bm . so that x = ai aj−i bm−j bm = uvw. Hence every xk = ai (aj−i )k am−j bm ∈ A.
In particular x0 = ai am−j bm = am−j+i bm ∈ A. But since j 6= i , m − j + i 6= m. Hence x0 6∈ A which is
a contradiction. Hence the assumption that A is regular is wrong.

The proof proper Since A is regular there exists a DF A D = h Q, Σ, δ, q0 , F i which accepts A. Let
|Q| = n. Since A may be assumed to be infinite it does have strings of length > n + 1. Choose any such
string |x| ≥ n + 1. Then δ ⋆ (q0 , x) = qf ∈ F consider the sequence of states in the DF A which lead to qf
from q0 . This is a sequence of n + 2 states.

q0 , q1 , . . . , qn−1 , qn , qf

By the pigeon hole principle there exists i < j ≤ n such that qi = qj . This implies that x = uvw such
that

|uv| ≤ n + 1 = m
{δ ⋆ (q0 , u) = qi δ ⋆ (qi , v) = qi = qj δ ⋆ (qj , w) = qf }
|v| > 0
Assume v = av ′ and w = bw′ . Clearly δ(qi , a) = qi+1 and δ(qj , b) = qj+1 = δ(qi , b)
It follows therefore that for any k ≥ 0

{δ ⋆ (q0 , u) = qi δ ⋆ (qi , v k ) = qi δ ⋆ (qi , w) = qf ⇒ xk = uv k w ∈ A

The Pumping Lemma as a game

Let Σ be any alphabet (6= φ) and let A ⊆ Σ⋆ be a langauge. The game is a 2-person game between
DEFEFDER who tries to defend the assertion “A is regular” and a CHALLENGER who challenges the
assertion. The game proceeds as follows :

1. DEFENDER : chooses a positive integer m > 0.

2. CHALLENGER : chooses an x ∈ A with |x| ≥ m.
3. DEFENDER : decomposes x into 3 parts u, v, w such that v =6= ε and |uv| ≤ m

28
 4. CHALLENGER : chooses k ≥ 0

Result : if xk = uv k w ∈ A then DEFENDER wins.

else CHALLENGER wins.

Now CHALLENGER has a winning strategy for A ⇒ A is not regular.

However if DEFENDER has a winning strategy for A ?
MAIN QUESTION. ARE THERE NON-REGULAR LANGUAGES
FOR WHICH DEFENDER HAS A WINNING STRATEGY

Example. Determine whether A = {a2n |n ≥ 0} is regular. Since the regular expression (a a)⋆ is a
representation of the language A the DEFENDER must have a winning strategy.

⋆
(a a) is accepted by

D: choose m = 2. (no of states in the DF A).

C : choose any n ≥ 2 , x = a2n
D: choose i = 0 , j = 2 with u = ε and v = a2 ⇒ x = a2n = a2 a2(n−1) ⇒ w = a2(n−1) .
C : choose any k ≥ 0.

DEFENDER’S STRATEGY
1. m = 3
2. u = ε, v = a2

Result : DEF has a winning strategy since for all n ≥ 2 and all k ≥ 0 , xk = (a2 )k a2(n−1) = a2(n+k−1) ∈ A.

Example. Determine whether A = {a2n+1 |n ≥ 0} is regular.

a a

D : choose m = 3.
C : choose any n ≥ 1.
D : choose i = 1 , j = 2 with u = a and v = a2
⇒ x = uvw = a3 a2n+1−3) ⇒ w = a2n−2 = a2(n−1) .
C : choose any k ≥ 0.
⇒ xk = a(a2 )k a2(n−1) = a2n−2+2k+1 = a2(n+k)+1 ∈ A.

29
 DEFENDER’S STRATEGY
1. m = 3
2. i = 1, j = 2 i.e. u = a, v = a2

2
Example Determine whether A = {an |n ≥ 0} is regular. We preove that C has a winning strategy.

D : Consider any m > 0.

2
C : Consider any n > m > 0. ⇒ n2 > n > m > 0 , x = an
D : Chooses any i ≥ 0, j > 1 such that i + j ≤ m.
⇒ 1 < j ≤ m < n < n2 .
2
⇒ ai v = ai w = a(n −(i+j)) .
C : Challenger chooses k such that
2 2
−(i+j) +kj
xk = ai (aj )k an = an 6∈ A.

CHALLENGER’S STRATEGY
2
1. x = an , n2 > n > m > 0
2. k = 1

challenger has a winning strategy provided.

n2 + kj is not a perfect square for C ′ s choice of k. Since 1 < j < n we have

n2 < n2 + j < n2 + n. < n2 + 2n + 1 = (n + 1)2

Hence challenger chooses k = 1 and it is guaranteed taht n2 < n2 + j < (n + 1)2 and hence n2 + j is
never a perfect square.
n
Example Consider the language A = {a2 |n ≥ 0}.
D : Chooses some m > 0.
n
C : Chooses x = a2 with n ≫ m > 0
D : Chooses i ≥ 0 , j > 0 with u = ai , v = aj
such that i + j ≤ m.
n
⇒ x = u v w and w = a(2 −(i+j))
n
C : Chooses k = 2 ⇒ xk = ai a2j a(2n−(i+j)) = a2 + j

CHALLENGER’S STRATEGY
2
1. x = an , n > m > 0
2. k = 2

Result : C always wins provided 2n + j is not a power of 2. 2n + j is a power of 2 for n > 0 only when
j = 2n . But n > m > 0 ⇒ 2n > m. But j ≤ m ⇒ j = 2n is impossible.
Example of a non-regular language which satisfies the condition of the pumping lemma

Let A = L (b⋆ c⋆ ) = {bm cn |m, n ≥ 0}

B = {abk ck |k ≥ q} (⋆ this is clearly a context-free language)

30
 C = {ai+2 bj ck |i, j, k ≥ 0} = L (aa a⋆ b⋆ c⋆ )
Claim 7. A ∪ B ∪ C is not regular.
Proof. Note : B 6⊆ A ∪ C since ∀x ∈ A ∪ C : #a(x) 6= 1.
whereas ∀y ∈ B : #a(y) = 1.
In fact A, B, C are mutually disjoint .
Proof by contradiction : Assume L = A∪B∪C is regular and accepted by a DF A D = h Q, {a, b, c}, δ, q0, F i.
Since A ∩ B = B ∩ C = C ∩ A = φ we have B = L − (A ∪ C) = L∩ ∼ (A ∪ B) from closure properties
that B is regular which is a contradiction.
Claim 8. |Q| > 1.

Proof. Since q0 ∈ Q, |Q| ≥ 1. Since a ∈ B, δ(q0 , a) 6= q0 otherwise any an for n ≥ 0 would be accepted
by D which is not the case. Hence |Q| > 1.
Assume |Q| = n > 1. Consider any string x = abk ck ∈ B, such that k > n. Let
a b b b c c
q0 −→ q1 −→ . . . −→ qk −→ qk+1 −→ . . . −→ q2k+1

Since k > n, there must be states 1 ≤ i ≤ j ≤ k such that ai = qj .

This implies that
a b b b b b b c c
q0 −→ q1 −→ . . . −→ qi = qj −→ . . . −→ qj = qi −→ . . . −→ qk+1 −→ . . . −→ q2k+1
| {z }
a cycle which may be pumped.

x = uvw where u = abi−1 , v = bj−i , w = bk+1−j ck and hence for any x0 = uw is accepted
by D.
But uw = abi−1 bk+1−j ck But 1 ≤ i < j ≤ k
= a bk−(j−i) ck ⇒ j − i ⇒ −(j − i) < 0
⇒ k − (j − i) < k
6∈ B since k − (j − i) 6= k
A ∪ C since #a(uw) = 1.

Hence x0 is accepted by D but x0 6∈ A ∪ B ∪ C. Hence there is no DF A which accepts A ∪ B ∪ C

DEFENDER’S WINNING STRATEGY

1. m = 2 (
a2 if x ∈ a2 b⋆ c⋆
2. u = ε and v =
x(1) else

Claim 9. By choosing m = 2, DEFENDER has a winning strategy.

Proof. Let CHALLENGER choose any x ∈ A ∪ B ∪ C with |x| ≥ 2
The DEFENDER then has the following cases

Case 0. x ∈ A with |x| ≥ 2

⇒ x=bi cj for some i, j ≥ 0 and i + j ≥ 2

 i
c with j ≥ 2 and i = 0
⇒ x = bi with i ≥ 2

 i j
b c with i ≥ 1 , j ≥ 1
DEFENDER chooses u = ε and v = x(1)

31
 
i i−1

b with i ≥ 2 ⇒ v = b ∧ w = b ⇒ v k w = bk+i−1 ∈ A
j j−1
⇒ x = c with j ≥ 2 ⇒ v = c ∧ w = c ⇒ v k w = ck+j−1 ∈ A

 i j
b c with i ≥ 1, j ≥ 1 ⇒ v = b ∧ w = bi−1 cj ⇒ v k w = bk+i−1 cj ∈ A
In each case v k w ∈ A, for all k ≥ 0 and hence DEFENDER wins.

Case 1. x ∈ B with |x| ≥ 2

⇒ x = a bi cj with i ≥ 1. (if i = 0 then |x| < 2). DEFENDER chooses u = ε and v = x(1) = a .
Then for every choice of k , we have

 i i
b c ∈ A if k = 0
v k w = abi ci ∈ B if k = 1

 k i i
a b c ∈ C if k > 1
and the DEFENDER wins.

Case 2. x ∈ C with |x| ≥ 2

⇒ x=(a2+i bj cl with i, j, l ≥ 0.
a2 bj cl with i = 0, l ≥ 0.
⇒ x=
ai+3 bj cl with i ≥ 0, j, l ≥ 0.
Case 2.2 x = a2 bj cl

Then DEFENDER chooses u = ε and v = a2

⇒ w = bj cl with j, l ≥ 0

for any k ≥ 0 we have v k w = a2k bj cl

k = 0 ⇒ v k w = bj cl ∈ A. and DEFENDER wins
k > 0 ⇒ v k w = a2k bj cl ∈ A. and DEFENDER wins
Case 2.3 x = ai+3 bj cl with i ≥ 0, j, l ≥ 0

The DEFENDER chooses u = ε and v = a

⇒ w = ai+2 bj cl
and for all k ≥ 0, v k w = ai+k+2 bj cl ∈ C and DEFENDER wins.

DEFENDER’S STRATEGY (summary)

1. m = 2
2. For any x ∈ A ∪ B ∪ C, (
a2 if x ∈ a2 b⋆ c⋆
u = ε and v =
x(1) otherwise

Other Application of Pumping Lemma

Theorem 12. Let D be a DF A with m states. Then L (D) 6= φ iff there exists a string x ∈ L (D) such
that |x| < m
Proof. (⇒) Let x be a minimal length string in L (D). If |x| ≥ m then by the pumping lemma x = uvw
with |uv| ≤ m and v = ε. Clearly uw = uv 0 w ∈ L (D). But since |v| > 0 |uw| < |x| contradicting the
assumption that x is a minimal lenth string in L (D).

32
The above theroem finishes a decision procedure to test whrther a given DF A , D accepts the empty
language. Since the number of strings of length < m is finite , it is neccessary to run the DF A on all
possible string of length < m , to determine it.

Theorem 13. If A is an infinte regular language then there exist strings u, v, w such that uv i w ∈ A for
all i ≥ 0.
Proof. Obvious
Theorem 14. Let D be a DF A with m states m > 0. Then L (D) is infinite iff L (D) contains a string
x, with m ≤ |x| < 2m.
Proof. (⇐) Assume x ∈ L (D) with m ≤ |x| < 2m. By the pumping lemma x may be decomposed into
u, v, w with v 6= ε and such that for all i ≥ 0, uv i w ∈ L (D). Hence L (D) is infinite.
(⇒) Assume L (D) is infinite. Then there exist strings x ∈ L (D) with |x| ≥ 2m. Let x be the stortest
such string and x = x1 x2 where |x1 | = m. and |x2 | ≥ m. Consider the proof of the pumping lemma with
q = δ(q0 , x1 ). Clearly the path from q0 to q. toches (m + 1) states while accepting x1 . This implies ther
exists a q ′ which appears twice

q0 −→ . . . −→ q ′ −→ . . . −→ q ′ −→ . . . −→ q.

Hence x1 may be decomposed into u, v, w such that v = ε, x1 = uvw and δ(q0 , u) = q ′ , δ(q ′ , v) = q ′ and
δ(q ′ , w) = q. Clearly x = uvwx2 ∈ L (D) and state q ′ is repeated implies x′ = uwx2 ∈ L (D). Further
since D is a DF A , v 6= ε. Hence |ε| > 0. |v| ≤ m.

|x′ | = |uwx2 | ≥ |x2 | ≥ m. and since v 6= ε , |x′ | < |x|

But since x is the stortest word of length ≥ 2m. It follows that |x′ | 6≥ 2m and must be < 2m. But
|x′ | ≥ m. Hence m ≤ |x′ | < 2m and x′ ∈ L (D)

Decision procedure to determine whether the language accepted by a DF A is

finite
Simply run the DF A on all strings of length in the interval (m, 2m). If it accepted none than the DF A
accepts only a finite language. Otherwise it accepts an infinite language.

Ultimate Periodicity
Definition 11. A subset ∪ ⊆ N = {0, 1, 2, . . . } is said to be ultimately periodic if there exist m ≥ 0 ,
p > 0 such that for all n ≤ m , n ∈ ∪ iff n + p ∈ ∪.

Example

1. Every finite subset ∪ of N is ultimately periodic with m = max ∪ +1 and any p > 0.
2. Every infinite arithmetic sequence is ultimately periodic with a period defined by the common dif-
ference between successive number and m the starting number
3. The union of any finite collection of infinite arithmetic sequences with different starting points but
the samw common difference is also ultimately periodic. That is [ if Ai = {mi + j d |j ≥ 0, mi ≥ 0},
i > 0 is a finite collection of k ≥ 0 infinite sets. Then ∪ = Ai is ultimately periodic with
i≤k
m = max mi and period d.

33
 HOMOMORPHISMS
Definition 12. Let Σ and Γ bealphabets. Then any function h : Σ⋆ −→ Γ⋆ is a homomorphism if
∀x, y ∈ Σ⋆ : h(x.y) = h(x).h(y)
Definition 13. Let h0 : Σ −→ Γ⋆ . h : Σ⋆ −→ Γ⋆ is said to be a homomorphic extension of h0 if
(i) h is a homomorphism
(ii) ∀a ∈ Σ : h(a) = h0 (a).
Lemma 7. For any homomorphism h : Σ⋆ −→ Γ⋆ , h(ε) = ε
Proof. Assume not. Then h(ε) = u 6= ε. This implies for any x ∈ Σ⋆
v = h(x) = h(ε.x) = h(ε).h(x) = uv = u2 v = . . .
which implies h is not even a function from Σ⋆ to Γ⋆
Lemma 8. Every h0 : Σ −→ Γ⋆ has a unique homomorphic extension.
Proof. Consider h : Σ⋆ −→ Γ⋆ a homomorphic extension of h0 . Since h is a homomorphic we have
h(ε) = ε
and ∀a ∈ Σ : h(a) = h0 (a). and ∀x, y ∈ Σ⋆ : h(x.y) = h(x).h(y). If h′ is any other homomorphic
extension of h0 . We have
h′ (ε) = ε = h(ε)
and by induction hypothesis is (assuming h′ (x) = h(x) f or all |x| < n) if y = ax then h′ (y) = h′ (ax) =
h0 (a).h′ (x) = h0 (a).h(x) = h(ax). Hence h = h′ and homomorphic extensions are unique.
Definition 14. Let h : Σ⋆ −→ Γ⋆ be a homomorphism. For any A ⊆ Σ⋆ h(A) = {h(x)|x ∈ A} and for
any B ⊆ Γ⋆ h−1 (B) = {x ∈ Σ⋆ |h(x) ∈ B}. h(A) is called the image of A under h and h−1 (B) is called
the pre-image of B under h.
Theorem 15. Let h : Σ⋆ −→ Γ⋆ be a homomorphism. for any regular B ⊆ Γ⋆ , h−1 (B) is also regular.
Proof. Let DB = h QB , Γ, δB , q0 , FB i be a DF A that accepts B. Let A = h−1 (B). Construct the DF A
DA = h QA , Σ, δA , q0 , FA i with QA = QB , FA = FB and the same start state q0 .
⋆
Define δA (q, a) = δB (q, h(a)) for all q ∈ QA , a ∈ Σ.
⋆ ⋆
Claim 10. δA (q, x) = δB (q, h(x)) for all x ∈ Σ
Proof. By induction on |x| since δA (q, ε) = q = δB (q, ε)
Claim 11. L (DA ) = h−1 (L (DB )).
⋆
Proof. x ∈ L (DA ) ⇔ δA (q0 , x) ∈ F
⋆
⇔ δA (q0 , h(x)) ∈ F
⇔ h(x) ∈ L (DB )
⇔ x ∈ h−1 (L (DB ))
Theorem 16. Let h : Σ⋆ −→ Γ⋆ be a homomorphism. If A ⊆ Σ⋆ is regular then so is h(A).
Proof. If A ⊆ Σ⋆ is regular then there exists a regular expression α ∈ REΣ such that L (α) = A. Let
h(A) = B ⊆ Γ⋆ . We define a function

REΣ −→ REΓ

which translates regular expression α into β i.e. h(α) = β. We define h by induction on the structure of
α as follows.

h(φ) = φ
h(ε) = ε

34
 h(a) = if h(a) = y , for each a ∈ Σ.
h(α + α′ ) = h(α) + h(α′ )
h(α.α′ ) = h(α).h(α′ )
h(α⋆ ) = (h(α))⋆
Claim 12. For any α ∈ REΣ , L (h(α)) = h(h(α)).
Proof. Follows by induction on the structure of α
Theorem 17. Any set A ⊆ {a}⋆ is regular iff the set {m|am ∈ A, m ≤ 0} is ultimately periodic.
Proof. (⇒) If A is finite then take m = max{i|ai ∈ A} + 1. Otherwise let A be infinite and consider any
DF A. D which accepts A. Clearly since it is deterministic the transition graph of the DF A (resticted
to the accessible states) looks as follows where there is an initial
a a a a
q0 −→ q1 −→ . . . −→ qi −→ qi+1

Figure 1:

linear sequence of distinct states followed by a loop of length p ≥ 1

Choose m = the number of distinct initial states and p the length of the loop.
Note If the path through the DF A extend beyond the loop

a a a a a a
q0 −→ . . . −→ qi −→ qi+1 −→ qi+1 −→ . . . −→ qm

Figure 2:

then an equivalent DF A which accepts the same language may be constructed which conforms to Fig 1.
(⇐) Conversely let ∪ be any ultimately periodic set with period p > 0 and starting point n4. Then
construct a DF A{a} with n + p + 1 states which accepts exactly the set {am |m ∈ ∪}

Application of homomorphisms
Corollary 2. Let A ⊆ Σ⋆ be any regular language. Then the set sup = {|x| | x ∈ A} is ultimately
periodic.

35
Proof. Since A is regular, h(A) is also regular where h : Σ −→ {1} is defined as ∀a ∈ Σ : h(a) = 1.
FRom the pervious theorem it follows that h(A) is ultimately periodic. But h(A) is merely th unary
representation of the elements of ∪.

Example
A = {ai bj ci+j | i ≥ 0, j ≥ 0} can be proven to be non-regular by considering the homomorphism
h(a) = h(b) = b and h(c) = c. Then h(A) = {bi+j ci+j |i + j ≥ 0} which is clearly non-regular.

DF A MINIMIZATION
In general, for any regular language there may be several possible DF A designs. It is possible to minimize
the number of states in two stages :
(i) Get rid of inaccessible states. A state is inaccessible if there is no path to it from the start state.
(ii) Collapse states which are “equivalent” in some sense without chnaging the language accepted.

In general inaccessible states may be identified by “walking” forwards from the start state to the final
states and marking all the the states reached. Hence a depth-first search algorithm or a greadth-first
search algorithm should do the job.
In more interesting question is that of collapsing equivalent states.
Given a DF AD consisting only of accessible states, for any string x, x ∈ L (D) iff δ ⋆ (q0 , x) ∈ F . If
q = δ ⋆ (q0 , x) 6∈ F then q is clearly a reject state.
Definition 15. Two states p, q are said to be indistinguishable (denoted p ≈ q) iff ∀x ∈ Σ⋆ : δ ⋆ (q0 , x) ∈
F ⇔ δ ⋆ (q, x) ∈ F . p is said to be distinguishable from q if there exists a distinguished wors w ∈ Σ⋆ such
that either δ ⋆ (p, w) ∈ F ∧ δ ⋆ (q, w) 6∈ F or δ ⋆ (p, w) 6∈ F and δ ⋆ (q, w) ∈ F .
Fact

1. Indistinguishability is an equivalence relation on the states of a DF A.

i.e. ∀p ∈ Q : p ≈ p reflexivity
∀p, q ∈ Q : p ≈ q ⇒ q ≈ p symmetry
∀p, q, r ∈ Q : p ≈ q ∧ q ≈ r ⇒ p ≈ r transitivity.
2. ≈ partitions the set of states Q into equivalence classes.
)
df for any p, q ∈ Q
[p]≈ = {q ∈ Q|p ≈ q}
[p]≈ ∩ [q]≈ = φ
p ≈ q ⇔ [p]≈ = [q]≈ ∨ [p]≈ = [q]≈
df
Q/ ≈ = { [p] | p ∈ Q }.
3. For any DF A D = h Q, Σ, δ, q0 , F i,
D/ ≈ = h Q/ ≈, Σ, δ≈ , [q0 ]≈ , F ≈ i
where δ≈ ([p], a) = [δ(p, a)]≈

We now have to show that this quotient construction is well-defined and that D/ ≈ is indeed a DF A.
This involves showing that δ≈ well-defined.
Lemma 9. If p ≈ q then δ(p, a) ≈ δ(q, a) for all a ∈ Σ
Proof. p ≈ q ⇔ ∀x ∈ Σ⋆ : δ ⋆ (p, x) ∈ F ⇔ δ ⋆ (q, x) ∈ F .
Consider any a ∈ Σ and y ∈ Σ⋆ .
Let δ(p, a) = pa and δ(q, a) = qa . For any y ∈ Σ⋆
we have

36
 
δ ⋆ (pa , y) ∈ F 


⇔ δ ⋆ (p, ay) ∈ F 
⇒ pa ≈ qa
⇔ δ ⋆ (q, ay) ∈ F 


⇔ δ ⋆ (qa , y) ∈ F

form this lemma it follows that the definition

δ≈ ([p]≈ , a) = [δ(p, a)]≈ is well defined.
Lemma 10. p ∈ F ⇔ [p]≈ ∈ F/ ≈

Proof. (⇒) follows from the definition.

(⇐) Let q ∈ [p]≈ ∈ F/ ≈.
Clearly if p ∈ F then δ ⋆ (p, ε) = p ∈ F and since q ≈ p, δ ⋆ (q, ε) = q ∈ F .
Lemma 11. ∀x ∈ Σ⋆ : δ≈
⋆
([p]≈ , x) = [δ ⋆ (p, x)]≈ .
Proof. By induction on |x|.
We now prove that D/ ≈ accepts the same language as D.

Theorem 18. L (D/ ≈) = L (D)

Proof. For any x ∈ Σ⋆ , x ∈ L (D/ ≈)

⋆
⇔ δ≈ ([q0 ]x , x) ∈ F/ ≈

⇔ [δ ⋆ (q0 , x)]≈ ∈ F/ ≈

⇔ δ ⋆ (q0 , x) ∈ F

⇔ x ∈ L (D)
Consider the indistinguishability relation on the states of D/ ≈. That is let

[p]≈ ∼ [q]≈ ⇔ ∀x ∈ Σ⋆ : δ≈
⋆ ⋆
([p]≈ , x) ∈ F/ ≈⇔ δ≈ ([q]≈ , x) ∈ F/ ≈
Claim 13. [p]≈ ∼ [q]≈ ⇒ [p]≈ = [q]≈ .
Proof. [p]≈ ∼ [q]≈

⇒ ∀x ∈ Σ⋆ : δ≈
⋆ ⋆
([p]≈ , x) ∈ F/ ≈⇔ δ≈ ([q]≈ , x) ∈ F/ ≈

⇒ ∀x ∈ Σ⋆ : [δ ⋆ ([p], x)]≈ ∈ F/ ≈⇔ [δ ⋆ ([q], x)]≈ ∈ F/ ≈

⇒ ∀x ∈ Σ⋆ : δ ⋆ (p, x) ∈ F/ ≈⇔ δ ⋆ (q, x) ∈ F

⇒p≈q

⇒ [p]≈ = [q]≈

37
 CONTEXT-FREE LANGUAGES
Definition 16. A grammar G = h V, Σ, S, P i is linear if every production has at most one variable on
the right hand side.

Example The following grammar G is linear but neither right linear nor left-linear.

S −→ A Equivalently
A −→ aB|ε S −→ ε|aB
B −→ Ab B −→ Sb
L (G) = {an bn | n ≥ 0}.
Definition 17. A grammar G = h V, Σ, S, P i is callled context -free if all productions have the form
A −→ α where A ∈ V and α ∈ (V ∪ Σ)⋆ . The language generated L (G) by a contextr-free grammar is
called a context-free language.

Example S> −→ AS= , S= −→ aS= b | ε , A −→ aA|a is a grammar with start symbol S> that
generates {am bn | m > n ≥ 0}. Similarly

S< −→ S= B , B −→ bB | b

generate {am bn | 0 ≤ n < n}. Then S6= −→ S> | S< generates the language {am bn | m 6= n}.

Example S −→ aSb | SS | ε is a grammar that generates balanced parentheses sequence. This grammar
is context-free but is not linear. The language is described by

{w ∈ {a, b}⋆ | #a(w) = #(w) ∧ ∀v 4 w : #a(v) ≥ #b(v)}

Facts And Remarks

F1. Every right linear or left linear language is a linear language.

F2. Every linear language is context-free.
R3. Whereas the language {am bn |m 6= n} has been generated by a non linear CF grammar , there does exist
a linear grammar which can generate it.
R4. The example above is of a language that cannot be generated by a linear grammar.

Derivation Trees
Let G = h V, Σ, S, P i be a CF G. A rooted ordered tree is called a derivation tree for G iff it satisfies the
following conditions.

0. The rooot is labelled S.

1. Every leaf has a unique label from Σ ∪ {ε}.
2. Any leaf labelled ε has no other siblings
3. Every non-leaf node is labelled (uniquely) bu a symbol from V .
4. If a non-leaf node has a label A and its children from left to right are labelled respectively with
symbols that from a string al ∈ (V ∪ Σ)+ then A −→ α ∈ P .
A partial derivation tree is one which is like a derivation true expcept that the condition ! is replaced by

1′ . Every leaf has a unique label from V ∪ Σ ∪ {ε}.

38
Definition 18. The yield of a tree is the string of symbols obtained by reading the labels on the leaves
from left to right (omitting all occurrences of ε).

fact Every derivation tree is also a partial derivation tree

Proof that the set of all balanced parenthesis is generated by the grammar

S −→ aSb | SS | ε

The set BP = {w ∈ {a, b}⋆ | #a(w) = #b(w) ∧ ∀v 4 w : #a(v) ≥ #b(v)}

The proof is divided into two claims for the

Lemma 12. S ⇒⋆ x ∈ {a, b}⋆ implies x ∈ BP .
Proof. By induction on the length of the derivation S ⇒⋆ x. We actually prove by induction on the
length of derivations for all sentential forms generated by the grammar that

S ⇒⋆ α ∈ (V ∪ Σ ∪ {ε})⋆ implies

α ∈ Bp′ = {β ∈ (V ∪ Σ ∪ {ε})⋆ | #a(β) = #b(β) ∧ ∀ −→ ′ 4 α : #a(α′ ) ≥ #b(α′ )}

By induction on the length of the derivation S ⇒⋆ α.

Basis 0. α = S and S ∈ BP ′ .
Ind Step. Assume for all derivations of length ≤ k, S ⇒⋆ α, α ∈ BP ′ . Consider any derivation of
length k + 1 with S ⇒⋆ α ⇒ β.

By IH α ∈ BP ′ . By case analysis we have the following possibilities.


(i) α = α1 Sα2 ⇒ α1 aSbα2 = β  In each case it is
(ii) α = α1 Sα2 ⇒ α1 SSα2 = β clear that β ∈ BP ′


(iii) α = α1 Sα2 ⇒ α1 α2 = β

Lemma 13. Every x ∈ BP is generated by the grammar.

Proof. By the induction on the length of x.

Basis |x| = 0 ⇒ x = ε and S ⇒ ε by the rule S −→ ε.

Induction Step If |x| > 0 then we have two cases.

case(i) ∃ y ≺: y ∈ BP
case(ii) ¬ ∃ y ≺: y ∈ BP

case(i) ∃ y ≺: y ∈ BP ⇒ x = y z for some z.

Claim 14. z ∈ BP
Proof. Since #a(x) = #b(x) and #a(y) = #b(y)
#a(x) = #a(y) + #a(z) and #b(x) = #b(y) + #b(z)
we have #a(z) = #a(x) − #a(y)
= #b(x) − #b(y)

39
 = #b(z)

For any prefix w 4 z we have

#a(w) = #a(yw) − #a(y)

0000
≥ #(yw) − #b(y)
= #b(w)
Hence since y ∈ BP and z ∈ BP and |y| < |x| and |z| < |x| we have by IH

S ⇒⋆ y
S ⇒⋆ z.

It follows that the following derivation generate x

S ⇒ SS ⇒⋆ yS ⇒⋆ yz.

case(ii) ¬ ∃ y ≺: y ∈ BP and |x| > 0.

This implies x = azb for some z ∈ BP . and |z| < |x|. Hence there exists a derivation S ⇒⋆ z.
Therefore the following derivation generate x

S ⇒ aSb ⇒⋆ azb = x
Definition 19. A leftmost derivation is a derivation in which the leftmost variable is rewrithen. Simi-
larly we have rightmost derivations.

Fact. For every derivation tree there is a unique leftmost (rightmost) derivation which corresponds to a
depthfirst traversal of the tree in which the leftmost rightmost subtree is explored before any other subtree.
Theorem 19. Let G = h V, Σ, S, P i be a CF G. Then for every w ∈ L (G) there exists a derivation tree
whose yield is w.
Proof. Since every derivation tree DT is also a partial derivation tree P DT it suffices to prove the
following claim.
Claim 15. For every sentential form α of G there exists a partial derivation tree whose yield is α.
Proof. By induction on the number fo steps in a derivation of G that ends in α.

Basis S is the only sentential from derivable in 0 steps.

Induction Step Assume for every sentential from α derivable in k steps there exists a P DT of depth k. Let
α = β in one step. Hence α = α1 Aα2 for some α1 , α2 ∈ (V ∪ Σ)⋆ and β = α1 γα2 for some A −→ γ ∈ P .
Consider the P DT for α, which exists by IH. The yield of this P DT is α. Hence there is a leaf node A
preceded by leaf nodes of the symbols of α1 and succeeses by the symbols of α2 . By expanding the node
labelled A to leaf nodes corresponding to the order of symbols in γ we get a new P DT whose yield id
α1 γα2 = β.
Theorem 20. Every P DT of G represents some sentential from of G.

40
Proof. By induction on the maximum depth of P DT s.
BAsis 0. The only sentential form is the single root node labelled S which is also the leaf node.

Induction Steps. Assume every P DT of depth ≤ k > 0 represents a sentential form α of G where the
yield of the P DT is α.
Consider any P DT of depth k + 1. Consider all the non-leaf nodes at depth k. Clearly all of them are
variable symbols say A1 , . . . , Aj , j > 0. For each Ai , 1 ≤ i ≤ j consider the ordered sequence of children
α1 , . . . , rj . Clearly there are productions Ai −→ γi ∈ P which yielded this tree. Consider the yield of the
P DT upto depth k. By the IH it represents a sentential form.

α = x0 A1 x1 A2 . . . Aj xj

where x0 , . . . , xj ∈ Σ⋆ . Again this is a sentential form of the grammar there must be a deivation

S ⇒⋆ α = x0 A1 x1 . . . Aj xj

Also the yield of the P DT of depth k + 1 is of the form

β = x0 γ1 x1 . . . γj xj

It is easy to see that in j-steps it is possible to derive β from α. i.e

S ⇒⋆ x0 A1 x1 . . . xj−1 Aj xj ⇒ x0 γ1 x1 . . . xj−1 Aj xj

⇒ 


.. j-steps
. 


⇒ x0 γ1 x1 . . . xj−1 Aj xj
Hence β is a sentential form of the grammar
Corollary 3. (Converse of yield theorem) The yield of any derivation tree is a sentence of L (G).
Q1. We need to distinguish between states q0 and q1 which generate a carry of 0 or 1. Initially there is
only a previous carry of 0. Depending on the carry a triple may acceptable or not. Once an unacceptable
triple arrives the machine goes into a fail state and never recovers. The

41
 Current state Next state

0 1 0 1
0 0
@0A @1A 4 q P a b c q′
0 ,1 ,
0 10 10 1 0 1
0 0 0 0 q0
0 0 1 1
@0A @1A @0A @1A 0 0 0 1 qf
1 0 0 1
, , , 0 0 1 0 qf
q0
0 0 1 1 q0
q0
0 1 0 1 0 1 0 0 qf
0 1
@0A @1A 0 1 0 1 q0
1 0 qf
0 1 1 0 q1
Σ
0 1 1 1 qf

q1 0 1 0 10 1 1 0 0 0 qf
0 S 1 1
@0A @0A @1A
1 0 0 1 q0
0, , , 1 0
1 0 1 0 q1
0 1 0 10 1
0 1 1 1 0 1 1 qf
@1A @0A @1A q1
0 0
, , 1 1 1 0 0 q1
1 1 0 1 qf
To make sure the ε is not accepted , it is a good
1 1 1 0 qf
idea to have a start state as well
1 1 1 1 q1

Q2. Given G1 = h V1 , Σ, S1 , P1 i and G2 = h V2 , Σ, S2 , P2 i we construct G1∪2 h V1∪2 , Σ, S1∪2 i as follows

V1∪2 = V1 ∪ V2 ∪ {S1∪2 }
P1∪2 = {S1∪2 −→ S1 |S2 } ∪ P1 ∪ P2
G12 = h V12 , Σ, S12 , P12 i as follows : Intuitively we generate L (G1 ) and then begin to generate L (G2 )
appended to it. So it would suffice get the desired language. However since this production is not right
linear we need to distribute the “appendage” S2 to all the terminal productions of P1 . Hence

P12 = {A1 −→ x1 S2 | A1 −→ x1 ∈ P, x1 ∈ Σ⋆ } ∪ P2

with V12 = V1 ∪ V2 and S12 ≡ S1 .

Let G = h V, Σ, S, P i be aright linear grammar. We construct G⋆ = h V⋆ , Σ, S⋆ , P⋆ i as follows .

P⋆ = {S⋆ −→ ε | S}∪
{A −→ xS⋆ | A −→ x ∈ P , x ∈ Σ⋆ }∪
{A −→ xB | A −→ xB ∈ P , x ∈ Σ⋆ }

where V⋆ = V ∪ {S⋆ }.

Since for right linear grammars all sentential forms generated are of the form ⇒ xA where x ∈ Σ⋆ and
A ∈ V , there is always at most one nonterminal at the end of a sentential form. So there is no danger of
having forms such as xAS2 yS2 .

42
 0 1 0 1
0 S 1
@0A @0 A
0, , 0 1
1
q1 can’t be a final
0 0 10
@0 A 0 1 state because the
0 11 @1 A 1 0 1
0 01 S @ 0A
q0 1 , ,0
@0 A 1
@1 A
carry it carries cannot
@ 0A ,1 , 0
1 be consumed. Hence
0,
0 1
0 1 0 1 an
0 1 Σ @ bn Ashould not
0 1 @0A @1A 1
0 cn
qs 1 1 0 0 11 1 A qf
1 generate any carry.
@1A 0
,
0 S @ 0A 0
@1
0 @ 0A
0
, ,
1

q1 1
0 1
1
10 1 @ 1A

,,
10 0 @ 0A 1
00 A 0
A@ 1
@0
1 , 0

Q3 A = {0000wwR | w ∈ Σ⋆ }. To show that A is not regular we prove as challenger that for any m > 0
chosen by the defender,the challenger has a winning strategy.

DEFENDER : Assume m > 0 is chosen.

CHALLENGER : Choose y ∈ Σ⋆ such that |y| > m.
Hence x = yy R ∈ A chosen with |x| > 2m.

DEFENDER : Since defender decomposes x into 3 parts u, v, w with v 6= ε and |uv| ≤ m, any decompo-
sition of the defender such that x = yy R = uvw necessarily decomposes y into 4 parts y = uvz. Hence

x = yy R = uvzz R v R uR and w = zz R v R uR .

CHALLENGER : Choose k = 2 with

xR = x2 = uv 2 w = uvvw
= uuvzz Rv R uR
= uvvz(vz)R uR 6∈ A.

Note : x = αβ
⇔ xR = β R αR
Hence A is not regular.
Note that A is not regular only if |Σ| > 1
If |Σ| = 1 i.e. Σ = {a} then A = {a2n | n ≥ 0} which is regular.
Othersolutions involve choosing a 6= b and x = am bb am and proceeding with the argument.

Q4 It is very difficult to give anything except an (inequational) characterization of r k s which we give

below starting from structural rules and chosing under commutatively fboxr k s = s k r
φks=φ
εks=s
a.r′ k s = a.(r′ k s) + s k (a.r′ )
(r1 + r2 ) k s = r1 k s + r2 k s
r⋆ k s = s + (r1 .r2 ) k s
finally we have the rule

r1 .r2 ≤ ⇒ (r1 .r2 ) k s ≤ s

and r1 .r2 ≥ r ⇒ (r1 .r2 ) k s ≥ r k s .

43
where ≤ is the partial order on regular expressions induced by the inequation

r ≤r+s

and ≥ is ≤−1 .

0.1 AMBIGUITY & PARSING

We have stated that the grammar

S −→ SS | aSb | ε

is ambiguous. For example the sentence abab has two possible derivation trees.

S S

S S
S S
a
b S
S Sa S b S S

ε
a S b a S b ε a S b a S b

ε ε ε ε

The ambiguity in this case has to do with now we group “chunks” of balanced parethesis where a “chuck”
is a maximal substring of the form ak bk . Since catenation is associative there are different ways of group-
ing the chunks into sequences of chunks — either (ab ab)ab or ab(ab ab). In this particular case we may
resolve the ambiguity by choosing a “regular” to denote a chunk and use it to generate tail recursive
sequences of chunks as follows.

S −→ ε | CS
C −→ aSb
↑
Wthin each chunks there may be sequences too !

With this grammar the sequence “ababdb” has only once derivation tree as follows.

C S This new variable with a

“determinishing” effect adds
S extra control.
a C b C S
ε
C
ε a C b

a C b
ε

44
The language generated by there grammars is BP . Suppose we need to generate BP + = BP − {ε}. We
could proceed as follows.

S −→ C | CS
C −→ ab | aSb.

Exercise

1. Prove that these two grammars generate BP + .

2. How do you show that this grammar is unambiguous ?

Parsing & Membership

A crucial decision problem in compilers is the question of “whether a given string x ∈ Σ⋆ belongs to
L (G)” for a grammar. This is the membership problem which gets solved automatically if there exists
an algorithms to determine “whether there exists a derivation tree for x” or equivalently “whether there
exists a derivation S ⇒⋆ x” equivalently “whether there exists a sequence of production applications
which will derive x from S”. This is called a “parse”.
There exists an exhaustive method of determining whether there is a parse of x ∈ Σ⋆ .
Definition 20. Given a CF G G, a production A −→ ε is called a null production, and any production
of the form. A −→ B for A, B ∈ V is called a unit production2 . A CEG which has no null productions
is called a positive CF G. A positive CF G which has no unit productions is called a branching CF G.

Facts

1. Every production of a positive CF G is length non-decreasing i.e. A −→ α ∈ (V ∪ Σ)+ implies

|α| ≥ 1. Hence for any derivation S ⇒⋆ αi ⇒ αi+1 it guaranteed that |αi | ≤ |αi+1 |.
2. In a branching CF G every production except terminal ones is strictly length increasing. Hence in
any derivation.

S ⇒⋆ αi ⇒i+1 ⇒ . . .

|αi | = |αi+1 | iff αi+1 is obtained from αi by appliction

of a length-preserving terminal production.
|αi | < |αi+1 | iff αi+1 is obtained from αi by appliction
of a length-increasing production.

Theorem 21. If G is a branching CF G then the parsing problem for G is solvable i.e. There exists an
algorithm which for any x ∈ Σ⋆ can decide membership and if x ∈ L (G) it produces a parse of x.
Proof.
Claim 16. For each steps αi ⇒ αi+1 in a derivation either |αi | < |αi+1 | or the number of terminal
symbols in αi is less than the number of terminal symbols in αi+1 .
Claim 17. Because of claim 1 ,the number of steps in any derivation S ⇒⋆ x is at most 2|x|.
Claim 18. There are only a finite number of leftmost derivations of any given length
2a terminal production is one of the form A −→ a.

45
Proof. In each derivation steps of a leftmost from a given sentential from αi there are atmost |P | different
αi+1 , since only one of the productions may be applied to a leftmost non-terminal in αi to obtain αi+1 .
Hence any derivation

S ⇒ α1 ⇒ . . . αm

allows for at most |P | possibilities for α1 and for each possibilty α1 , there are at most |P | possibities
of α2 , which implies there are only |P |2 different leftmost derivation. Proceeding in this fashion for any
partial derivation of length k. there are at most |P |k different possibilities.
Exhaustively enumerate all partial derivations of length ≤ 2|x|. x ∈ L (G) iff x is the string generated
by one of the derivations and if so the leftmost derivation that generates x also yields a parse of x in the
grammar.
The previous theorem indicates that the parsing problem for general CF G would be solvable provided

(i) it is possible to eliminate unit productions and

(ii) it is possible to isolate ε productions and transform the grammar into a branching grammar. In effect
we would like to prove the following theorems.
Theorem 22. A language A ⊆ Σ⋆ is context-free iff there is a positive grammar G such that L (G) =
A − {ε}. Moreover any CF G G may be transformed into a positive CF G, G+ such that L (G+ ) =
L (G) − {ε}.
Theorem 23. Any positive context-free grammar may be transformed into an equivalent branching
context-free grammar.

Transformations of CF Gs
The substitution Rule. Let G = hV, Σ, S, P i be a CF G.
Lemma 14. If A −→ αBβ ∈ P for A 6= B and B −→ Γ1 | . . . | Γ are all productions of B. Then
b = hV, Σ, S, Pb i where Pb = P − {A −→ αBβ} ∪ {A −→ αΓi β | 1 ≤ i ≤ k} is an equivalent grammar.
G
Proof. Consider any derivation in G such that

S ⇒G ⋆ρAσ ⇒G ραBβσ ⇒G . . . ⇒G τ BΘ ⇒G τ Γi Θ ⇒ . . .

b exists with
where ρα ⇒ ⋆τ and βρ ⇒ ⋆Θ. A corresponding derivation in G

S ⇒Gb ⋆ ρAσ ⇒ ραΓi βσ ⇒ . . . ⇒ τ Γi Θ ⇒ . . .

Since ρα ⇒ ⋆τ and βσ ⇒ ⋆Θ. Hence

S ⇒G ⋆ x ⇒ S ⇒G ⋆ x. i.e. b
L (G) ⊆ L (G)

b such that
Similarly for any derivation in G

S ⇒Gb ⋆ ρAσ ⇒Gb ραΓi βσ ⇒Gb . . .

there exists a corresponding derivation

S ⇒G ⋆ ρAσ ⇒ ραBβσ ⇒ ραΓi βρ ⇒ . . .

b ⊆ L (G).
Hence S ⇒Gb ⋆ x ⇒ S ⇒G ⋆ x i.e. L (G)

46
Example. G : S −→ ε | cs, c −→ aSb
b : S −→ ε | aSbS
G
As a converse of the substitution rule we could have a factoring rule which allows a rule A −→ αβΓ ∈ P
to be replaced by A −→ α × Γ and a new rule with X 6∈ V introduced as X −→ β.
Factoring allows us to transform the grammar G b : S −→ ε|aSbS to the grammar G : S −→ ε|CS −→ aSb

Removal of Useless productions

Definition 21. a variable A ∈ V is useful iff

∃x ∈ L (G) : S ⇒ ⋆ αAβ ⇒ ⋆ x

otherwise a variable is called useless3 . A production is useless if it involves a useless variable (either on
the LHS or in the RHS).

Example. S −→ A , A −→ aA | ε , B −→ bA. Here B is a useless variable and B −→ bA is a useless

production.
b = hVb , Σ,
Theorem 24. Let G = h V, Σ, S, P i be a CF G. There exists an equivalent CF G , G b S, Pbi
which contains no useless productions.
Theorem 25. There exists an algorithm to construct V1 ⊆ V such that V1 = {A ∈ V | A ⇒ ⋆x ∈ Σ⋆ }.

3A variable is useless if either it can’t be reached from S or it can’t produce a terminal string

47
Algorithm 1
V1 old := φ; V1 := {A ∈ V | ∃x ∈ Σ⋆ : A −→ x ∈ P }

IN V : ∀A ∈ V1 : A ⇒ ⋆x for some x ∈ Σ⋆

while V1 old 6= V1 do
{
V1 old : V1 ;
V1 := V1 ∪ {A ∈ V | ∃ A −→ α ∈ P : variable(α) ⊆ V1 }
P1 := { A −→ α ∈ P | symbols (A −→ α) ⊆ V1 ∪ Σ }
}
end while

Claim 19. There exists an algorithm to construct Vb ⊆ V1 such that Vb = { A ∈ V1 | S ⇒ ⋆ . . . A . . . }.

Algorithm 2
Vbold := φ; Vb1 := {A ∈ V1 | S −→ . . . A · · · ∈ P1 }

while Vbold 6= Vb do
Vbold := Vb
{
foreach B ∈ V1
Vb := Vb ∪ {B ∈ V1 | ∃ A ∈ Vb : ∃ A −→ . . . B · · · ∈ P1 }
Pb := { A −→ α ∈ P1 | symbols (A −→ α) ⊆ Vb ∪ Σ }
}
end while

It is clear that Vb consists of only useful variables and Pb only useful productions from the grammar. It is
also possible to extend the algorithms to construct Σ, and Σ b which gives only the useful terminal symbols

PROOF OF UNAMBIGUITY
To prove that the grammar

S −→ ε | CS C −→ aSb

is unambiguous

C ⇒⋆ x ⇒ #a(x) = #b(x) ∧

∀y 4 x : #a(y) ≥ #b(y)

S ⇒⋆ x ⇒ #a(x) = #b(x) ∧

∀y 4 x : #a(y) ≥ #b(y)

By the substitution lemma

48
 G : S −→ ε|aSbS

Proof by contradiction. Assume L (G) is ambiguous.

⇒ ∃ ∈ L (G) : x has two distinct leftmost derivations.
Let x ∈ L (G) be the smallest length word that is ambiguous ⇒ x 6= ε
i.e. ∀y ∈ L (G) : |y| < |x| ⇒ y has a unique leftmost derivation
x 6= ε ⇒ S ⇒ aSbS ⇒⋆ aubv = x where |u| < |x| and |v| < x.
But by IH S ⇒⋆ u and S ⇒⋆ v both have unique leftmost derivation.
For allm derivation of x, ε 6= x ∈ L (G) there is a unique starting point viz.

S ⇒ aSbS.

Since x has ≤ two leftmost derivations and both derivations come from aSbS , there must exists u′ , v ′
such , u′ , v, ∈ L G

x = au′ bv ′ = aubv

but u′ 6= u
Claim 20. u 6= u′ ⇒ u ≺ u′ or u′ ≺ u
Facts. u, u′ , v, v ′ ∈ L (G)
Proof. u 6= u′ ⇒ v 6= v ′ . Since
x = aubv = au′ bv ′
⇒ ubv = u′ bv ′
WLOG assume u ≺ u′
⇒ u′ = uy with |y| > 0.(y 6= ε)
⇒ ubv = uybv ′
⇒ bv = ybv ′
⇒ y(1) = b ∧ y = bz for some |z| ≥ 0

But this is a contradication.

Since y 6= ε, y ∈ L (G) ⇒ y = ay ′
Hence the assumption u ≺ u′ is false. Similarly we may prove that the assumption u′ ≺ u is also false.

Definition 22. Any production of the A −→ ε is a null production and a variable A ∈ V is called nullable
if A ⇒⋆ ε.
Fact. A −→ ε implies A is nullable.
Lemma 15. There exists an algorithm to construct the set of all nullable variables.
Proof.

Lemma 16. There exists an algorithm to transform a given CF G , G = h V, Σ, S, P i into a positive

CF G , G+ = h V, Σ, S, P + i such that L (G+ ) = L (G) − {ε}.

Proof.

Claim 21. L (G+ ) ⊆ L (G)

Claim 22. For any A ∈ V , if A ⇒⋆G+ x 6= ε then A ⇒⋆G+ x

49
Algorithm 3
VN old := φ ;
VN := {A ∈ V | A −→ ε ∈ P } ;
while VN old 6= VN do
{
VN old : VN ;
foreach A −→ B1 . . . Bk ∈ P : {B1 , . . . , bk } ⊆ VN old do

{VN : VN ∪ {A}}
}
end while

Algorithm 4
1. Compute VN the set of all nullable variables.
2. P0+ = P − {A −→ ε | A −→ ε ∈ P } ;
foreach A −→ α ∈ P0+ : α = αB1 α1 B2 . . . αn−1 Bn αn ∧
variables(α0 α1 . . . αn ) ∩ VN = φ ∧
∀i : 1 ≤ i ≤ k : Bi ∈ VN do

{
foreach sequence s : {1, . . . , b} −→ {0, 1} :
P + := P0+ ∪ {A −→ αs | s : {1, . . . , n} −→ {0, 1}}
where the sequnce s determines which variable occurrences from α must be deleted to obtain αs.
}

Proof. By induction on the length of the derivation A ⇒⋆G x.

Theorem 26. A language A ⊆ Σ⋆ is context-free iff there exists a positive context-free grammar G+ =
h V, Σ, S, P + i with L (G+ ) = A − {ε}.
Proof. (⇐) If L (G+ ) = A − {ε} then G = h V, Σ, S, P + ∪ {S −→ ε}4 i
(⇒) from the previous lemma.

Unit productions
Definition 23. Any production of the form A −→ B , A, B ∈ V is called a unit production.

Fact. A positive grammar with no unit productions is a braching grammar.

Theorem 27. Let G+ = h V, Σ, S, P + i be a positive CF G. Then there exists a branching grammar
b = h V, Σ, S, Pb i which generates the same language.
G
Proof. Clearly G+ +
1 = h V, Σ, S, P1 i with

P1+ = P + − {A −→ A | A ∈ V }

generates the same language as G+ .

Consider all unit productions A −→ B ∈ P1+ with A 6= B. Now construct a dependency graph to find all
A 6= B such that A ⇒⋆ B.

4 This would lead to trouble with aoccuring on the RHS of a production.000

50
Let Pb = {A −→ α |α| ≥ 1, α 6∈ V } ∪

{A −→ Γ1 |Γ2 | . . . |Γk | B −→ Γ1 . . . |Γk in Pb }.

↑
there are the only rules of B in Pb

b = L (G+ ).
Claim 23. L (G)
Proof. Follows from the substitution lemma
Example 1. S −→ Aa | B
B −→ A | bb
A −→ a | bc | B

S B
S ⇒⋆ A
S ⇒⋆ B
A ⇒⋆ B
B ⇒⋆ A
A

Pb = {S −→ Aa , B −→ bb , A −→ a|bc} ∪
{S −→ a|bc|bb , A −→ bb , B −→ a|bc}.

Note that the removal of unit productions has made B useless.

b = h Vb , Σ, S, Pb i such
Theorem 28. For every CF G , G = h V, Σ, S, P i there exists a branching CF G , G
b b
that L (G) = L (G)−{ε} and G has no useless productions , no null productions and no unit productions.
Proof. We note the following properties of our transformations.
1. The removal of ε-productions may create unit productions.
2. The removal of unit - productions may create useless productions.
However , it is also true that
1. The removal of unit productions does not create any null productions.
2. The removal of useless productions does not create any fresh null or unit productions.
Hence the following algorithm proves the theorem.

Algorithm 5
1: Remove all ε-productions (to obtain a positive grammar G+ )
2: Remove all unit productions (to obtain a branching grammar Gc1 )
3: Remove all useless productions (to obtain a branching grammar Gc1 )

51
NORMAL FORMS
Definition 24. A CF G is in Chomsky Normal form (CNF) if all productions are of the form

A −→ BC or A −→ a

where A, B, C ∈ V and a ∈ Σ. It is in Greibach Normal form (GNF) if each its productions is of the
form
A −→ aB1 B2 . . . Bk
for some k ≥ 0 , A, B1 , B2 , . . . , Bk ∈ V .

Fact
1. Every grammar in CN F or GN F is positive and branching. Hence these normal forms do not
allow for generation of ε or unit productions.
Theorem 29. Every branching CF G , G may be transformed into an equivalent one in CN F .
Proof. Since G is branching it has
)
(i) no null productions Even if it has we may
assume wlog that they
and (ii) no unit productions have been removed.
The construction of the equivalent CF G , Gb in CN F proceeds as follows.

Step 1. Let G = h V, Σ, S, P i be a branching CF G. Let G′ = h V ′ , Σ, S, P ′ i be a CF G obtained from

G as follows.
(i) ∀a ∈ Σ , let Xa be a new nonterminal (Xa 6∈ V )
V ′ = V ∪ XΣ where XΣ = {Xa |a ∈ Σ}.
∀a ∈ Σ : Xa −→ a ∈ P ′
Claim 24. Every production in P ′ is of the form

A −→ a or A −→ B1 B2 . . . Bk , k > 1

and L (G′ ) = L (G).

Step 2. Obtain G′′ = h V ′′ , Σ, S, P ′′ i from G′ as follows :
Proof. Let P1 ′′ = {A −→ a|A −→ a ∈ P ′ }.
We construct P2 ′′ as follows from P ′ − P1 ′′ .
Every production in P ′ − P1 ′′ is of the form
A −→ B1 B2 . . . Bk , K > 1.
Introduce new variable C2 , . . . , Ck−1 and let
Pb2 contain the following productions

A −→ B1 C2 



C2 −→ B2 C3 
 Repeat this for every


.. such production in P ′
.  V ′′ = V ′ ∪ {C|C introduced in this step}


Ck−2 −→ Bk−2 Bk−1   P ′′ = P1 ′′ ∪ P2 ′′



Ck−1 −→ Bk−1 Bk
Claim 25. G′′ = h V ′′ , Σ, S, P ′′ i is in CN F and L (G′′ ) = L (G′ )
b from G′′ by removing useless productions.
Step 3. Obtain G

52
 b = L (G′′ ) = L (G′ ) = L (G)
Claim 26. L (G)

Greibach Normal form

Definition 25. A production is left-recursive if it is of the form A −→ Aα.
Lemma 17. (left-recursive removal). Let G = h V, Σ, S, P i be a CF G and let A −→ Aα1 | . . . |Aαr
be all the left recursive productions of A and A −→ β1 | . . . |βs be all the other productions of A. Let
G′ = h V ∪ {B}, Σ, S, P ′ i be a CF G obtained from G by adding a new variable B and replacing all the
left-recursive productions of A by the following productions.

A −→ βB | . . . | βs B.
and adding the following productions for B

B −→ α1 . . . | αr
B −→ α1 B | . . . |αr B.

Then L (G′ ) = L (G).

Proof. Consider any leftmost derivation of A in G which terminal in a sentential form not containing A.
A ⇒ Aαi1 ⇒ Aαi2 αi1 ⇒ . . . ⇒ Aαim . . . αi1 ⇒ βj αim . . . αi1 .
where 1 ≤ j ≤ s and 1 ≤ i1 , i2 , . . . , im ≤ r.
This derivation may be replaced by the following derivation in G′ .

A ⇒ βj B ⇒ βj αm B ⇒ βj αim αim−1 B ⇒ . . . ⇒ βj αim . . . αi2 B ⇒ βj αim . . . αi2 αi1

(which is not leftmost but is still a derivation). The reverse transformation may also be made. Hence
L (G) = L (G′ )

NORMAL FORMS (contd.)

Lemma 18. Any CF G in CF G may be transformed into an equivalent CF G in CN F containing no
left-recursive productions.
Proof. Let G be a grammar in CN F . The set of production of a variable A may be divided into 3 classes.
class 1 Terminal productions of the form A −→ a, A ∈ Σ.
class 2 Non-left-recursive non terminal productions of the form A −→ BC where A 6= B, A, B, C ∈ V
class 3 left-recusive productions of the form
A −→ AD where A, D ∈ V

Let A −→ AD1 | . . . |ADp be all the left recursive productions of A ,

A −→ B1 C1 | . . . |Bn Cn be all the non-left-recursive productions
and A −→ a1 | . . . |am be all the terminal productions

The removal of left-recursion involves adding a new variable E 6∈ V , replacing the left-recursive produc-
tions of A by
A −→ B1 C1 E | . . . | Bn Cn E
|a1 E| . . . |am E
and adding the following productions for E

E −→ D1 | . . . |Dp (1)
|D1 E| . . . |Dp E (2)

and adding the following productions for E

53
 E −→ D1 | . . . |Dp (3)
|D1 E| . . . |am E (4)

Claim 27. Let G′ = hV ′ , Σ, S, P ′ i where V ′ = V ∪{E} and P ′ = (P − {A −→ AD1 | . . . |ADp }) ∪ 1 ∪ 2 ∪ 3 ∪ 4

then L (G′ ) = L (G).
b in CN F such that L (G′ ) = L (G).
Claim 28. G′ may be transformed into a grammar G b

Proof. Since G′ was derived from a CN F G, it is only necessary to transform the productions of the
forms 1 , 2 ,tc3, 4 into CN F .
1 Replace A −→ B1 C1 E | . . . | Bn Cn E by
A −→ B1 E1 | . . . |Bn En —– 1′
where E1 , . . . , En are new variable
) and add the productions
E1 −→ C1 E
—– 1′′
En −→ Cn E
2 Since G was already in CN F there exist variables Xa ∈ V for each a ∈ Σ and the productions
Xa −→ a for each a ∈ Σ in G.
Replace all the productions A −→ Xa1 E | . . . | Xam E —– 2′
3 Since D1 , . . . , Dm are variables belonging to the original CN F G , their productions in G

D1 −→ δ11 | . . . | δ1k1
..
.
Dm −→ δm1 | . . . | δmkm
are already in the required form for a CN F . Hence by replacing the productions E −→ D1 | . . . | Dp by
the set of productions.

E1 −→ δ11 | . . . |δ1k1 


| 

.. —– 1′′
. 




|δm1 | . . . |δmkm
we obtain the productions 3 of E to conform to CN F .

4 These productions already conform to the CN F .

Hence Gb = h Vb , Σ, S, Pb i with
Vb = V ′ ∪ {E1 , . . . , En }.
Pb = (P − {A −→ AD1 | . . . |ADp }) ∪ 1′ ∪ 1′′ ∪ 2′ ∪ 3′ ∪ 4 is a grammar in CN F which is equivalent to
G′ .
Definition 26. A CF G G has the increasing index property(IIP) if there exists a total ordering < on
the variables such that for every production of the form A −→ Bα for A, B ∈ V , A < B. A production
satisfies the IIP if it is either of the form α −→ aα for a ∈ Σ or it is of the form A −→ Bα with B ∈ V
and A < B.
Lemma 19. Let G be a CF G in CF G with no left recursive productions. These exists an equivalent
b in CN F and an ordering on the variables of G
grammar G b such that G
b such that G has the IIP.

54
Proof. Since G is in CN F we may assume without loss of generality that for every a ∈ Σ (which occurs
in a sentence of L (g)) there exists an unique associated variable Xa and a unique production Xa −→ a.
Even if there is a variable Xab (with Xab −→ a|b) which has more then 1 terminal production , it may
be split into as many different variables as the number of terminals. Correspondingly all occurences of
Xab in any production may be spilt into two productions (one for each terminal).
Claim 29. V may be partitioned into two disjoint sets
V = N ∪ XΣ

where XΣ = Xa |a ∈ Σ and N = V − XΣ .

The ordering: Define a total ordering that satisfies the condition.

∀A ∈ N. ∀Xa ∈ XΣ : A < Xa .

Transforming G : If G already satistfies the IIP there is nothing to prove. Otherwise assume for conve-
nience that V = {A1 , . . . , Am } are all the varibles of V with index(Ai ) = i and ordered by the usual < on
integers.
Let k, 1 ≤ k ≤ m be the smallest index which violates the IIP ,i.e there exists a production.

Ak −→ Aj Al , k > j —— 1
and ∀i : 1 ≤ i < k : all the productions of Ai have the IIP.
In particular, all the productions of Aj viz.

PROCESS Aj −→ Al Al′ 1 | . . . |Alm Al′ m

I
|a1 | . . . |aq
have IIP i.e. j < l 1 , l 2 , . . . , lm
Repalce Ar −→ Aj Al by the productions

Ak −→ Al1 Al′ 1 Al | . . . | Alm Al′ mAl —— 2

| a1 Al | . . . | aq Al

Repeat PROCESS I for all Ak -productions that violate IIP.

case k > min(l1 , . . . , lm ). However since j < min(l1 , . . . , lm ) we have

j < min(l1 , . . . , lm ) < k ⇒ k − j > k − min(l1 , . . . , lm ). which implies PROCESS I has reduced the
value of.

Repeat process I as many times as necessary till either all productions of Ak have the IIP or there are
some left-recursive productions of Ak .

Case : All productions of Ak have IIP. Then

Claim 30. The smallest index that violates IIP in the grammar is > k

Case. There are left-recursive productions of Ak but all non-left-recursive productions satisfy IIP.

55
Remove left-recursion in each case by introducing a new varibale B and extend the ordering < so that
B < any existing variable of V .
Claim 31. The smallest index of the new grammar that may violate IIP is > k.
However the grammar may not be in CN F . The following case takes this into account
Case k < l1 , . . . , lm , . . . . Then the smallest index that violates IIP is > k. However the grammar is no
longer in CN F . Let the new productions of Ak be

Ak −→ Aα1 | . . . | Aln αn where |α1 | = |α2 | = · · · = |αn | = 2

|a1 Al1 | . . . |ap Alp and ∀i : 1 ≤ i ≤ n : αi ∈ V 2 .

Introduce n new variables B1 , . . . , Bn and replace the above productions by

Ak −→ Al1 B1 | . . . |Aln Bn

| Xa1 Al1 | . . . |Xap Alp

and add the new productions

B1 −→ α1
..
. where ∀i : 1 ≤ i ≤ n : αi ∈ V 2 .
Bn −→ αn
Extend the total ordering < to include the new variables B1 , . . . , Bn such that

∀i, j : Bi < Bj
Claim 32. The grammar obtained above is in CN F with the smallest index violating the IIP > k.
From the above two cases it follows that the diffence (m − k) reduces by the above processes. Hence the
above processes need to be repeated only at most (m − k) times to obtain an equivalent grammar in CN F
with the IIP
Theorem 30. (GN F ) Every branching CF G may be transformed into an equivalent one in GN F .
Proof. W LOG we may assume that G is a CN F with the IIP. Let V = {A1 , . . . , Am } be the variables
in increasing order of index in the total ordering.
Claim 33. Am has only terminal productions.

PROCESS 2
Replace every production of the form
B −→ Am C
with B −→ a1 C | . . . | an C
wher Am −→ a1 | . . . | an are the only productions of Am
Claim 34. the highest indexed variable occuring as first symbol on the right hand side of any production
is Am−1 .
Claim 35. Since there is no left recursion all the productions of Am−1 are of the form

Am−1 −→ a1 α1 | . . . | ap αp .

where α1 , . . . , αp ∈ V .

56
Proof. The replacement of Am by terminal symbol all prod. implies all productions are either of the form

A −→ aβ where β ∈V⋆, a∈Σ

or A −→ BC where A < B < Am .
PROCESS 3
By repeating PROCESS 2 for m − 1 we get that
the higest indexed variable occuring as the first
symbol on the RHS of every production is < m − 1.
This PROCESS 3 may be repeated (at most) m − 1 times to obtain a grammar in which every
production is of the form.
A −→ aα with α ∈ V ⋆
which is in GN F .

1 Pumping lemmas for linear & CF Ls

1.1 Pumping Lemma for Linear Languages
A linear language is one which may be generated by a linear grammar i.e. a grammar which may have
both right-linear & left linear productions

Fact. Every linear grammar is also contxt-free with the proof of a pumping lemma for linear languages.
We will also show the strict containment of the linear languages within the class of CF Ls
 
an
 bn 
LΣ cn

RΣ

Theorem 31. Let A ⊆ Σ⋆ be an infinite linear language. Then there exists m > 0 such that any
x ∈ A, |x| ≥ m may be decomposed into z = tuvwx with
|tuwx| ≤ m
|uw| ≥ 1
such that for all i ≥ 0, tui vwi x ∈ A.

In other words

A ⊆ Σ⋆ is linear
⇒ ∃ m > 0 : ∀z ∈ A : |z| ≥ m :
z }| {
∃t, u, v, w, x : z = tuvwx ∧ | tuwx| ≤ m ∧ |uw| ≥ 1
∧ ∀k ≥ 0 : zk = tuk vwk x ∈ A

57
Proof. Since the language is linear there exists a linear grammar gnerating it. As in the case of a CF G
we may use the same techniques to
)
(i) remove unit productions transfer G into a positive
(ii) remove ε-productions branching grammar.
(iii) transfer G to generate A − {ε}

The proof then proceeds as follows:

Consider a derivation tree as in the case of the lemma for CF Ls. Since the derivation tree may be
arbitrarily deeep consider only the first and second occurences of the first variable A that repeats itself
in the derivation tree.
Since |V | is bounded it follows that the other variables which do not repeat can only produce only
bounded-length terminal strings t and x. Similarly between the two occurences of A there are only
bounded length strings u and w may be produced.
Choose m = |t| + |u| + |w| + |x|.
Finally u and w may be pumped as many times as we please to obtain derivations of

tuk vwk x ∈ A − {ε}

2 The Pumping Lemma for CF Ls

Theorem 32. Let A =⊆ Σ⋆ be an infinite CF language. There exists m > 0 such that for any z ∈ A
with |z| ≥ m, z may be decomposed into z = tuvwx with |uvw| ≤ m , |uw| ≥ 1 such that for any
k ≥ 0 , tuk vwk x ∈ A.

In other words

A ⊆ Σ⋆ is an infinite CF L
⇒ ∃ m > 0 : ∀z ∈ A : |z| ≥ m :
∃t, w, x, v : z = tuvwx ∧ |uvw| ≤ m ∧ |uw| ≥ 1
∧ ∀k ≥ 0 : zk = tuk vwk x ∈ A

Proof. Consider A − {ε}. Let be a 

CF G with
(i) L (G) = A − {ε}. 

Let G be a branching CF G
(ii) G has no unit productions
 with L (G) = A − {ε}.

(iii) G has no ε-productions
⇒ for every production A −→ α ∈ P

α∈Σ V |A| < |α|.

Let n = max{|α| | ∃A ∈ V : A −→ α ∈ P }.
⇒ Any z ∈ A − {ε} has a derivation of length ≥ |z| n . Since A − {ε} is infinite, there exists arbitrarily
long derivations and hence there exists derivation trees of arbitrary height.

We have the following derivation

58
 S Consider such a derivation tree of

sufficient depth. Since |V | < ∞ ,

there must be at least one variable that

t x
A repeats in the derivation tree.

Let A be the variable that repeats

u w alnog the path such that no other

A
variable repeats itself between the

occurences of A.
v

S ⇒⋆ tAx ⇒⋆ tuAwx ⇒⋆ tuvwx.

where t, u, v, w, x ∈ Σ⋆ . Clearly A ⇒⋆ uAw, is a sub-derivation of the original derivation. We

may “pump” this derivation as many times as we want to obtain derivations of tuk vwk x ∈ A − {ε}. Since
G has no unit productions and is positive it follows that |uw| ≥ 1 and we may choose m = |uvw| > 1.

3 Pumping lemma : Contrapositives

∀m > 0 : ∃z ∈ A : |z| ≥ m ∧ ∀u, v, w : z = uvw ∧ |uv| ≤ m ∧ |v| ≥ 1

⇒ ∃k ≥ 0 : zk = uv k w 6∈ A

⇒ A is not regular

∀m > 0 : ∃z ∈ A : |z| ≥ m ∧ ∀u, v, w, x : z = tuvwx ∧ |uvw| ≤ m ∧ |uw| ≥ 1

⇒ ∃k ≥ 0 : zk = tuk vwk x 6∈ A

⇒ A is not a CF L

∀m > 0 : ∃z ∈ A : |z| ≥ m ∧ ∀t, u, v, w : z = tuvwx ∧ |tuwx| ≤ m ∧ |uw| ≥ 1

⇒ ∃k ≥ 0 : zk = tuk vwk x 6∈ A

⇒ A is not linear
Examples
1. an bn cn | n ≥ 0 is not CF . For any m > 0 chosen by D,C chooses am bm cm and does a case analysis.

BEST STRATEGY FOR D TO WIN

Pick a decomposition of am bm cm such that the string to be pumped viz uw satisfies
#a(uw) = #(uw) = #c(uw)

59
 But that is impossible since that implies uvw = ai bm ci and |uvw| > m for i > 0 and |v| = m if
i = 0 we may use the b′ s in v to ensure that
zk 6∈ {an bn cn | n ≥ 0}.
2. {ww | w ∈ {a, b}⋆ } is not CF . For any m > 0, C chooses am bm am bm and do case analysis.

BEST STRATEGY FOR D TO WIN

Pick a decomposition of xx = am bm am bm such that uvw contains the same length of string from
the left x as from the right.
Suppose defender chooses i > 0 with v = ε and u = bi and w = ai such that 2i ≤ m. Then it is
possible to find k > 1 with
zk = am bmm+i(k−1) amm+i(k−1) bm 6∈ {ww|w ∈ {a, b}+ }

3. {an! |n ≥ 0} is not CF Use essentially some argument as for regular case since there is only one
terminal symbol.
2
4. {an bm |n = m2 } is not CF . C chooses am bm .
5. {w ∈ {a, b}⋆ | #a(w) = b(w)} is not linear5 but is CF
since S −→ SS | aSb | bSa | ε generate it.

BEST STRATEGY FOR D TO WIN

Given z = am b2m am , DEFENDER is forced to choose tuvwx = ai for some i ≤ m, i > 0. Clearly
then challenger can choose k = 0 and ensure that #a(z0 ) 6= #b(z0 ) and |uw| ≤ 1. So defender loses

Using Brackets to remove amgiguity in CF Gs

Let G = h V, Γ, S, P i be a CF G in CN F . Assume P consists of n rules of the form
A −→ BC , 1 ≤ i ≤ n
in addition to terminal productions of the form A −→ α, a ∈ Γ.

Define Gs = h V, Γ ∪ Π, S, Ps i where
Ps = {A −→ [i B ]i C | A −→ BCis the i-th rule in P }.
∪ { A −→ a | A −→ a ∈ P }.

Example. Let G consist of the productions.

S −→ AB | BA , B −→ CC , A −→ a , C −→ a.
The string aaa ∈ L (G) has two derivation trees

5C chooses am b2m am

60
 S S

A B B A

a C C C C a

a a a a

The Chansky-Schutzenberger Theorem

Let Σ = Γ ∪ Π wher Π = { [i , ]i | 1 ≤ i ≤ n } is a collection of n pairs of parentheses of different kinds.
The language ≺ is generated by the following grammar
 
 S −→ a , for each a ∈ Γ
 

GENERALIZATION
GP i S −→ [i S]i , for each i = 1, 2, . . . , n

  OF BP = P AR{a,b} (φ)

S −→ SS | ε
Lemma 20. ≺ is a CF L satisfying the following properties
1. Γ⋆ ⊆≺.
2. ∀x, y ∈≺: xy ∈≺ i.e ≺ is closed under catenation.
3. ∀x ∈ pr : ∀i : 1 ≤ i ≤ n : [i x ]i ∈ ≺.
4. ∀x ≺:
x 6∈ 0000 ⇒ ∃u, v, w : ∃i : 1 ≤ i ≤ n :
u ∈ Γ⋆ ∧ u, w ∈≺ ∧
x = u[i v ]i w.
Proof. 1,2,3 are obvious and so we prove only 4.
4.
Claim 36. x 6∈ Γ⋆ ⇒ |x| > 1 since there is at least one pair of parenthesis in x.

⇒ Any derivation of x begins with an application of one of the rules

S −→ [i S ]i , 1 ≤ i ≤ n
S −→ SS.

We now proceed by induction assuming the result holds for all strings of length < |x|. i.e.

IH : ∀y ∈≺: |y| < |x| ⇒

[ y 6∈ Γ ⇒ ∃r, s, t : ∃j : 1 ≤ j ≤ n :
r ∈ Γ⋆ ∧ s, t ∈≺ ∧
y = r[j S ]j ] t

61
Induction steps. Do an analysis of the first step of S ⇒⋆ x.

Case. S ⇒ [i S ]i ⇒⋆ [i v ]i = x where S ⇒⋆ v. The result then follows with u = ε = w.

Case. S ⇒ SS ⇒⋆ yz = x wher S ⇒⋆ y and S ⇒⋆ z. We may safely aSSUME Y 6= ε 6= z ,

otherwise it is possible to derive x without the first step S ⇒ SS. Hence 0 < |y| < |x| and 0 < |Z| < |x|.

Subcase. y ∈ Γ⋆ . By the IH z = r[i s ]i t and x ∈ yr[i s ]i t with yr ∈ Γ⋆ , s, t ∈≺

6 Γ⋆ . By the IH y = r[i s ]i t and x ∈ r[i s ]i tz with r ∈ Γ⋆ , s, tz ∈≺
Subcase. y ∈

The modified grammar Gs has π = { (, ), [′ ], , }. and the productions.

S −→ (A)B | [B]A , B −→ {C}C , A −→ a , C −→ a.

and the corresponding derivation trees generate different strings.

S S

B A
[ B ]
( A ) C
{ C }
{ C } C
a
a a a a
a
(a) {a}a [ {a}a ]a]

Moral : Bracketing may be used as a means to specify

the derivation tree which generates a string

Theorem 33. Let h : Γ ∪ Π −→ Γ ∪ {ε} be the homomorphism which maps every symbol of Γ to itself
and erases every bracket i.e. h([i ) = ε = h(]i ). Then

h(L (Gs )) = L (G)

Proof. Show by induction on the height of derivation trees that both languages have a 1-1 correspondence
on their derivation trees
Lemma 21. L (Gs ) ⊆≺.
Proof. We prove the stonger result L (Gs ) ⊆ {x|∃A ∈ V : A ⇒⋆ x} ⊆≺. By induction on the length of
derivations A ⇒⋆ x for any A ∈ V and x ∈ (Γ ∪ Π)⋆ .

Basis. Clearly a terminal production has been applied.

Ind Steps. Otherwise A ⇒ [i B ]i C ⇒⋆ [i v ]i w = x and it follows that there exists smaller derivations
B ⇒⋆ v and C ⇒⋆ w and it is also clear that x is of the form u[i v ]i w with u = ε, v, w ∈≺. By the

62
IH there exists derivations S ⇒⋆ v and S ⇒⋆ w in ≺. Hence we have S ⇒ SS ⇒ [i S ]i S ⇒⋆ [i v ]i w
in ≺

Definition 27. Let GR = h V, Γ ∪ P i, S, PR i be the grammar with

PR = {A −→ a | A −→ a ∈ Ps } ∪ {A −→ [i B | ∃A −→ [i B]i C ∈ Ps } ∪
A −→ a]i C | A −→ a ∈ Ps , 1 ≤ i ≤ n ∧ ∃X −→ [i B ]i C C Ps
Lemma 22. L (Gr ) is regular
Proof. Since the productions are all obviously right linear , L (G′ ) is regular.
NOTE : Since GR is derived from Gs which in term is derived from G which is in CN F , it is clear that
∀A ∈ V : A 6 ⇒⋆ ε.
Lemma 23. We prove that for any A ∈ V , A ⇒⋆ x ∈ (Γ ∪ Π)⋆ then A ⇒′ ⋆ x by induction on the length
Gs G
l of derivations in Gs .

Basis l = 1. Clearly a terminal production.

Ind Step. Let A ⇒ [i B ]i ⇒⋆ x = [i v ]i w Clearly B ⇒ ⋆ v and C ⇒ ⋆ w by the induction hypothesis.
Gs GR GR
The following derivation in GR then proves the required result. Clearly since v =6 ε, we may assume
v = ua for some a ∈ Γ.
Further there must be a productions in GR of the form there and ∀i : 1 ≤ i ≤ n : Xa −→ a]i C for each
A −→ [i B]i C ∈ Ps . Hence B ⇒ v = ua would look like
GR
B ⇒ uXa ⇒ ua = v
GR
We may instead use
B ⇒ ⋆ uXa ⇒ ua]i C
GR GR
to obtain the derivation
A ⇒ [i B ⇒ ⋆ [i uXa ⇒ [i ua ]i C ⇒ ⋆ [i v ]i w
GR GR GR GR

Corollary 4. L (Gs ) ⊆≺ ∩ L (GR )

Lemma 24. ≺ ∩ L (GR ) ⊆ L (Gs )
Proof. Consider any derivation S ⇒′ ⋆ x ∈≺. We need to prove that S ⇒⋆ x. We proceed by complete
G Gs
induction on the number of occurences of symbols from Π in x.(denoted #Π(x)).

Basis #Π(x) = 0. Clearly a terminal production which also exists in Gs .

Ind steps. Assume the result is true for all strings y ∈≺ ∪ L (GR ) with #Π(y) < #Π(x) and let
#Π(y) ≥ 0. Then clearly

S ⇒ [i B for some b ∈ V and 1 ≤ i ≤ n.

GR

which implies x = [i v ]i w by the properties of ≺.

Hence
S ⇒ [i B ⇒ ⋆ [i uXa ⇒ [i ua ]i C ⇒ ⋆ [i v ]i w
GR GR GR GR

and by the induction hypothesis we have

S ⇒ [i B ]i C ⇒⋆ [i v ]i w = x
Gs Gs

Corollary 5. L (Gs ) =≺ ∩ L (GR )

63
Corollary 6. Let G = h V, Γ, S, P i be a grammar in CN F . Then for a suitably chosen Π ∩ Γ = φ of
pairs of bracketing symbols,there exists a regular language R such that
L (G) = h(R ∩ L (Gs )).
Theorem 34. CHOMSKY − SCHUTZEENBERGER. A language L ⊆ Γ⋆ is context-free if and
only if there is a regular language R and a collection Π of matching bracket symbol pairs such that

L = h(R ∩ ≺)
Proof. (⇒) For any G in CN F with L (G) = L − ε, the previous corollary shows the result.
If ε ∈ L , then

L = h((R ∪ {ε}) ∩ ≺).

(⇒) Since ≺ is context-free ot follows that L is context-free , provided we can show that (i) the intersection
of regular and a CF L is a CF L and (ii) homomorphisms preseve CF − ness
Theorem 35. The family C FΣ is closed union,concatenation and ⋆-closure
Proof. Let G1 = hV1 , Σ, S1 , P1 i and G2 = hV2 , Σ, S2 , P2 i be CF Gs generating A1 , A2 ∈ C FΣ respectively.
Assume V1 ∩ V2 = φ
Union. Let G∪ = h V∪ , Σ, S∪ , P∪ i be defined as follows
V∪ = V1 ∪ V2 ∪ {S∪ }
P∪ = P1 ∪ P2 ∪ {S∪ −→ S1 |S2 }.
Claim 37. L (G∪ ) = L (G1 ) ∪ L (G2 ) = A1 ∪ A2
Concatenation. Let G = h V •, Σ, S•, P • i be the grammar

V • = V1 ∪ V2 ∪ {S•} P • = P1 ∪ P2 ∪ {S• −→ S1 S2 }.
Claim 38. L (G•) = L (G1 ) • L (G2 ) = A1 • A2
⋆-closure. Let G⋆ = h V⋆ , Σ, S⋆ , P⋆ i where

V⋆ = V1 ∪ {S⋆ } , P⋆ = P1 ∪ { S⋆ −→ ε|S1 S⋆
Claim 39. L (G⋆ ) = L (G1 )⋆ = A⋆1

PUSHDOWN AUTOMATA
Definition 28. A nondeterministic push-down automaton(NPDA)

N = h Q, Σ, Γ, δ, q0 , ⊥, F i
Q is a finite set of states.
Σ is a finite input alphabet.
Γ is a finite stack alphabet.
δ ⊆ (Q × (Σ ∪ {ε}) × Γ) × (Q × Γ⋆ ) is a finite transition relation .
= Q × (Σ ∪ {ε}) × Γ −→f 2Q×Γ
⋆

q0 ∈ Q is the start state.

⊥ ∈ Γ is the initial stack symbol.(⊥ 6∈ Σ).
F ⊆ Q is the set of final states.
Note
N = h Q, Σ, Γ, δ, q0 , ⊥, F i
↑

64
 a Input

Finite State
P Control

A Stack

B1
..
.
Bk
β = B1 . . . Bk ⇒
↑
((p, a, A), (q, β)) ∈ δ
↑
machine is in state p next machine goes into state q
input symbol is a and top and replaces the A on the
of stack is A stack by β

this is an atomic
step.

needed only to define the initial configuration

Q × (Σ ∪ {ε}) × Γ −→f 2Q×Γ

⋆

↑
no move is possible if the stack is empty

Configurations. A configuration of the machine is a 3-tuple h q, x, α i where q ∈ Q, x is the rest of the

A1
input to be consumed and α is the contents of the stack (α = A1 . . . Ak ⇒ ..
.
Ak
⋆
(p, ay, Aβ) −→ (q, y, αβ) if ((p, a, A), (q, α)) ∈ δ
N

(p, ay, Aβ) −→⋆ (q, y, αβ) if ((p, ε, A), (q, α)) ∈ δ
N

Acceptance. There are two notions of acceptance. Actually there are two kinds of N P DAs

1. Acceptance by empty stack.(E-acceptance)

65
 L (NE ) = {x ∈ Σ⋆ |(q0 , x, ⊥) −−→⋆ (q, ε, ε)} −→ Then F is irrelevent and may be taken to be φ
NE

2. Acceptance by final state(F -acceptance)

L (N ) = {x ∈ Σ⋆ | ∃γ ∈ Γ⋆ : (q0 , x, ⊥) −−→⋆ (qf , ε, γ), qF ∈ F }

NF

Note. But in both cases the input should have been completely consumed.

Example 1. Balanced parantheses. Acceptance by empty atack


Q = {q} ((q, [ , ⊥), (q, [ ⊥))


Σ = {[, ]} ((q, [ , [ ), (q, [ [)) 
deterministic
Γ = {⊥, [} ((q, ], [ ), (q, ε))  


((q, ε, ⊥), (q, ε))

Example 2. A = {w ∈ {a, b}⋆ | #a(w) = #b(w)}

 

 ((q, ε, ⊥), (q, ε))

 


 


 
 At any stage the stack contains
Q = {q} 
 

 ((q, a, ⊥), (q, A)) 

Σ {a, b} 
 


 
 (i) only A′ s or
Γ = {A6 , B 7 , ⊥} 
 


 


 ((q, ε, ⊥), (q, ε)) 


 
 (ii) only B ′ or
Note 
 


 

1. A indicates there are more a′ s than b′ s 
 


 ((q, b, ⊥), (q, B)) 
 (iii) neither A′ s nor B ′ s
2. B indicates there are more b′ s than a′ s 
 


 ((q, a, A), (q, AA))  The stack would not

 


 
 contain a mix

 


 
 of A′ s & B ′ s

 ((q, a, B), (q, ε)) 


 


 


 


 


 ((q, b, A), (q, ε)) 


 


 


 


 

((q, b, ⊥), (q, BB))

Example 3. { wwr | w ∈ {a, b}⋆ }

Q = { qL 8 , qR } ∪ {qp 9 }
Acceptances
Σ = {a, b}
by empty
stack
Γ = {a, b, ⊥}
Note
1. qL is for start state.

66
2. qp is for acceptace by final state.

((qL , ε, ⊥), (qR , ⊥)) −→ Candidates for the middle of the string.
((qL , a, ⊥), (qL , a⊥))
NONDETERMINISTIC
ր տ
((qL , a, a), (qL , aa)) ((qL , a, a), (qR , ⊥))

((qL , a, b), (qL , ab)) ((qL , a, a), (qR , ε))

↑
((qL , b, ⊥), (qL , b⊥))
Candidates for the middle of the string.
((qL , b, a), (qL , ba)) ↓
((qL , b, b), (qL , bb)) ((qL , b, b), (qR , ε))


((qR , a, a), (qR , ε)) This does not directly allow acceptance by
final state since then the string ‘abb′

 could be accepted
((qR , b, b), (qR , ε))
((qR , ε, ⊥), (qR , ε))
| {z }
This transition allows acceptance by empty stack too.

Alternatively add a final state qF

with the rule
((qR , ε, ⊥), (qF , ⊥)).

Acceptances : E vs F

Note. When a N P DA accepts by empty stack,then F is irrelevant and may safely be taken to be φ.
Theorem 36. The two acceptances are equivalent i.e. for every NE there exists a NF with LE (NE ) =
LF (NF ) and vice-versa.
Proof. Let N = h Q, Σ, Γ, δ, q0 , ⊥, F i be any N P DA either E-type or F -type. Let s and t be two new
states not in Q. and let 4 be a new stack symbol. Now consider

N ′ = h Q ∪ {s, t}, Σ, Γ ∪ {4}, δ ′ , s ⊥, {t} i

such that δ ′ = δ ∪

Claim 40. N ′ accepts the same language by both acceptances.

Proof. Outline
(i) N ′ cannot reach its final state unless it has consumed all its input.
(ii) It cannot empty it stack (i.e. 4 cannot be popped) unless it first reaches its final state.
(iii) Once it has reached its final state,it can only empty its stack.
Hence LE = LF (N ′ ) = L N ′ .
Claim 41. LE (N ′ ) ⊆ LE (N ) = L (N ′ ) if N is of type NE
and LE (N ′ ) ⊆ LF (N ) if N is of type NF .

67
Proof. Suppose x ∈ L (N ′ ). Then for some n ≥ 0.
(s, x 4) −−−′→ (q0 , x, ⊥ 4) −−−′→n (q, y, γ, 4) −−−′→ (t, y, γ 4) −−−′→⋆ (t, ε, ε).
N N N N
| {z }
Since y = ε ⇓
(q0 , x, ⊥) −−−→n (q, ε, γ)
N
where q ∈ G.
Now consider (q, ε, γ 4) −−−′→ (t, ε, γ 4)
N

Case N by empty stack. Then G = Q and △ = {4}.

The transition (q, ε, γ 4) −−−′→ (t, ε, γ 4) implies γ = ε.
N
i.e. (q, ε, 4) −−−′→ (t, ε, 4).
N
⇓
(q0 , x, 4) −−−→n (q, ε, ε). Hence x ∈ LE (N ).
N

Case N accepts by final state. Then G = F and △ = Γ ∪ {4}.

Hence the transition (q, ε, γ 4) −−−′→ (t, ε, γ 4). implies q ∈ F .
N
Hence (q0 , x, ⊥) −−−→n (q, ε, γ) where q ∈ F . Hence x ∈ LF (N ).
N

Claim 42. LE ⊆ L (N ′ ) if N = NE
and LF ⊆ L (N ′ ) if N = NF
Proof. If x ∈ LE (N ) and N = NE then (q0 , x, ⊥) −−−→⋆ (q, ε, ε). Then it is easy to see that
N
⋆
(s, x 4) −−−′→ (q0 , x, ⊥ 4) −−−′→ (q, ε, 4) −−−′→ (t, ε, 4) −−−′→ (t, ε, ε) and hence x ∈ LE (N ′ ). On the
N N N N
other hand, if x ∈ LF (N ) and N = NF then (q0 , x, ⊥) −−−′→⋆ (qF , ε, γ) and we may construct
N
⋆
(s, x, 4) −−−′→ (q0 , x, ⊥ 4) −−−′→ (qF , ε, γ 4) −−−′→ (t, ε, γ 4) −−−′→⋆ (t, ε, ε)
N N N N

NP DAs For CF Ls

Theorem 37. For any, A ⊆ Σ⋆ , there exists a N P DA N such that L (N ) = A.

Proof. We first consider the case of A+ = A − {ε}. We know there exists a grammar in CN F (Greibach
Normal Form)
G = h V, Σ, S, P i where productions are all of the form
A −→ c B1 B2 . . . Bk
with c ∈ Σ and {b1 , . . . , bk ⊆ V .
Let N = h {q0 }, Σ, V, δ, q0S, φ10 i where
(q0 , α) ∈ δ(q0 , a, A) iff A −→ aα ∈ P

Claim 43. N simulates the leftmost derivations of G i.e.

S ⇒⋆ xα iff (q0 , x, S) −−−→⋆ (q0 , ε, α), for 0000
G N
10 i.e. we show that LE (N ) = A+

68
(⇐) By induction on n where (q0 , x, S) −−−→n (q0 , δ, α).
N

Basis n = 0. Then x = ε and s = α.

Ind Step. Let x = ya. Then clearly we have

(q0 , ya, S) −−−→n−1 (q0 , a, β) −−−→ (q0 , ε, α)
N N

from the (n − 1) steps it is clear that

(q0 , y, S) −−−→n−1 (q0 , ε, β)

N

Hence by the induction hypothesis we have S ⇒n−1 yβ. The move (q0 , a, β) −−−→ (q0 , ε, α) is possible
G N
only if β = Aγ for some A ∈ V and γ ∈ V ⋆ and further α = ηγ for some η ∈ V ⋆ . But from the
construction of N we know that this is possible only if

(q0 , η) ∈ δ(q0 , a, A)
iff A −→ aη ∈ P .

Hence S ⇒n−1 yβ ⇒ yaηγ = xα.

G G

(⇒) Suppose S ⇒n xα by a leftmost derivation. Again by induction on n we show (q0 , x, S) −−−→n

G N
(q0 , ε, α)

Ind Step. x = ya and S ⇒n−1 yAγ ⇒ yaηγ : By the IH we

G G
have (q0 , y, S) −−−→n−1 (q0 , ε, Aγ) and hence
N
(q0 , ya, S) −−−→n−1 (q0 , a, Aγ) :
N
and yAγ ⇒ yaηγ is possible only if A −→ aη ∈ P . which implies (q0 , η) ∈ δ(q0 , a, A). Hence
G
(q0 , a, Aγ) −−−→ (q0 , ε, ηγ) = (q0 , ε, α).
N

The Case When ε ∈ A. The grammar G for A+ is modified to include a new start symbol S0 6∈ V
with the productions S0 −→ Sε. It is then clear that

G0 = h V0 , Σ, S0 , P0 i where V0 = V ∪ {S0 } , P0 = P ∪ {S0 −→ S|ε}. has the property

L (G0 ) = A+ ∪ {ε} = A.

The N P DA, N may then be modified N0 with S0 being the new stack marker and

δ 0 = δ{((q0 , ε, S0 ), (q0 , ε, S)), ((q0 , ε, S0 ), (q0 , ε, ε))}

such that N0 = h {q0 }, Σ, V0 , δ0 , q0 , S0 , φ i with LE (N0 ) = LE (N ) ∪ {ε} = A+ ∪ {ε} = A.

CF Gs For NP DAs

The converse of the previous theorem viz. that the language accepted by a N P DA is a CF L is proven
in two parts

69
Lemma 25. The language E-accepted by a N P DA with a single state is a CF L
↑
The Construction of the previous theorem
needs to be inverted

Proof. Let N = h {q0 }, Σ, Γ, δ, q0 , ⊥, φ i be a N P DA. Construct G = h Γ, Σ, ⊥, P i where for c ∈ Σ ∪ {ε},

B1 . . . Bk ∈ Γ⋆

A −→ cB1 . . . Bk ∈ P iff (q0 , B1 ) . . . Bk ∈ δ(q0 , c, A).

It is easy to see that G is CF and L (G) = LE (N ).

Lemma 26. Every N P DA may be transformed into an equivalent one with a single state.

⋆ This implies that all state information may be maintained on the stack.
Theorem 38. The language accepted by any N P DA is context-free.
Lemma 27. Every N P DA may be simulated by a N P DA with a single state.
Proof. WLOG from the construction of N ′ we may assume that the given N P DA N has a single final
state.

N = h Q, Σ, Γ, δ, s, ⊥, {t} i

and that N can empty its stack after entering this state.

Consider the machine N × (which accepts by empty stack) defined by

N ⋆ = h {⋆}, Σ, Γ⋆ , δ ⋆ , ⋆, h s⊥t i, φ i

where Γ× = Q × Γ × Q.

δ × = {((⋆, c, h pAqk i), (⋆, h q0 B1 q1 ih q1 b2 q2 i . . . h qk−1 Bk qk i))

| ((p, c, A), (q0 , B1 B2 . . . Bk )) ∈ δ, c ∈ Σ ∪ {ε},
and for all possible choices of q1 , q2 , . . . , qk ∈ Q}

Intiution N × simulates the behaviour of N by guessing

nondeterministically what states N will pass through in
the future and saves those guesses on the stack and
verifies their correctness.

Claim 44. ∀n ≥ 0 : (p, x, B1 B2 . . . Bk ) −−−→n (q, ε, ε) ⇔

N
∃q0 , q1 , . . . , qk ∈ Q : p = q0 ∧ q = qk ∧
(⋆, x, h q0 B1 q1 ih q1 B2 qk i . . . h qk−1 Bk qk i) −−−→n (⋆, ε, ε)
N

Proof. By induction on n.
Basis n = 0. p = q , x = ε and k = 0 is trivial.
Ind Step. Assume n > 0 and the first step is such that
(p, x, B1 B2 . . . Bk ) −−−→ (r, y, c1 . . . cm B2 . . . Bk ) −−−→n (q, ε, ε)
N N
with x = cy and ((p, c, B1 ), (r, C1 . . . Cm )) ∈ δ and c ∈ Σ ∪ {ε}.

70
By the IH there exists r0 , r1 , . . . , rm−1 q1 , . . . , qk−1,qk such that r0 = r , q = qk and
,
(y , h r0 c1 r1 ih r1 c2 r2 i . . . h rm−1 cm q1 ih q1 B2 q2 i · · · = lanqk−1 bk qk −−→n (⋆, ε, ε).
N×
Also by the construction of N ⋆
((⋆, c, h pB1 q1 i), (⋆, h r0 c1 r1 ih r1 c2 r2 i . . . h rm−1 cm q1 i)) ∈ δ ×
combing these we get
(⋆, x, h pB1 q1 ih q1 B2 q2 i . . . h qk−1 Bk qk i)
−−→1 (⋆, y, h r0 c1 r1 ih r1 c2 r2 i . . . h rm−1 cm q1 ih q1 B2 q2 i . . . h qk−1 bk qk i)
N×
−−→1 (⋆, ε, ε).
N×
n
(⇐) Suppose (⋆, x, h q0 B1 q1 ih q1 B2 q2 i . . . h qk−1 Bk qk i) −−→ (⋆, ε, ε)
× N
and let ((⋆, c, h q0 B1 q1 i), (⋆, h r0 c1 r1 ih r1 c2 r2 i . . . h rm−1 cm q1 i)) be the first transition applied where
c ∈ Σ{ε} and m ≥ 0.

Then x = cy for some y and

(⋆, x, h q0 B1 q1 ih q1 B2 q i . . . h qk−1 Bk qk i)
−−→ (⋆, y, h r0 c1 r1 ih r1 c2 r2 i . . . h rm−1 cm q1 ih q1 B2 q2 i . . . h qk−1 bk qk i)
N ×
n
−−→ (⋆, ε, ε).
N ×

By the IH we have (r0 , y, c1 c2 . . . cm B2 . . . Bk ) −−−→n−1 (qk , ε, ε) and also by the construction of N × we

N
have ((q0 , c, B1 ), (r0 , c1 . . . cm ))inδ. Considering these we get

(q0 , x, B1 . . . Bk ) −−−→ (r0 , y, C1 . . . Cm B2 . . . Bk ) −−−→n−1 (qk , ε, ε).

N N
⋆
Proof of lemma. for all x ∈ Σ
x ∈ LF (N ⋆ ) ⇔ (⋆, x, h s⊥t i) −−→⋆ (⋆, ε, ε)
N×
⇔ (s, x, ⊥) −−−→⋆ (t, ε, ε)
N
⇔ x ∈ LE (N ).

DERTERMINISTIC P DAs
Definition 29. A DP DA is a N P DA satisfying the following constraints
∀a ∈ Σ ∪ {ε} : ∀A ∈ Γ : ∀q ∈ Q : |δ(q, a, A)| ≤ 1
i.e. δ : Q × (Σ ∪ {ε}) × Γ−7→Q × Γ⋆

(i) in other words δ is a partial function which may be undefined for certain triples in Q × (Σ ∪ {ε}) × Γ
(ii) δ(q, ε, A) = (q ′ , α) ⇒ ∀a ∈ Σ : ⇒ δ(q, a, A) is undefined.

i.e. geven any input symbol a there is at most

choice whether to consumed it or not and if it
is to be consumed there is exactly one next
configuration.
Examples. The language {an bn | n ≥ 0} has a DP DA
{w ∈ {a, b}⋆ | #a(w) = #b(w)} has a DP DA.

Some of our other examples e.g. {wwr | w ∈ {a, b}⋆ } have non-deterministic transitions which decide
whether the midpoint has been reached.

71
However wcwR | w ∈ {a, b}⋆ has a deterministic P DA since the midpoint is clearly marked by a different
symbol.

THE POWER OF DP DAs

Example. The language A2 = {an b2n | n > 0} is accepted by a DP DA

Q2 = {qa , qb }, Σ = {a, b} , Γ2 = {B, ⊥} , F2 = φ

δ2 : (qa , a, ⊥) 7→ (qa , BB⊥) (qa , b, B) 7→ (qb , ε)
(qa , a, B) 7→ (qa , BBB) (qb , b, B) 7→ (qb , ε)
(qb , ε, ⊥) 7→ (qb , ε, ε).0000

Example. Similarly the language A1 = {an bn | n > 0} has a DP DA.

The construction is exactly as above except that δ1
δ1 : (qa , a, ⊥) 7→ (qa , B⊥) (qa , b, B) 7→ (qb , ε)
(qa , a, B) 7→ (qa , BB) (qb , b, B) 7→ (qb , ε)
(qb , ε, ⊥) 7→ (qb , ε, ε).

Note that by changing Σ to Σ ∪ {c} where c is a new terminal symbol,the basic construction of P DAs
remains unchanged.

Clearly the language A1 ∪ A2 has a P DA obtained by taking the union of δ1 ∪ δ2 . However this P DA is
nonderterministic because of the first two rules. But we can go further.
Theorem 39. A1 ∪ A2 is not accepted by any DP DA.
Proof. Assume there exists a DP DA D12 = h Q12 , Σ, Γ, δ12 , q0 , ⊥, F12 i accepting A1 ∪ A2 .
Claim 45. There exists a N P DA which accepts the language A! ∪ A@ ∪ A3 where A3 = {an bn cn |n > 0}.
Proof. Let N123 = h Q123 , Σ, Γ, δ123 , q0 , ⊥, F123 i
where Q123 = F12 ∪ F3 i.e. Q3 = {b q | q ∈ Q12 }
F123 = F12 ∪ F3 with F3 = {b q | q ∈ F12 }
δ123 = δ12 ∪ δ3 where δ3 = {((qF , ε, A), (c qF , A) | qF ∈ F12 , A ∈ Γ}
∪ {((qbi , c, A), (qbj , α) | α ∈ Γ⋆ ∧
δ12 (qi , b, A) = (qj , α)}

δ3
cn

ε ε

D12
a n bn bn

72
We have for d12 to accept an bn , n > 0
(q0 , an bn , ⊥) −−→⋆ (qF , ε, α) with qF ∈ F is unique and since D12 is determinstic ,for an b2n we
D12
have
(q0 , an b2n , ⊥) −−→⋆ (qF , bn , α) −−→⋆ (qF ′ , ε, α′ )
D12 D12
with qf ′ which implies that for aN bn cn we have by construction
(q0 , an bn cn , ⊥) −−−→⋆ (qF , cn , α) −−−→⋆ (qc′ ′
F , ε, α ) with q
cF ∈ F123 .
N123 N123
Hence N123 acceptes A123
But we know that A123 is not CF since A3 is not CF . Hence the assumption that there is a DP DA D12
which accepts A12 is wrong !

73