Context Free Grammars

𝐺!

}
S → 0S1
S→R (Substitution) Rules
R→ε
In 𝐺! :
Rule: Variable → string of variables and terminals 3 rules
Variables: Symbols appearing on left-hand side of rule R,S
Example of 𝐺! generating a string
Terminals: Symbols appearing only on right-hand side 0,1
Start Variable: Top left symbol S Tree of S S Resulting
substitutions string
Grammars generate strings 0S1 0S1
1. Write down start variable
0S1 00S11
2. Replace any variable according to a rule
Repeat until only terminals remain R 00R11
3. Result is the generated string
ε 0011 ∈ 𝐿 𝐺!
4. 𝐿(𝐺) is the language of all generated strings.
𝐿 𝐺! = 0" 1" 𝑘 ≥ 0}
11 Check-in 3.3
 Context Free Grammars
𝐺!

}
S → 0S1
S→R (Substitution) Rules
R→ε
In 𝐺! :
Rule: Variable → string of variables and terminals 3Check-in
rules 3.3 𝐺#
Variables: Symbols appearing on left-hand side of rule R,S
S → RR
Example R
of 𝐺
→! generating
0R1 a string
Terminals: Symbols appearing only on right-hand side 0,1
Start Variable: Top left symbol S Tree of RS → ε S Resulting
substitutions string
Grammars generate strings Check all of the strings 0that
S 1 are 0inS1𝐿(𝐺# ):
1. Write down start variable (a) 001101 0 S 1 00S11
2. Replace any variable according to a rule
Repeat until only terminals remain
(b) 000111
R 00R11
3. Result is the generated string (c) 1010 ε 0011 ∈ 𝐿 𝐺!
4. 𝐿(𝐺) is the language of all generated strings.
(d) ε 𝐿 𝐺! = 0" 1" 𝑘 ≥ 0}
11 Check-in 3.3
 Context Free Grammars (CFGs)
𝐺! Shorthand:
S → 0S1
S→R S → 0S1 | R
R→ε R→ε
Recall that a CFG has terminals, variables, and rules.
Example of 𝐺! generating a string
Grammars generate strings
1. Write down start variable Tree of S S Resulting
2. Replace any variable according to a rule substitutions string
Repeat until only terminals remain “parse tree” 0S1 0S1
3. Result is the generated string
0S1 00S11
4. 𝐿(𝐺) is the language of all generated strings
5. We call 𝐿(𝐺) a Context Free Language. R 00R11
ε 0011 ∈ 𝐿 𝐺!
𝐿 𝐺! = 0" 1" 𝑘 ≥ 0}
2
 CFG – Formal Definition
Check-in 4.1
Defn: A Context Free Grammar (CFG) Which of these are valid CFGs?
𝐺 is a 4-tuple (𝑉, Σ, 𝑅, 𝑆) 𝐶!: B → 0B1 | ε 𝐶#: S → 0S | S1
B1 → 1B R → RR
𝑉 finite set of variables 0B → 0B
Σ finite set of terminal symbols
∗ a) 𝐶! only
𝑅 finite set of rules (rule form: 𝑉 → 𝑉 ∪ Σ ) b) 𝐶# only
𝑆 start variable c) Both 𝐶! and 𝐶#
d) Neither
For 𝑢, 𝑣 ∈ 𝑉 ∪ Σ ∗ write
1) 𝑢 ⇒ 𝑣 if can go from 𝑢 to 𝑣 with one substitution step in 𝐺
2) 𝑢 ⇒∗ 𝑣 if can go from 𝑢 to 𝑣 with some number of substitution steps in 𝐺
𝑢 ⇒ 𝑢! ⇒ 𝑢# ⇒ ⋯ ⇒ 𝑢$ = 𝑣 is called a derivation of 𝑣 from 𝑢.
If 𝑢 = 𝑆 then it is a derivation of 𝑣.

𝐿 𝐺 = 𝑤 𝑤 ∈ Σ∗ and 𝑆 ⇒∗ 𝑤}
Defn: 𝐴 is a Context Free Language (CFL) if 𝐴 = 𝐿(𝐺) for some CFG 𝐺.
3 Check-in 4.1
 CFG – Example
𝐺# Parse E E Resulting
E → E+T | T tree string
T → T×F | F E+T E+T

F→(E)|a T T×F T+T×F

F F a F+F×a
𝑉= {E, T, F}
Σ= {+, ×, (, ), a} a a a a+a×a ∈ 𝐿 𝐺#
𝑅= the 6 rules above
Generates a+a×a, (a+a)×a, a, a+a+a, etc.
𝑆= E
Check-in 4.2
Observe that the parse tree contains additional
information, such as the precedence How many reasonable distinct meanings
of × over + . does the following English sentence have?
The boy saw the girl with the mirror.
If a string has two different parse trees, then it is
derived ambiguously, and we say that the (a) 1
grammar is ambiguous. (b) 2
4 (c) 3 or more Check-in 4.2
 Ambiguity
𝐺# 𝐺% E
E → E+T | T E → E+E | E×E | ( E ) | a
E E
T → T×F | F
E E
F→(E)|a
a + a × a
Both 𝐺! and 𝐺" recognize the same language, i.e., E E
𝐿 𝐺! = 𝐿 𝐺" . E E
However, 𝐺! is an unambiguous CFG and 𝐺" is E
ambiguous.

5
 Pushdown Automata (PDA)
“head”
Finite
control a b a b a … a
input appears on a “tape”

c Schematic diagram for PDA

Schematic diagram (pushdown)
d
for DFA or NFA stack
d

Operates like an NFA except can write-add or read-remove symbols

from the top of stack.
push pop

Example: PDA for 𝐷 = 0# 1# 𝑘 ≥ 0

1) Read 0s from input, push onto stack until read 1.
2) Read 1s from input, while popping 0s from stack.
3) Enter accept state if stack is empty. (note: acceptance only at end of input)
6
 PDA – Formal Definition
Defn: A Pushdown Automaton (PDA) is a 6-tuple (𝑄, Σ, Γ, 𝛿, 𝑞0, 𝐹)
Σ input alphabet
Γ stack alphabet
𝛿: Q×Σ% ×Γ% → 𝒫(𝑄×Γ% ) Accept if some thread is in the accept state
𝛿 𝑞, a, c = 𝑟!, d , 𝑟#, e at the end of the input string.

Example: PDA for 𝐵 = {𝑤𝑤 ℛ | 𝑤 ∈ 0,1 ∗ } Sample input: 0 1 1 1 1 0

1) Read and push input symbols.
Nondeterministically either repeat or go to (2). The nondeterministic forks replicate the stack.
2) Read input symbols and pop stack symbols, compare.
If ever ≠ then thread rejects. This language requires nondeterminism.
3) Enter accept state if stack is empty. (do in “software”) Our PDA model is nondeterministic.

7
 Converting CFGs to PDAs
Theorem: If 𝐴 is a CFL then some PDA recognizes 𝐴 𝐺# E → E+T | T
Proof: Convert 𝐴’s CFG to a PDA T → T×F | F
F→(E)|a
…
E → E+T | T PDA
T→…
F→… CFG E E
E+T E+T
IDEA: PDA begins with starting variable and guesses substitutions.
It keeps intermediate generated strings on stack. When done, compare with input. T+T×F T T×F
E E T T F+F×a F F a
+ Input: a + a × a
+ +
T T T a+a×a a a a
×
F

Problem! Access below the top of stack is cheating!

Instead, only substitute variables when on the top of stack.
If a terminal is on the top of stack, pop it and compare with input. Reject if ≠.

8
 Converting CFGs to PDAs (contd)
Theorem: If 𝐴 is a CFL then some PDA recognizes 𝐴
Proof construction: Convert the CFG for 𝐴 to the following PDA. 𝐺# E → E+T | T
1) Push the start symbol on the stack. T → T×F | F
2) If the top of stack is F→(E)|a
Variable: replace with right hand side of rule (nondet choice).
Terminal: pop it and match with next input symbol. E E
3) If the stack is empty, accept. E+T E+T
T+T×F T T×F
Example: a + a × a
F+F×a F F a
E E F T a + T T
+ + + + T × a+a×a a a a
T T T T F

9
 Equivalence of CFGs and PDAs
Theorem: 𝐴 is a CFL iff* some PDA recognizes 𝐴
Done.
In book.
* “iff” = “if an only if” means the implication goes both ways.
So, we need to prove both directions: forward (→) and reverse (←).
Given PDA 𝑃 = 𝑄, Σ, Γ, 𝛿, 𝑞G , 𝑞HIIJKL we
Check-in 4.3
construct CFG G:
Is every Regular Language also
1. For each 𝑝, 𝑞, 𝑟, 𝑠 ∈ 𝑄, 𝑢 ∈ Γ, 𝑎, 𝑏 ∈ ΣM if 𝛿(𝑝, 𝑎, 𝜀) a Context Free Language?
contains (𝑟, 𝑢) and 𝛿(𝑠, 𝑏, 𝑢) contains (𝑞, 𝜀), put (a) Yes
𝐴KN → 𝑎𝐴OP 𝑏 in G. (b) No
2. For each 𝑝, 𝑞, 𝑟 ∈ 𝑄 put 𝐴KN → 𝐴KO 𝐴ON in G. (c) Not sure
3. For each 𝑝 ∈ 𝑄 put 𝐴KK → 𝜀 in G. Check-in 4.3
Start variable is 𝐴N" N#$$%&' .
10
 Recap
Recognizer Generator

Regular Regular
DFA or NFA
language expression
Context Free Context Free
PDA
language Grammar

Context Free
languages
Regular
languages

11
 Equivalence of CFGs and PDAs
Recall Theorem: 𝐴 is a CFL iff some PDA recognizes 𝐴
Done.
Need to know the fact, not the proof

Corollaries:
1) Every regular language is a CFL.
2) If 𝐴 is a CFL and 𝐵 is regular then 𝐴 ∩ 𝐵 is a CFL.
Proof sketch of (2):
While reading the input, the finite control of the PDA for 𝐴 simulates the DFA for 𝐵.

Note 1: If 𝐴 and 𝐵 are CFLs then 𝐴 ∩ 𝐵 may not be a CFL (will show today).
Therefore the class of CFLs is not closed under ∩.
Note 2: The class of CFLs is closed under ∪,∘,∗ (see Pset 2).

2
 Proving languages not Context Free
Let 𝐵 = 0" 1" 2" 𝑘 ≥ 0}. We will show that 𝐵 isn’t a CFL.

Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝

such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 ' 𝑥𝑦 ' 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε ≥𝑝
3) 𝑣𝑥𝑦 ≤ 𝑝 𝑢 𝑣 𝑥 𝑦 𝑧
𝑠= ∈𝐴
Informally: All long strings in 𝐴 are pumpable and stay in 𝐴.
𝑦 𝑦 𝑧 ∈𝐴
𝑢 𝑣 𝑣 𝑥
≤𝑝

3
 Pumping Lemma – Proof
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 & 𝑥𝑦 & 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝 E

Proof by picture: E R
E
R
𝑢 𝑣 𝑦 𝑧 R
R
R 𝑥
𝑣 𝑥 𝑦
Generates 𝑢𝑣𝑣𝑥𝑦𝑦𝑧 𝑢 𝑧
R Generates 𝑢𝑥𝑧
= 𝑢𝑣 " 𝑥𝑦 " 𝑧 = 𝑢𝑣 # 𝑥𝑦 # 𝑧
𝑠= 𝑢 𝑣 𝑦 𝑧
𝑥 Long 𝑠 → “cutting and pasting” argument
tall parse tree

4
 Pumping Lemma – Proof details
For 𝑠 ∈ 𝐴 where 𝑠 ≥ 𝑝, we have 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where:
1) 𝑢𝑣 & 𝑥𝑦 & 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0 …cutting and pasting
2) 𝑣𝑦 ≠ ε …start with the smallest parse tree for 𝑠
3) 𝑣𝑥𝑦 ≤ 𝑝 …pick the lowest repetition of a variable

Let 𝑏 = the length of the longest right hand side of a rule (E → E+T)
= the max branching of the parse tree E E
Let ℎ = the height of the parse tree for 𝑠. E+T
A tree of height ℎ and max branching 𝑏 has at most 𝑏' leaves. want
So 𝑠 ≤ 𝑏' . R ℎ> 𝑉
Let 𝑝 = 𝑏 ( + 1 where 𝑉 = # variables in the grammar.
R
So if 𝑠 ≥ 𝑝 > 𝑏 ( then 𝑠 > 𝑏|(| and so ℎ > 𝑉 .
Thus at least 𝑉 + 1 variables occur in the longest path. 𝑠= 𝑢 𝑣 𝑥 𝑦 𝑧
So some variable 𝑅 must repeat on a path.
use 𝑠 > 𝑏 (
set 𝑝 = 𝑏 ( + 1
5
 Example 1 of Proving Non-CF
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 & 𝑥𝑦 & 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε
3) 𝑣𝑥𝑦 ≤ 𝑝

Let 𝐵 = 0" 1" 2" 𝑘 ≥ 0}

Show: 𝐵 is not a CFL
Proof by Contradiction:
Check-in 5.1
Assume
Let 𝐴! = get
(to 0$ 1a$contradiction) that 𝐵#s
2+ 𝑘, 𝑙 ≥ 0} (equal is aofCFL . 1s)
0s and
The CFL pumping + $ lemma
$ gives 𝑝 as above. Let 𝑠 = 0* 1* 2* ∈ 𝐵.
Let 𝐴# = 0 1 2 𝑘, 𝑙 ≥ 0} (equal #s of 1s and 2s) 𝑠 = 00 ⋯ 0011 ⋯ 1122 ⋯ 22
Pumping lemma says that can divide 𝑠 = 𝑢𝑣𝑥𝑦𝑧 satisfying the 3 conditions.
Condition 3that
Observe PDAs≤can
( 𝑣𝑥𝑦 𝑝) recognize and cannot
𝐴! 𝑣𝑥𝑦
implies that 𝐴#. What can we
contain bothnow
0s conclude?
and 2s. 𝑢 𝑣 𝑥 𝑦 𝑧
Soa)𝑢𝑣The
# 𝑥𝑦class
# of unequal
𝑧 has CFLs is not closed under
numbers intersection.
of 0s, 1s, and 2s. ≤𝑝
b) 𝑢𝑣
Thus, The#𝑥𝑦Pumping
# 𝑧 ∉ 𝐵,Lemma
violatingshows that 𝐴1.
Condition 𝐴# is not a CFL .
! ∪Contradiction!
c) The class
Therefore, of CFLs is closed
our assumption (𝐵 is under
a CFL) complement.
is false. We conclude that 𝐵 is not a CFL .
6
Check-in 5.1
 Example 2 of Proving Non-CF
Pumping Lemma for CFLs: For every CFL 𝐴, there is a 𝑝
such that if 𝑠 ∈ 𝐴 and 𝑠 ≥ 𝑝 then 𝑠 = 𝑢𝑣𝑥𝑦𝑧 where
1) 𝑢𝑣 & 𝑥𝑦 & 𝑧 ∈ 𝐴 for all 𝑖 ≥ 0
2) 𝑣𝑦 ≠ ε 𝑠! = 000 ⋯ 001000 ⋯ 001
3) 𝑣𝑥𝑦 ≤ 𝑝
𝑢 𝑣 𝑥 𝑦 𝑧
Let 𝐹 = 𝑤𝑤 𝑤 ∈ Σ ∗ } . Σ = {0,1}.
Show: 𝐹 is not a CFL.
𝑠# = 0 ⋯ 01 ⋯ 10 ⋯ 01 ⋯ 1
Assume (for contradiction) that 𝐹 is a CFL.
𝑢 𝑣 𝑥 𝑦 𝑧
The CFL pumping lemma gives 𝑝 as above. Need to choose 𝑠 ∈ 𝐹. Which 𝑠?
Try 𝑠! = 0* 10* 1 ∈ 𝐹. But 𝑠! can be pumped and stay inside 𝐹. Bad choice of 𝑠.
Try 𝑠# = 0* 1* 0* 1* ∈ 𝐹.
Show 𝑠# cannot be pumped 𝑠# = 𝑢𝑣𝑥𝑦𝑧 satisfying the 3 conditions.
Condition 3 implies that 𝑣𝑥𝑦 does not overlap two runs of 0s or two runs of 1s.
Therefore, in 𝑢𝑣 #𝑥𝑦 #𝑧, two runs of 0s or two runs of 1s have unequal length.
So 𝑢𝑣 #𝑥𝑦 #𝑧 ∉ 𝐹 violating Condition 1. Contradiction! Thus, 𝐹 is not a CFL.

7
 Turing Machines (TMs)
head
a b a b b ˽ ˽ ...
Finite
control read/write input tape

1) Head can read and write

2) Head is two way (can move left or right)
3) Tape is infinite (to the right)
4) Infinitely many blanks “˽“ follow input
5) Can accept or reject any time (not only at end of input)

8
 TM – example

TM recognizing 𝐵 = a" b" c" 𝑘 ≥ 0

1) Scan right until ˽ while checking if input is in a∗b∗c∗, reject if not.
2) Return head to left end. head
input tape
3) Scan right, crossing off single a, b, and c. a a a b b b c c c ˽ ˽
Finite
4) If the last one of each symbol, accept. control
5) If the last one of some symbol but not others, reject.
6) If all symbols remain, return to left end and repeat from (3). accept

Check-in 5.2
How do we get the effect of “crossing off” with a Turing machine?
a) We add that feature to the model.
b) We use a tape alphabet Γ = {a, b, c, a, b, c, ˽ }.
c) All Turing machines come with an eraser.

9 Check-in 5.2
 TM – Formal Definition
Defn: A Turing Machine (TM) is a 7-tuple (𝑄, Σ, Γ, 𝛿, 𝑞: , 𝑞acc , 𝑞rej)
Σ input alphabet
Γ tape alphabet (Σ ⊆ Γ)
𝛿: Q×Γ → 𝑄×Γ× {L, R} (L = Left, R = Right)
𝛿 𝑞, a = (𝑟, b, R)

On input 𝑤 a TM 𝑀 may halt (enter 𝑞acc or 𝑞rej)

Check-in 5.3
or 𝑀 may run forever (“loop”).
This Turing machine model is deterministic.
So 𝑀 has 3 possible outcomes for each input 𝑤: How would we change it to be nondeterministic?
1. Accept 𝑤 (enter 𝑞acc ) a) Add a second transition function.
2. Reject 𝑤 by halting (enter 𝑞rej ) b) Change 𝛿 to be 𝛿: Q×Γ → 𝒫( 𝑄×Γ× {L, R} )
3. Reject 𝑤 by looping (running forever) c) Change the tape alphabet Γ to be infinite.

10 Check-in 5.3