F ORMAL L ANGUAGES , AUTOMATA AND

C OMPUTATION
D ECIDABILITY

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 1 / 34

T URING M ACHINES -S YNOPSIS
The most general model of computation
Computations of a TM are described by a sequence of
configurations. (Accepting Configuration, Rejecting
Configuration)
Turing-recognizable languages
TM halts in an accepting configuration if w is in the
language.
TM may halt in a rejecting configuration or go on indefinitely
if w is not in the language.
Turing-decidable languages
TM halts in an accepting configuration if w is in the
language.
TM halts in a rejecting configuration if w is not in the
language.
Nondeterministic TMs are equivalent to Deterministic TMs.
( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 2 / 34
D ESCRIBING T URING M ACHINES AND T HEIR
I NPUTS
For the rest of the course we will have a rather standard
way of describing TMs and their inputs.
The inputs to TMs have to be strings.
Every object O that enters a computation will be
represented with a string hOi, encoding the object.
For example if G is a 4 node undirected graph with 4 edges
hGi = (1,2,3,4)((1,2),(2,3),(3,1),(1,4))
Then we can define problems over graphs,e.g., as:

A = {hGi | G is a connected undirected graph}

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 3 / 34

D ECIDABILITY
We investigate the power of algorithms to solve problems.
We discuss certain problems that can be solved
algorithmically and others that can not be.
Why discuss unsolvability?
Knowing a problem is unsolvable is useful because
you realize it must be simplified or altered before you find an
algorithmic solution.
you gain a better perspective on computation and its
limitations.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 4 / 34

OVERVIEW
Decidable Languages
Diagonalization
Halting Problem as a undecidable problem
Turing-unrecognizable languages.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 5 / 34

D ECIDABLE L ANGUAGES
S OME NOTATIONAL DETAILS

hBi represents the encoding of the description of an

automaton (DFA/NFA).
We need to encode Q, Σ, δ and F .

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 6 / 34

E NCODING F INITE AUTOMATA AS S TRINGS
Here is one possible encoding scheme:
Encode Q using unary encoding:
For Q = {q0 , q1 , . . . qn−1 }, encode qi using i + 1 0’s, i.e.,
using the string 0i+1 .
We assume that q0 is always the start state.
Encode Σ using unary encoding:
For Σ = {a1 , a2 , . . . am }, encode ai using i 0’s, i.e., using the
string 0i .
With these conventions, all we need to encode is δ and F !
Each entry of δ, e.g., δ(qi , aj ) = qk is encoded as

0i+1 1 |{z}
|{z} 0j 1 |{z}
0k+1
qi aj qk

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 7 / 34

E NCODING F INITE AUTOMATA AS S TRINGS
The whole δ can now be encoded as
00100001000
| {z {z } · · · 1 |000000100000010
} 1 |000001001000000 {z }
transition1 transition2 transitiont

F can be encoded just as a list of the encodings of all the

final states. For example, if states 2 and 4 are the final
states, F could be encoded as

000 1 |00000
|{z} {z }
q2 q4

The whole DFA would be encoded by

11 |00100010000100000
{z · · · 0} 11 |0000000010000000
{z } 11
encoding of the transitions encoding of the final states

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 8 / 34

E NCODING F INITE AUTOMATA AS S TRINGS
hBi representing the encoding of the description of an
automaton (DFA/NFA) would be something like
hBi = 11 |00100010000100000
{z · · · 0} 11 |0000000010000000
{z } 11
encoding of the transitions encoding of the final states

In fact, the description of all DFAs could be described by a

regular expression like
11(0+ 10+ 10+ 1)∗ 1(0+ 1)+ 1
Similarly strings over Σ can be encoded with (the same
convention)
hwi = 0000 {z } 1 · · · |{z}
| {z } 1 |000000 0
a4 a6 a1

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 9 / 34

E NCODING F INITE AUTOMATA AS S TRINGS
hB, wi represents the encoding of a machine followed by an
input string, in the manner above (with a suitable separator
between hBi and hwi.
Now we can describe our problems over languages and
automata as problems over strings (representing automata
and languages).

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 10 / 34

D ECIDABLE P ROBLEMS
R EGULAR L ANGUAGES

Does B accept w?
Is L(B) empty?
Is L(A) = L(B)?

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 11 / 34

T HE ACCEPTANCE P ROBLEM FOR DFA S

T HEOREM 4.1
ADFA = {hB, wi | B is a DFA that accepts input string w} is a
decidable language.

P ROOF
Simulate with a two-tape TM.
One tape has hB, wi
The other tape is a work tape that keeps track of which state
of B the simulation is in.
M = “On input hB, wi
1 Simulate B on input w
2 If the simulation ends in an accept state of B, accept; if it
ends in a nonaccepting state, reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 12 / 34

T HE ACCEPTANCE P ROBLEM FOR NFA S

T HEOREM 4.2
ANFA = {hB, wi | B is a NFA that accepts input string w} is a
decidable language.

P ROOF
Convert NFA to DFA and use Theorem 4.1
N = “On input hB, wi where B is an NFA
1 Convert NFA B to an equivalent DFA C, using the
determinization procedure.
2 Run TM M in Thm 4.1 on input hC, wi
3 If M accepts accept; otherwise reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 13 / 34

T HE G ENERATION P ROBLEM FOR R EGULAR
E XPRESSIONS

T HEOREM 4.3
AREX = {hR, wi | R is a regular exp. that generates string w} is
a decidable language.

P ROOF
Note R is already a string!!
Convert R to an NFA and use Theorem 4.2
P = “On input hR, wi where R is a regular expression
1 Convert R to an equivalent NFA A, using the Regular
Expression-to-NFA procedure
2 Run TM N in Thm 4.2 on input hA, wi
3 If N accepts accept; otherwise reject.”
( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 14 / 34
T HE E MPTINESS P ROBLEM FOR DFA S

T HEOREM 4.4
EDFA = {hAi | A is a DFA and L(A) = Φ} is a decidable
language.

P ROOF
Use the DFS algorithm to mark the states of DFA
T = “On input hAi where A is a DFA.
1 Mark the start state of A
2 Repeat until no new states get marked.
Mark any state that has a transition coming into it from any
state already marked.
3 If no final state is marked, accept; otherwise reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 15 / 34

T HE E QUIVALENCE P ROBLEM FOR DFA S

T HEOREM 4.5
EQDFA = {hA, Bi | A and B are DFAs and L(A) = L(B)} is a
decidable language.

P ROOF
Construct the machine for
L(C) = (L(A) ∩ L(B)) ∪ (L(A) ∩ L(B)) and check if L(C) = Φ.
T = “On input hA, Bi where A and B are DFAs.
1 Construct the DFA for L(C) as described above.
2 Run TM T of Theorem 4.4 on input hCi.
3 If T accepts, accept; otherwise reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 16 / 34

D ECIDABLE P ROBLEMS
C ONTEXT- FREE L ANGUAGES

Does grammar G generate w?

Is L(G) empty?

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 17 / 34

T HE G ENERATION P ROBLEM FOR CFG S

T HEOREM 4.7
ACFG = {hG, wi | G is a CFG that generates string w} is a
decidable language.

P ROOF
Convert G to Chomsky Normal Form and use the CYK
algorithm.
C = “On input hG, wi where G is a CFG
1 Convert G to an equivalent grammar in CNF
2 Run CYK algorithm on w of length n
3 If S ∈ Vi,n accept; otherwise reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 18 / 34

T HE G ENERATION P ROBLEM FOR CFG S
A LTERNATIVE P ROOF
Convert G to Chomsky Normal Form and check all
derivations up to a certain length (Why!)
S = “On input hG, wi where G is a CFG
1 Convert G to an equivalent grammar in CNF
2 List all derivations with 2n − 1 steps where n is the length of
w. If n = 0 list all derivations of length 1.
3 If any of these strings generated is equal to w, accept;
otherwise reject.”

This works because every derivation using a CFG in CNF either

increase the length of the sentential form by 1 (using a rule like
A → BC or leaves it the same (using a rule like A → a)
Obviously this is not very efficient as there may be exponentially
many strings of length up to 2n − 1.
( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 19 / 34
T HE E MPTINESS P ROBLEM FOR CFG S

T HEOREM 4.8
ECFG = {hGi | G is a CFG and L(G) = Φ} is a decidable
language.

P ROOF
Mark variables of G systematically if they can generate
terminal strings, and check if S is unmarked.
R = “On input hGi where G is a CFG.
1 Mark all terminal symbols G
2 Repeat until no new variable get marked.
Mark any variable A such that G has a rule A → U1 U2 · · · Uk
and U1 , U2 , . . . Uk are already marked.
3 If start symbol is NOT marked, accept; otherwise reject.”

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 20 / 34

T HE E QUIVALENCE P ROBLEM FOR CFG S

EQCFG = {hG, Hi | G and H are CFGs and L(G) = L(H)}

It turns out that EQDFA is not a decidable language.

The construction for DFAs does not work because CFLs are
NOT closed under intersection and complementation.
Proof comes later.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 21 / 34

D ECIDABILITY OF CFL S

T HEOREM 4.9
Every context free language is decidable.

P ROOF
Design a TM MG that has G built into it and use the result of
ACFG .
MG = “On input w
1 Run TM S (from Theorem 4.7) on input hG, wi
2 If S accepts, accept, otherwise reject.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 22 / 34

ACCEPTANCE P ROBLEM FOR TM S

T HEOREM 4.11
ATM = {hM, wi | M is a TM and M accepts w} is undecidable.

Note that ATM is Turing-recognizable. Thus this theorem

when proved, shows that recognizers are more powerful
than deciders.
We can encode TMs with strings just like we did for DFA’s
(How?)

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 23 / 34

ACCEPTANCE P ROBLEM FOR TM S
The TM U recognizes ATM
U = “On input hM, wi where M is a TM and w is a string:
1 Simulate M on w
2 If M ever enters its accepts state, accept; if M ever enters its
reject state, reject.
Note that if M loops on w, then U loops on hM, wi, which is
why it is NOT a decider!
U can not detect that M halts on w.
ATM is also known as the Halting Problem
U is known as the Universal Turing Machine because it can
simulate every TM (including itself!)

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 24 / 34

T HE D IAGONALIZATION M ETHOD
S OME BASIC DEFINITIONS

Let A and B be any two sets (not necessarily finite) and f be

a function from A to B.
f is one-to-one if f (a) 6= f (b) whenever a 6= b.
f is onto if for every b ∈ B there is an a ∈ A such that
f (a) = b.
We say A and B are the same size if there is a one-to-one
and onto function f : A −→ B.
Such a function is called a correspondence for pairing A
and B.
Every element of A maps to a unique element of B
Each element of B has a unique element of A mapping to it.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 25 / 34

T HE D IAGONALIZATION M ETHOD
Let N be the set of natural numbers {1, 2, . . .} and let E be
the set of even numbers {2, 4, . . .}.
f (n) = 2n is a correspondence between N and E.
Hence, N and E have the same size (though E ⊂ N ).
A set A is countable if it is either finite or has the same size
as N .
Q = { mn | m, n ∈ N } is countable!
Z the set of integers is countable:
 n
 2 n even
f (n) =
 n+1
− 2 n odd

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 26 / 34

T HE D IAGONALIZATION M ETHOD

T HEOREM
R is uncountable

P ROOF .
Assume f exists and every number in R is n f (n)
listed. 1 3.14159. . .
2 55.77777. . .
Assume x ∈ R is a real number such that
3 0.12345. . .
x differs from the j th number in the j th
decimal digit.
4 0.50000. . .
.. ..
. .
If x is listed at some position k, then it x = .4527 . . .
differs from itself at k th position; otherwise
defined as
the premise does not hold
such, can not
f does not exist be on this list.
( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 27 / 34
D IAGONALIZATION OVER L ANGUAGES

C OROLLARY
Some languages are not Turing-recognizable.

P ROOF
For any alphabet Σ, Σ∗ is countable. Order strings in Σ∗ by length
and then alphanumerically, so Σ∗ = {s1 , s2 , . . . , si , . . .}

The set of all TMs is a countable language.

Each TM M corresponds to a string hMi.
Generate a list of strings and remove any strings that do not
represent a TM to get a list of TMs.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 28 / 34

D IAGONALIZATION OVER L ANGUAGES

P ROOF ( CONTINUED )
The set of infinite binary sequences, B, is uncountable. (Exactly
the same proof we gave for uncountability of R)

Let L be the set of all languages over Σ.

For each language A ∈ L there is unique infinite binary sequence

XA
The i th bit in XA is 1 if si ∈ A, 0 otherwise.

Σ∗ = { , 0, 1, 00, 01, 10, 11, 000, 001, ··· }

A={ 0, 00, 01, 000, 001, ··· }
XA ={ 0 1 0 1 1 0 0 1 1 ··· }

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 29 / 34

D IAGONALIZATION OVER L ANGUAGES

P ROOF ( CONTINUED )
The function f : L −→ B is a correspondence. Thus L is
uncountable.

So, there are languages that can not be recognized by some TM.
There are not enough TMs to go around.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 30 / 34

T HE H ALTING P ROBLEM IS U NDECIDABLE

T HEOREM
ATM = {hM, wi | M is a TM and M accepts w}, is undecidable.

P ROOF
We assume ATM is decidable and obtain a contradiction.
Suppose H decides ATM

accept if M accepts w
H(hM, wi) =
reject if M does not accept w

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 31 / 34

T HE H ALTING P ROBLEM IS U NDECIDABLE

P ROOF ( CONTINUED )
We now construct a new TM D
D = “On input hMi, where M is a TM
1 Run H on input hM, hMii.
2 If H accepts, reject, if H rejects, accept”
So 
accept if M does not accept hMi
D(hMi) =
reject if M accepts hMi

When D runs on itself we get


accept if D does not accept hDi
D(hDi) =
reject if D accepts hDi

Neither D nor H can exist.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 32 / 34
W HAT H APPENED TO D IAGONALIZATION ?
Consider the behaviour of all possible deciders:
hDi
hM1 i hM2 i hM3 i hM4 i ··· hMj i ···
M1 accept reject accept reject ··· accept ···
M2 accept accept accept accept ··· accept ···
M3 reject reject reject reject ··· reject ···
M4 accept accept reject reject ··· accept ···
.. .. ..
. . .
D = Mj reject reject accept accept ··· ? ···
.. .. ...
. .
D computes the opposite of the diagonal entries!

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 33 / 34

A T URING U NRECOGNIZABLE L ANGUAGE
A language is co-Turing-recognizable if it is the complement
of a Turing-recognizable language.
A language is decidable if it is Turing-recognizable and
co-Turing-recognizable.
ATM is not Turing recognizable.
We know ATM is Turing-recognizable.
If ATM were also Turing-recognizable, ATM would have to be
decidable.
We know ATM is not decidable.
ATM must not be Turing-recognizable.

( L ECTURE 15) S LIDES FOR 15-453 S PRING 2011 34 / 34