Chapter:5 Undecidability

Church Turing Thesis[1]

What does computable mean?
➢ Meaning of term Computable was given by:
• Alonzo church (Lambda calculus)
• Allen Turing (Turing Machine)

➢ Turing Machine can be carried out any algorithmic procedure that can be carried
out at all,by a human computer or a team of humans and electronic
[Link] statement usually referred to as Church’s thesis, or the Church-
Turing thesis.

➢ The Turing machine is a general model of computation. Turing Machine can be

solved any algorithmic procedure.
➢ Church Turing thesis can be stated as, the assumption that the intuitive notation
of computable function can be identified with partial recursive functions.
➢ Some enhancement to TM made the church-turing thesis acceptable. This
enhancement are: [1]

• Multi-tape
• Multi-head
• Infinite taps
• Non-determinism

➢ Machine that are closer to modern computers in their operation, the various
notational system that can be used to describe computations. In every case
model has been shown to equivalent to the Turing machine.
➢ Since the introduction of TM, no one has suggested an algorithm that can be
solved by a computer but cannot be solved by a TM.

➢ Here is an informal summary of some of the evidence.

➢ Humans normally work with a two-dimensional sheet of paper. A TM tape
could be organized so as to simulate two dimensions; one likely
 consequence would be that the TM would require more moves to do what
a human could do in one.
➢ Various enhancements of the TM model have been suggested to make
the operation more like that of a human computer, or more convenient &
efficient. The multitape TM is an example.
➢ Other theoretical models includes abstract machines with two stacks or
with a queue.
➢ Since the introduction of the Turing machine, no one has suggested any
type of computation that ought to be included in the category of
“algorithmic procedure” and cannot be implemented on a TM.

Universal Turing Machine[1]

➢ A Turing Machine can also behave like a general purpose computer. General
purpose computer solves a problem as given below :
• A program written in a high level language and its machine code obtained
with the help of compiler.
• Machine code is loaded in to main memory.
• Input to the program can be also loaded into main memory.
• Program stored in a memory is executed line by line.

➢ We can follow a similar approach for a TM. Such, a TM is known as Universal

Turing Machine (UTM). UTM can solve all type of solvable problem.

Finite control

I /P

Tape
000101000 ….
State

➢ Universal Turing Machine should be able to simulate every Turing machine.

Simulation of a Turing will involve:[1]
• Encoding behaviour of a particular TM as a program.
• Execution of the above program by UTM.
➢ We can assume the UTM as a 3-tape Turing machine.[1]
• Input is written on the first tape.
• Moves of the TM is encoded from is written on the second tape.
 • The current state of TM is written on third tape.

There are two types of TMs (based on halting):

➢ Halting Turing Machine: Halting Turing machine Accepts Recursive
[Link] Machine that always halt, no matter accepting or non no matter
accepting or [Link] is called as decidable problems.
➢ Turing Machine: Turing Machine Accepts Recursively [Link]
Machine that are guaranteed to halt are guaranteed to halt only on acceptance
only on acceptance. If non-accepting, it may or may not halt (i.e., could loop
forever).It is either decidable or partially decidable.

❖ Decidable Problem
➢ If there is a Turing machine that decides the problem, called as Decidable
problem.
➢ A decision problem that can be solved by an algorithm that halts on all inputs in a
finite number of steps.[2]
➢ A problem is decidable, if there is an algorithm that can answer either yes or no.
➢ A language for which membership can be decided by an algorithm that halts on
all inputs in a finite number of steps.
➢ Decidable problem is also called as totally decidable problem, algorithmically
solvable, recursively solvable.[2]

❖ Undecidable Problem (Semi-dedidable or Totally not decidable)[2]

➢ A problem that cannot be solved for all cases by any algorithm.
➢ Equivalent Language cannot be recognized by a Turing machine that halts for all
inputs.

The following problems are undecidable problems:

➢ Halting Problem: A halting problem is undecidable problem. There is no general
method or algorithm which can solve the halting problem for all possible inputs.
➢ Emptyness Problem: Whether a given TM accepts Empty?
➢ Finiteness Problem: Whether a given TM accepts Finite?
➢ Equivalence Problem: Whether Given two TM’s produce same language?.
Is L(TM1) = L(TM2) ?
Is L(TM1) ⊆ L(TM2) ? (Subset Problem)
Is L(TM1) Ո L(TM2) = CFL?
Is L(TM1) = Σ* ? (Totality Problem)
 Is the complement of L(G1) context-free ?

➢ Undecidable problems are two types: Partially decidable (Semi-decidable) and

Totally not decidable.
• Semi decidable: A problem is semi-decidable if there is an algorithm that
says yes. if the answer is yes, however it may loop infinitely if the answer
is no.

• Totally not decidable (Not partially decidable): A problem is not

decidable if we can prove that there is no algorithm that will deliver an
answer.

Undecidability[3]
We shall prove that not all problems/languages are decidable. In other words there
exists languages that are not accepted by any halting Turing machine. The proof uses
the diagonalization method to construct a language and then argue that there does not
exist any TM that accepts this language.

A Non Turing Recognizable Language[3]

We use the diagonalization argument to show the existence of a non Turing
recognizable language. First let us lay down a few notations and conventions that will
be required for our proof.
➢ Every TM has a finite description (states, transition function, etc.).
➢ Description of a TM can be thought of as a finite string over {0, 1}. This can be
achieved in many ways. If the description contains k different symbols, then for
each symbol associate a string over {0, 1} of length [log k]. Now replace every
symbol by its corresponding bit string. Hence we get a string over {0, 1} for
every TM. We denote this description of a TM M as (M ).
➢ Using the bit string representation described earlier, we can associate a natural
number to every bit string in the following natural way. Append a 1 to the left of
the string (to distinguish between preceding 0’s) and convert it to decimal.
➢ If a natural number does not correspond to any TM then associate it with some
fixed TM (say a TM that accepts all strings).
➢ By the above scheme, every natural number corresponds to some TM and for
every TM there is at least one natural number representing it. For i ≥ 1, let Mi
denote the i-th TM. This gives us a surjective function from natural numbers to
the set of all TMs.
 ➢ Theorem 1. There exists a language that is not Turing recognizable.[3]

Proof.
Let s1, s2, . . . be the lexicographic ordering of all binary strings. Define
Ld = {si | Mi does not accept si}
Suppose Ld is Turing recognizable. Then there is a TM Mj such that Ld = L(Mj)
(that is, Mj is the j-th TM in the sequence described above).
Now,
Mj accept sj ⇐⇒ sj ∈ L(Mj) ⇐⇒ sj ∈ Ld ⇐⇒ Mj does not accept sj.

Therefore we have a contradiction and Ld is not Turing recognizable.

Undecidable Languages[3]
➢ Consider the language
ATM = {(M, w) | M accepts w}.

➢ It is easy to show that AT M is Turing recognizable. Given an input (M, w),

simulate M on w. If M accepts w then accept and if M rejects w then reject .

Theorem 2. ATM is undecidable.[3]

Proof:
Suppose ATM is decidable. Then there exists a halting TM H that on input (M, w),
accepts if M accepts w and rejects if M does not accept w.

Using H, construct another machine N as follows:

Description of Turing machine N

Input: (M ) where M is a TM
• Simulate H on (M, (M )). (A description of H is hardcoded into the machine
N .)
• If H rejects then accept else reject.

Note that the machine H is a halting TM, hence N is also a halting TM. Now
consider what happens when the machine N is provided with the input (N ).

N accepts (N ) ⇐⇒ H rejects (N, (N )) ⇐⇒ N does not accept (N )

This is a contradiction. Hence ATM is undecidable.

Halting Problem[3]
➢ We can show that a language is undecidable by “reducing” a known undecidable
language to the given language. Later we will formalize the definition of
reduction. But for now let us show the undecidability of the halting problem by
proving that if the halting problem is decidable then AT M is also decidable.
The halting problem is defined as

HTM = {(M, w) | M halts on w}.

➢ Suppose HTM is decidable via a halting TM [Link] H we will construct a halting

TM H’ for ATM as follows:

Description of Turing machine H’

➢ Input: (M, w) where M is a TM and w is a string
• Simulate H on (M, w).
• If H rejects (M, w) then reject.
• If H accepts then simulate M on w.

➢ If M accepts then accept and if M rejects w then reject.

Observe that when we simulate M on w in the above algorithm it will always halt. This is
because if M does not halt on w then H rejects in Step 2 itself and we do not proceed to
Step 3.

Exercise 1. Show that if ATM is decidable then HTM is decidable (essentially converse of
the above construction).
Exercise 2. Show that HTM is Turing recognizable.

Note 1:ATM is not Turing recognizable. This is because we know that A TM is Turing
recognizable and if ATM is also Turing recognizable then it implies that A TM is decidable.
The same is true for HTM as well.

This can be generalized to give the following Proposition.

Proposition 3. If a language L is undecidable and Turing recognizable then L is not
Turing recognizable.
Reducibility[3]
➢ Converting one problem to another.
Examples:
• going around in a city reduces to ability to read a map
• getting good grades reduces to solving problems
• finding max/min reduces to sorting
• multiplication reduces to addition (doing it several times)

➢ Computable Functions
Henceforth, we will also discuss TMs that compute a general function as opposed to
outputting just accept or reject . We will assume that a TM has three tapes, namely

➢ a read-only input tape,

➢ a read-write work tape, and
➢ a write-only output tape.
➢ Since the output tape is write-only, the tape head of the output tape is
restricted to move only from left to right. There is a special halting state in the
TM. On any input, as soon as the TM enters the halting state, the
computation of the TM stops and whatever string is written on the output tape,
is the output of the TM on that particular input. For a TM M and on an input x,
we will denote M (x) to be the output of M on x.

Deftnition 4.1:[3] A function f : Σ∗ → Σ∗ is said to be computable if there exists a TM M

with output tape, such that for all x ∈ Σ∗, f (x) = M (x).

Deftnition 4.2:[3] We say that a language L1 reduces to a language L2 if there exists a

computable function f , such that for all x ∈ Σ∗,
x ∈ L1 ⇐⇒ f (x) ∈ L2.
This is denoted as L1 ≤m L2.

L1
L2
f

L1
L2
 ➢ Note that f is a function, hence every string gets mapped to some string. But
every string in L2 or L̄2 need not have a pre-image.

The m in the notation of reduction stands for “many-to-one”.

➢ If L1 ≤m L2 then intuitively it means that L2 is at least as hard as L1. In other
words, a solution to L2 would give a solution to L1.
➢ A halting TM is also known as a decider.

Theorem 4: If L1 ≤m L2 and L2 is decidable then L1 is decidable.[3]

➢ Proof:
Let f be the computable function such that for all x ∈ Σ∗, x ∈ L1 ⇐⇒ f (x) ∈ L2. Let
M
be a decider for L2. We can then construct a decider for L1 as follows.

Given x ∈ Σ∗, first compute f(x). Simulate M on f(x). If M accepts then accept else
reject.

Notes: Let L1 and L2 be two languages such that L1 ≤m L2.

➢ If L1 is undecidable then L2 is undecidable.
➢ If L1 is not Turing recognizable then L2 is not Turing recognizable.

Examples
Example 1:Consider the language
ETM = {(M ) | M is a TM and L(M ) = ∅}.[3]

➢ Prove that ATM ≤m ETM.

We will construct a computable function f that takes as input (M, w) and produces an
output (M’) such that M does not accept w ⇐⇒ L(M’) = ∅.

The reduction function f

Input: (M, w)
• Construct a TM M’ that on input x does the following (note that M’ has
description of M and w hardcoded into its description):
a. If x ƒ= w then reject
 b. Else simulate M on w. If M accepts then accept and if M rejects then reject
. Output: (M’)

➢ Proof of correctness
Now,
M does not accept w =⇒ M’ does not accept any input =⇒ L(M’) = ∅
M accepts w =⇒ M‘ accepts only the string w =⇒ L(M’) = {w}

Therefore, M does not accept w ⇐⇒ L(M’) = ∅ and hence ATM≤m [Link] proves
that ETM is not Turing recognizable.

Example 2: Consider the language

EQTM = {(M1, M2) | M1, M2 are TMs and L(M1) = L(M2)}.[3]

➢ Prove that ETM ≤m EQTM.

We will construct a computable function f that takes as input (M ) and produces an

output
(M1, M2) such that L(M) = ∅ ⇐⇒ L(M1) = L(M2).

The reduction function f

Input: (M)
• Set M1 := M .
• Construct a TM M2 that rejects all inputs (that is L(M2) = ∅).
Output: (M1, M2)

➢ Proof of correctness
Now,
L(M ) = ∅ ⇐⇒ L(M1) = ∅ ⇐⇒ L(M1) = L(M2)

Therefore, L(M ) = ∅ ⇐⇒ L(M1) = L(M2) and hence ETM ≤m EQTM. This proves that
EQTM is not Turing recognizable.

Example 3: Consider the language

REGTM = {(M ) | M is a TM and L(M ) is regular}.[3]

➢ Prove that ATM ≤mREGTM .

We will construct a computable function f that takes as input (M, w) and produces an
output (M’) such that M does not accept w ⇐⇒ L(M’) is regular.

The reduction function f

Input: (M, w)
• Construct a TM M’ that on input x does the following (note that M’ has description
of M and w hardcoded into its description):
a. Simulate M on w.
b. If M accepts w then check if x is of the form 0n1n or not. If so then accept else
reject.
c. If M rejects w then reject .
Output: (M’)

➢ Proof of correctness
Now,
M does not accept w =⇒ M’ does not accept any input =⇒ L(M’) = ∅ =⇒ L(M’) is
regular
M accepts w =⇒ L(M’) = {0n1n | n ≥ 0} =⇒ L(M’) is not regular

Therefore, M does not accept w ⇐⇒ L(M’) is regular.

Hence ATM ≤[Link] proves that REGTM is not Turing recognizable.

Similar approach can be used to show that checking whether the language of a TM is a
CFL or not is undecidable.

Example 4: Consider the language

CONTM = {(M1, M2, M3) | M1, M2, M3 are TMs and L(M1) = L(M2) • L(M2)}.[3]

➢ Prove that EQTM ≤m CONTM.

We will construct a computable function f that takes as input (N1, N2) and produces an
output (M1, M2, M3) such that L(N1) = L(N2) ⇐⇒ L(M1) = L(M2) • L(M3).

The reduction function f

Input: (N1, N2)
 • Set M1 := N1 and M2 := N2.
• Construct a TM M3 that accepts the string s only and rejects every other string.
Output: (M1, M2, M3)

➢ Proof of correctness
Now,
L(N1) = L(N2) ⇐⇒ L(N1) = L(N2) • {€} ⇐⇒ L(M1) = L(M2) • L(M3)

Therefore, L(N1) = L(N2) ⇐⇒ L(M1) = L(M2) • L(M3) and hence EQTM≤m CONTM.
This proves that CONTM is not Turing recognizable.

Exercise 3. If L1 ≤m L2 then L1 ≤m L2.

Rice’s Theorem[3]
➢ A property of a language is a function,
P : {set of all languages} −→ {0, 1}.
➢ For a language L, if P (L) = 1 then we say that L satisfies P and if P (L) = 0 then
we say that L does not satisfy P .

➢ Examples of properties of languages:

• the language contains the string 01001
• the language is empty
• the language has exactly 340 strings

➢ A property P is said to be a non-trivial property of languages of Turing machines

if there exists two TMs M1 and M2 such that P (L(M1)) = 0 and P (L(M2)) = 1.

Theorem 10 (Rice’s Theorem). Let P be a non-trivial property of languages of TMs.

Then the language
LP = {(M ) | L(M ) satisfies P}
is undecidable. [3]

Proof:
➢ We will show that AT M ≤m LP .
Case 1
P (∅) = 0
 Since P is a non-trivial property, there exists a TM N such that P (L(N )) = [Link]
will construct a computable function f that takes as input (M, w) and produces an
output (M’) such that M accepts w ⇐⇒ L(M’) satisfies P.

➢ The reduction function f

Input: (M, w)
Construct a TM M J that on input x does the following (note that M J has
description of M and w hardcoded into its description):
• Simulate M on w.
• If M accepts w then simulate N on x. If N accepts then accept and if N rejects
then reject .
• If M rejects w then reject .
Output: (M’)

➢ Proof of correctness
Now,
M accepts w =⇒ L(M J) = L(N ) =⇒ L(M’) satifies P
M does not accept w =⇒ L(M’) = ∅ =⇒ L(M’) does not satisfy P

Therefore ATM ≤ Lp .

Case 2
P (∅) = 1
We can reduce this to Case 1. The complement property ̅P is also a non-trivial property
of languages of TMs. Since P (∅) = 1 therefore ̅P(∅) = [Link] Case 1 we have ATM ≤m L ̅ p
. Hence ̅ATM ≤m ̅Lp (see Exercise 3).

But ̅LP = LP .
Therefore ATM ≤ LP.
Therefore LP is undecidable.
Refrences
1. Darshan TOC study Material: [Link]
Computer-Engineering-Study-Material
2. Gradeup Study Notes Undecidability: [Link]
32c748fa-bea5-11e5-8fa3-b0415042f6e8
3. IIT Kanpur TOC Material: [Link]
[Link]