THEORY OF COMPUTATION

Automata theory (also known as Theory Of Computation) is a theoretical

branch of Computer Science and Mathematics, which mainly deals with the
logic of computation with respect to simple machines, referred to as
automata.
Automata enables scientists to understand how machines compute the
functions and solve problems. The main motivation behind developing
Automata Theory was to develop methods to describe and analyse the
dynamic behaviour of discrete systems.
Automata originated from the word “Automaton” which is closely related to
“Automation”.

Basic Terminologies of Theory of Computation:

Symbol:
A symbol (often also called a character) is the smallest building block, which
can be any alphabet, numbers, or picture.

Alphabets (Σ):
Alphabets are a set of symbols, which are always finite.

String:
A string is a finite sequence of useful symbols from some alphabet. A string
is generally denoted as w and the length of a string is denoted as |w|.
Empty string is the string with zero occurrence of symbols,
represented as ε.
Number of Strings (of length 2)
that can be generated over the alphabet {a, b}:
- -
a a
a b
b a
b b

Length of String |w| = 2

Number of Strings = 4

1
For alphabet {a, b} with length n, number of
strings can be generated = 2 n.

Empty String

The empty string or null string, denoted by ε, is the string consisting of no

letters, no matter what type of language we are considering.

String concatenation (join)

Given two strings w1 and w2, we define the concatenation of w1 and w2 to be

the string as w1w2.

Examples

• If w1 = pq and w2 = r, then w1w2 = pqr.

• If w1 = acc and w2 = ac, then w1w2 = accac and w2w1 = acacc.
• If w1 = ∧ and w2 = cd, then w1w2 = cd.
• If w1 = aa and w2 = ∧, then w1w2 = aa.
• If w1 = ∧ and w2 = ∧, then w1w2 = ∧; because ∧∧ = ∧.

Substring

Given a string s, substring of s is any part of the string s means w is a

substring of s.

Examples

Take the string 472828. Then ∧, 282, 4, and 472828 are all substrings of
472828.

48 is not a substring of 472828.

Prefix of a string: Any number of leading symbols of the string is known as

prefix of a string.

Ex. Let w=abc

Where , prefix of w are, ε, a ,ab, abc.

Where , ε u an empty string with length zero i.e. |ε|=0.

2
 Proper Prefix of a string: prefix other than the string itself is known as
proper prefix of a string.

Ex. Let w= abc

Where, proper prefix of string is : ε, a, ab.

Suffix of a string: Any number of trailing symbols of the string is

known as suffix of a string.

Ex. Let w= abc

Where, suffix of string were, ε, c, bc, abc.

Proper Suffix of a string: the suffix other than the string itself
is known as suffix of a string.

Ex. Let w= abc

Where, suffix of string were, ε, c, bc.

Power set: If A is a set then the power set of A , denoted as

2A={P/P IS A SUBSET OF A } THAT IS SET OF ALL POSSIBLE SUBSETS OF A
.

EXAMPLE: A= {0,1}

2A=[{},{0},{1},{0,1}]

Closure Representation in TOC:

The Kleene star –
∑* is also called as “Kleene Operator” or “Kleene closure” is a unary operator on a
set of symbols or strings.
∑*, that gives the infinite set of all possible strings of all possible lengths over ∑
including ε.

From the above two statements, it can be concluded that:

Σ * = εL

3
Example:
Regular expression for language accepting all combination of g's over
Σ={g}:
L = g*
L={ε,g,gg,ggg,gggg,ggggg,...}
*******************************

Kleen Plus –
The set ∑+ is the infinite set of all possible strings of all possible lengths over ∑
excluding ε.

Regular Expression for language accepting all combination of g's over

Σ={g} excluding null:
L = g+
L={g,gg,ggg,gggg,ggggg,gggggg,...}
Language:
A language is a set of strings, chosen from some alphabet (Σ*). A language that can
be formed over ‘ Σ ‘ can be Finite or Infinite.
Example of Finite Language: Σ(x,y)
L1 = { set of string of length 2 }
L1 = { xy, yx, xx, yy }

Example of Infinite Language: Σ(a,b)

L1 = { set of all strings starts with 'b' }

L1 = {b, babb, baa, ba, bbb, baab, ....... }

4
 Chomsky hierarchy
According to Chomsky hierarchy grammar is divided into 4 types as
follows:
1. Type 0 is known as unrestricted grammar.
2. Type 1 is known as context-sensitive grammar.
3. Type 2 is known as a context-free grammar.
4. Type 3 Regular Grammar.

Type 0: Unrestricted Grammar:

Type-0 grammars include all formal grammar. Type 0 grammar languages
are recognized by Turing machine. These languages are also known as the
Recursively Enumerable languages or unrestricted grammer.

Grammar Production in the form of

⍺ --> β

Where ⍺ is ( V + T)* V ( V + T)* and

β is (V+T).
Where:
V : Variables
T : Terminals.

In type 0 there must be at least one variable on the Left side of production.
For example:
abS --> ba
A --> S
Here, Variables are S, A, and Terminals a, b.

5
 Type 1: Context-Sensitive Grammar
Type-1 grammars generate context-sensitive languages. The language generated
by the grammar is recognized by the Linear Bound Automata
In Type 1
• First of all Type 1 grammar should be Type 0.
• Grammar Production in the form of
⍺ --> β
|⍺| <= |β|
That is the count of symbol in ⍺ is less than or equal to β
+
Also β ∈ (V + T)
i.e. β can not be ε

For Example:
S --> AB
AB --> abc
B --> b

Type 2: Context-Free Grammar: Type-2 grammars generate context-free

languages. The language generated by the grammar is recognized by a Pushdown
automata. In Type 2:
• First of all, it should be Type 1.
• The left-hand side of production can have only one variable and there is
no restriction on β
For example:
S --> AB
A --> aABb
B --> b
Type 3: Regular Grammar: Type-3 grammars generate regular languages. These
languages are exactly all languages that can be accepted by a finite-state
automaton. Type 3 is the most restricted form of grammar.
Type 3 should be in the given form only :
V --> VT / T (left-regular grammar)
(or)
V --> TV /T (right-regular grammar)
For example:
S --> a
The above form is called strictly regular grammar.
There is another form of regular grammar called extended regular grammar. In
this form:
S --> VT* / T*. (extended left-regular grammar)
(or)
S --> T*V /T* (extended right-regular grammar)
For example :
S --> ab

6
S --> ab
A --> aaaaabbbbbbA
B --> abB

S --> ab
A --> Bab
B --> Sab
Finite Automata(FA):

Finite Automata(FA) is the simplest machine to recognize patterns. It is used

to characterize a Regular Language.
Also it is used to analyze and recognize regular Expressions. The finite
automata or finite state machine is an abstract machine that has five
elements or tuples. It has a set of states and rules (transition function) for
moving from one state to another The following figure shows some
essential features of general automation.

Figure: Features of Finite Automata

The above figure shows the following features of automata:

Finite automata can be represented by input tape and finite control.

Input tape: It is a linear tape having some number of cells. Each input symbol is placed
in each cell.

Finite control: The finite control decides the next state on receiving particular input
from input tape. The tape reader reads the cells one by one from left to right, and at a
time only one input symbol is read.

A Finite Automata consists of the following:

Q : Finite set of states.

7
 Σ : set of Input Symbols.
q : Initial state.
F : set of Final States.
δ : Transition Function.
Formal specification of machine is { Q, Σ, q, F, δ }
FA is characterized into two types:
1) Deterministic Finite Automata (DFA):
DFA consists of 5 tuples {Q, Σ, q, F, δ}.
Q : set of all states.
Σ : set of input symbols. ( Symbols which machine takes as input )
q : Initial state. ( Starting state of a machine )
F : set of final state.
δ : Transition Function, defined as δ : Q X Σ --> Q.

In a DFA, for a particular input character, the machine goes to one state only. A transition
function is defined on every state for every input symbol. Also in DFA null (or ε) move is
not allowed, i.e., DFA cannot change state without any input character.
For example, construct a DFA which accept a language of all strings ending with ‘a’.
Given: Σ = {a,b}, q = {q0}, F={q1}, Q = {q0, q1}
First, consider a language set of all the possible acceptable strings in order to construct
an accurate state transition diagram.
L = {a, aa, aaa, aaaa, aaaaa, ba, bba, bbbaa, aba, abba, aaba, abaa…….}
Above is simple subset of the possible acceptable strings there can many other strings
which ends with ‘a’ and contains symbols {a,b}.

Fig 1. State Transition Diagram for DFA with Σ = {a, b}

Strings not accepted are,

ab, bb, aab, abbb, etc.
State transition table for above automaton,

8
 ⇣State \Symbol⇢ a b

q0 q1 q0

q1 q1 q0

One important thing to note is, there can be many possible DFAs for a pattern. A DFA
with a minimum number of states is generally preferred.

2) Nondeterministic Finite Automata(NFA): NFA is similar to DFA except

following additional features:
1. Null (or ε) move is allowed i.e., it can move forward without
reading symbols.
2. Ability to transmit to any number of states for a particular
input.
However, these above features don’t add any power to NFA. If we compare
both in terms of power, both are equivalent.
Due to the above additional features, NFA has a different transition
function, the rest is the same as DFA.
δ: Transition Function
δ: Q X (Σ U ε ) --> 2 ^ Q.
As you can see in the transition function is for any input including null (or ε),
NFA can go to any state number of states. For example, below is an NFA for
the above problem.

Fig 2. State Transition Diagram for NFA with Σ = {a, b}

State Transition Table for above Automaton,

⇣State \Symbol⇢ a b

q0 {q0,q1} q0

9
⇣State \Symbol⇢ a b

q1 ∅ ∅

One important thing to note is, in NFA, if any path for an input string leads
to a final state, then the input string is accepted. For example, in the
above NFA, there are multiple paths for the input string “aa”. Since one of
the paths leads to a final state, “aa” is accepted by the above NFA.

DFA NFA

DFA stands for Deterministic NFA stands for Nondeterministic Finite

Finite Automata. Automata.

For each input symbol, there is For each input symbol, there may be
only one state transition in DFA. many state transition in NFA.

DFA can be understood as one NFA can be understood as multiple little

machine. machines computing at the same time.

DFA is more difficult to

NFA is easier to construct.
construct.

All DFA are NFA. Not all NFA are DFA.

δ: QxΣ -> Q i.e. next possible δ: Qx(Σ U ε) -> 2^Q i.e. next possible
state belongs to Q. state belongs to power set of Q.

Conversion of Regular Conversion of Regular expression to

expression to DFA is difficult. NFA is simpler compared to DFA.

Epsilon move is not allowed in

Epsilon move is allowed in NFA
DFA

Moore Machine Mealy Machine

10
 Output depends only upon the Output depends on the present state
present state. as well as present input.

More states are required. Less number of states are required.

There is less hardware requirement There is more hardware requirement

for circuit implementation. for circuit implementation.

They react slower to inputs They react faster to inputs.

Output is placed on states. Output is placed on transitions.

Easy to design. It is difficult to design.

If input changes, output does not If input changes, output also

change changes.

Has more or the same states as that Has fewer or the same states as that
of the Mealy machine. of the Moore machine.

Pushdown Automata
Pushdown Automata is a finite automata with extra memory called stack
which helps Pushdown automata to recognize Context Free Languages.

A Pushdown Automata (PDA) can be defined as :

• Q is the set of states
• ∑is the set of input symbols
• Γ is the set of pushdown symbols (which can be pushed and
popped from stack)
• q0 is the initial state
• Z0 is the initial pushdown symbol (which is initially present in
stack)
• F is the set of final states
• δ is a transition function which maps Q x {Σ ∪ ∈} x Γ -→Q x Γ*.
• In a given state, PDA will read input symbol and stack symbol
(top of the stack) and move to a new state and change the symbol
11
 of stack.

• PDA Components:
• Input tape: The input tape is divided in many cells or symbols. The input head
is read-only and may only move from left to right, one symbol at a time.
• Finite control: The finite control has some pointer which points the current
symbol which is to be read.
• Stack: The stack is a structure in which we can push and remove the items from
one end only. It has an infinite size. In PDA, the stack is used to store the items
temporarily.

[Note: draw transition table and tuple for this diagram ]

12
 DPDA(Deterministic Pushdown NPDA(Non-deterministic
Automata) Pushdown Automata)

It is less powerful than NPDA. It is more powerful than DPDA.

It is possible to convert every DPDA to It is not possible to convert every

a corresponding NPDA. NPDA to a corresponding DPDA.

The language accepted by DPDA is a The language accepted by NPDA

subset of the language accepted by is not a subset of the language
NDPA. accepted by DPDA.

The language accepted by DPDA is The language accepted by NPDA

called DCFL(Deterministic Context- is called NCFL(Non-deterministic
free Language). Context-free Language).

Q.1 explain the model of Turing machine.

Q. 2 define TM
Q 3 What is Turing machine? Explain the basis model of Turing machine in details.
Q4 Explain the model of Turing machine in brief. Give the formal definition.
Q. Explain with the help of mathematical representation, what is Turing machine?
Turing machine:
Turing Machine was invented by Alan Turing in 1936 and it is used to
accept Recursive Enumerable Languages (generated by Type-0 Grammar).
Turing machines are a fundamental concept in the theory of computation
and play an important role in the field of computer science. They were first
described by the mathematician and computer scientist Alan Turing in 1936
and provide a mathematical model of a simple abstract computer.
A Turing machine is a finite automaton that can read, write, and erase
symbols on an infinitely long tape. The tape is divided into squares, and
each square contains a symbol. The Turing machine can only read one
symbol at a time, and it uses a set of rules (the transition function) to
determine its next action based on the current state and the symbol it is
reading.
The Turing machine begins in the start state and performs the actions
specified by the transition function until it reaches an accept or reject state.

13
 If it reaches an accept state, the computation is considered successful; if it
reaches a reject state, the computation is considered unsuccessful.
A turing machine consists of a tape of infinite length on which read and
writes operation can be performed. The tape consists of infinite cells on
which each cell either contains input symbol or a special symbol called
blank. It also consists of a head pointer which points to cell currently being
read and it can move in both directions.

Figure: Turing Machine

A TM is expressed as a 7-tuple (Q, Σ, Γ, δ, q0, B, F) where:

• Q is a finite set of states

• Σ is the tape alphabet (symbols which can be written on Tape)
• Γ symbol permitted on tape ( ΣUB)
• δ is a transition function which maps (Q × Γ )-→× Q × Γ × {L,R}..
• q0 is the initial state
• F is the set of final states. If any state of F is reached, input string is accepted.
• B is blank symbol (every cell is filled with B except input alphabet initially)
[Give any one example]

14
 Recursive language
Recursively enumerable language

Three states: Two states:

1. Halt and accept 1. Halt and accept
2. Halt and reject 2. Halt and reject
3. Never halt
W
W
TM
TM

accept Reject
accept Reject

Halt Halt
Halt May
/not
Halt

A language ‘w’ is said to be a recursively A language ‘w’ is said to be recursive if

enumerable language if there exists a Turing there exists a Turing machine which will
machine which will accept for all the input accept all the strings in ‘w’ and reject all
strings which are in ‘L’. the string not in ‘w’
But may or may not halt for all input string which Turing machine will halt every time and
are not in ‘w’. give on answer (accepted or rejected) for
each and every string input.

15
 Properties of Recursive and Recursively enumerable language:
1. The complement of recursive language is recursive.
2. The union of two recursive language is recursive.
3. The union of two recursively enumerable language is recursively enumerable.
4. If a language L and its complement L are both recursively enumerable then both (L and L)
are recursive.
5. If L is a recursive language, then Σ *-L is recursive

Properties 1: Complement of a recursive language is recursive.

Or
Show that recursive language are closed under complementation.
1. Let L be the recursive language and TM is a Turing machine that halts on both
case that is in accepting case and rejecting case.
2. M’ is another machine which works on the principle :
i. If M accept the string and enters the final state and halts,
M’ will reject the string and enter the final state and halts.
ii. If M reject the string and enters the final state and halt.
M’ will accept the string and enter the final state and halt.
3. Hence it is clear that complement of recursive language is recursive.

16
Properties 2: The union of two recursive language is recursive.
1. If L1 and L2 are recursive language then there exists Turing machine M1 and M2
accepting L1 and L2 respectively and their halting is guaranteed as shown in fig.

[Note {a,b,c} {a,c,d}

Union set {a,b,c,d}
Intersection {a,c}]

2. Construct a Turing machine M which takes a w string as input and simulates M1 on w

3. If M1 accepts w then M accepts w and halt.
4. But it M1 rejects w then M carries out simulation M2 on w
5. If M2 accepts w then M also accepts w and if M2 rejects w M also rejects w and halt as
shown in fig
6. Therefore, it is clear that Turing machine M accepts L(M)=L(M1) ∪ L(M2) i.e. L1 ∪ L2
7. Hence, we conclude that if L1 and L2 are recursive then L1 ∪ L2 also recursive.

17
Properties 3: The union of two recursively Enumerable language is
recursively enumerable.
1. If L1 and L2 are recursive enumerable language then there exists Turing machine
M1 and M2 accepting L1 and L2 respectively as shown in fig.

2. Construct a Turing machine M which takes a w string as input.

3. M1 and M2 will both will simulate on w.
4. If any of them or both of them accept w , then M accepts w and halt.
5. But w is not accepted by M1 and M2 both, then there is no guarantee whether M1
and M2 will halt or not.as shown in fig
6. Therefore, it is clear that Turing machine M accepts L(M)=L(M1) ∪ L(M2) i.e. L1 ∪
L2
7. Hence, we conclude that if L1 and L2 are recursive enumerable then L1 ∪ L2 also
recursive enumerable.

18
Properties 4: If a language L and its complement L are both
recursively enumerable then L is recursive.

1. If L and L are recursively enumerable language then there exists Turing machine M1
and M2 accepting L and L respectively as shown in fig.
2. We can construct a Turing machine M which takes w as string.
3. M1 simulates M1and M2 on w and halt as shown in fig.
4. If M1 accepts w then M accepts w if M2 accept w then M reject w
5. Thus, M will always say either “yes” or “no” but will never say both “yes” or both “no”.
6. Since L and L are complementary language w will be either in L or in L and hence it
will be either accepted to L or L. Thus halting of M is guaranteed and it is cleared that
M accepts L.
7. We concludes that it is always possible to construct Turing machine accepting L , it
follows that L is recursive.
8. Since L is recursive, L is also recursive.
9. Let L and L ne the pair of complementary language. Then either:
a. Both L and L are recursive
b. Both L and L are recursive enumerable
c. One of L and L is recursively enumerable, but not recursive, the other is not
recursively enumerable.

Undecidable problem:

The problems for which we can’t construct an algorithm that can answer the
problem correctly in the infinite time are termed as Undecidable Problems in
the theory of computation (TOC).

19
 A problem is undecidable if there is no Turing machine that will always halt an
infinite amount of time to answer as ‘yes’ or ‘no’.

Examples

The examples of undecidable problems are explained below. Here, CFG refers
to Context Free Grammar.

• Whether two CFG L and M equal − Since, we cannot determine all the strings of any
CFG, we can predict that two CFG are equal or not.
• Given a context-free language, there is no Turing machine (TM) that will always halt
an infinite amount of time and give an answer to whether language is ambiguous or
not.
• Given two context-free languages, there is no Turing machine that will always halt an
infinite amount of time and give an answer whether two context-free languages are
equal or not.
• Whether CFG will generate all possible strings of the input alphabet (∑*) is
undecidable.

Types of Turing machine:

1. Two-way infinite Tape Turing Machine:

• Infinite tape of two-way infinite tape Turing machine is unbounded

in both directions left and right.
• Two-way infinite tape Turing machine can be simulated by one-
way infinite Turing machine(standard Turing machine).

[Note: write down same tuple as that of Turing machine]

2. Multiple track Turing Machine:

• A k-track Turing machine(for some k>0) has k-tracks and one R/W
head that reads and writes all of them one by one.

20
 • A k-track Turing Machine can be simulated by a single track Turing
machine

A Multi-track Turing machine can be formally described as a 6-

tuple (Q, X, ∑, δ, q0, F) where −

• Q is a finite set of states

• X is the tape alphabet
• ∑ is the input alphabet
• δ is a relation on states and symbols where
δ(Qi, [a1, a2, a3,....]) = (Qj, [b1, b2, b3,....], Left_shift or
Right_shift)
• q0 is the initial state
• F is the set of final states

3. Multi-tape Turing Machine:

• It has multiple tapes and is controlled by a single head.
• The Multi-tape Turing machine is different from k-track Turing
machine but expressive power is the same.
• Multi-tape Turing machine can be simulated by single-tape Turing
machine.

21
A Multi-tape Turing machine can be formally described as a 6-
tuple (Q, X, B, δ, q0, F) where −

• Q is a finite set of states

• X is the tape alphabet
• B is the blank symbol
• δ is a relation on states and symbols where
δ: Q × Xk → Q × (X × {Left_shift, Right_shift, No_shift })k
where there is k number of tapes
• q0 is the initial state
• F is the set of final states

4. Multi-head Turing Machine:

22
23
24
5. Multi-dimensional Tape Turing Machine:

• It has multi-dimensional tape where the head can move in any

direction that is left, right, up or down.
• Multi dimensional tape Turing machine can be simulated by one-
dimensional Turing machine
6. Multi-head Turing Machine:

• A multi-head Turing machine contains two or more heads to read

the symbols on the same tape.
• In one step all the heads sense the scanned symbols and move or
write independently.
• Multi-head Turing machine can be simulated by a single head
Turing machine.
7. Non-deterministic Turing Machine:

A non-deterministic Turing machine is a variant of the simple Turing

machine. For every input at a state, there can be multiple paths/actions
performed by the TM, meaning the transitions are not deterministic.

The difference between a deterministic TM and a non-deterministic TM is the

same as a DFA and an NFA. A multi-tape Turing machine is used to construct
an NTM.

25
 [note: write down same tuple as that of turing machine]

Ackermann function
In computability theory, the Ackermann function, named after Wilhelm
Ackermann, is one of the simplest and earliest-discovered examples of a
total computable function that is not primitive recursive.

The Ackermann function is a two-parameter function that takes non-negative

integers as inputs and produces a non-negative integer as its result.
Ackermann function is defined as:

For example: 1
Compute the value of A(1,2)
A(1,2)= A(0, A(1, 1))
=A(0, A(0, A(1,0)))
=A(0,A(0,A(0,1)))
=A(0,A(0,2))
=A(0,3)
=4

26
Ex.2:
A(2,1)

Ex 3:
A(2,2)

Post Correspondence Problem

The undecidability of the string is determined with the help of Post's Correspondence
Problem (PCP). Let us define the PCP.

"The Post's correspondence problem consists of two lists of string that are of equal
length over the input. The two lists are A = w1, w2, w3, .... , wn and B = x1, x2, x3, .... xn then
there exists a non empty set of integers i1, i2, i3, .... , in such that,
w1, w2, w3, .... wn = x1, x2, x3, .... xn"

To solve the post correspondence problem we try all the combinations of i 1, i2, i3, .... ,
in to find the w1 = x1 then we say that PCP has a solution.

For example 1:
Find whether the lists M=(abb,aa,aaa) and N=(bba,aaa,aa) have a poste
correspondence problem.
Solution:

x1 x2 x3

M abb aa aaa
N bba aaa aa

y1 y2 y3

x2x1x3=aaabbaaa
y2y1y3=aaabbaaa
as we can see , x2x1x3= y2y1y3
x=2; y=1;Z=3

For example 2:
Find whether the lists A=(10, 011,101) and N=(101,11,011) have a post
correspondence problem.
Ans : 2113

For example 3:

27
Fine whether the losts M= (ab,bab,bbaaa)
And N=(a,ba,bab) have a post correspondence solution for x2x1x3 Δ y2y1y3
Ans:
X1 X2 X3
List M ab bab bbaaa

List N a ba bab
Y1 Y2 Y3

x2x1x3=bababbbaaa
y2y1y3=baabab
In this case therer is no solution because x2x1x3 != y2y1y3 the length aren’t
same . so the problem is undecidable.

For example 4:
Prove that PCP with two list x=(0 1 1), y=(102,,10, 12) has no solution.
[ans: home work]
*********************************************************************************
Modified post correspondence problem:
The modified PCP (MPCP) has one additional requirement than PCP First pair in the
solution is the first pair in the list remaining sequence in the solution cab be in any
order.
The MPC problem is to determine whether there exist a sequence of one or more
integer

i1, i2, …, im such that w1wi1 wi2 … wim = x1xi1 xi2 … xim.
Example:
List A (10,11,110)
List B (10,011,11)

X1 X2 X3
List A 10 11 110

List B 10 011 11
Y1 Y2 Y3
X1X3X2 Y1= 10 110 11
Y3Y2=10 11 011
as we can see , X1X3X2=Y1Y3Y2

28
x=1; y=3; z=2

Universal turing machine

29