Formal Languages & Finite Theory of

Automata
BS Course

Slide # : 03
Muhammad Faizan Tahir
Nondeterministic Finite Automata

Nondeterminism
Subset Construction

2
 Nondeterminism
• A nondeterministic finite automaton has the
ability to be in several states at once.
• Transitions from a state on an input symbol
can be to any set of states.

3
 Nondeterminism – (2)
• Start in one start state.
• Accept if any sequence of choices leads to a
final state.
• Intuitively: the NFA always “guesses right.”

4
 Example: Moves on a Chessboard
• States = squares.
• Inputs = r (move to an adjacent red square)
and b (move to an adjacent black square).
• Start state, final state are in opposite corners.

5
 Example: Chessboard – (2)
r b
1 2 3
1 2,4 5
2 4,6 1,3,5
4 5 6 3 2,6 5
4 2,8 1,5,7
7 8 9 5 2,4,6,8 1,3,7,9
6 2,8 3,5,9
r b b 7 4,8 5
1 2 1 5 8 4,6 5,7,9
4 3 1 * 9 6,8 5
5 3
7 7
9 Accept, since final state reached6
 Formal NFA
• A finite set of states, typically Q.
• An input alphabet, typically Σ.
• A transition function, typically δ.
• A start state in Q, typically q0.
• A set of final states F ⊆ Q.

7
 Transition Function of an NFA
• δ(q, a) is a set of states.
• Extend to strings as follows:
• Basis: δ(q, ε) = {q}
• Induction: δ(q, wa) = the union over all states
p in δ(q, w) of δ(p, a)

8
 Language of an NFA
• A string w is accepted by an NFA if δ(q0, w)
contains at least one final state.
• The language of the NFA is the set of strings it
accepts.

9
 1 2 3
Example: Language of
4 5 6
an NFA 7 8 9
• For our chessboard NFA we saw that rbb is
accepted.
• If the input consists of only b’s, the set of
accessible states alternates between {5} and
{1,3,7,9}, so only even-length, nonempty
strings of b’s are accepted.
• What about strings with at least one r?

10
 Equivalence of DFA’s, NFA’s

• A DFA can be turned into an NFA that

accepts the same language.
• If δD(q, a) = p, let the NFA have δN(q, a) =
{p}.
• Then the NFA is always in a set containing
exactly one state – the state the DFA is in
after reading the same input.

11
 Equivalence – (2)

• Surprisingly, for any NFA there is a DFA that

accepts the same language.
• Proof is the subset construction.
• The number of states of the DFA can be
exponential in the number of states of the
NFA.
• Thus, NFA’s accept exactly the regular
languages.
12
 Subset Construction

• Given an NFA with states Q, inputs Σ,

transition function δN, state state q0, and
final states F, construct equivalent DFA with:
– States 2Q (Set of subsets of Q).
– Inputs Σ.
– Start state {q0}.
– Final states = all those with a member of F.

13
 Critical Point

• The DFA states have names that are sets of

NFA states.
• But as a DFA state, an expression like {p,q}
must be read as a single symbol, not as a
set.
• Analogy: a class of objects whose values
are sets of objects of another class.

14
 Subset Construction – (2)
• The transition function δD is defined by:
δD({q1,…,qk}, a) is the union over all i = 1,…,k of
δN(qi, a).
• Example: We’ll construct the DFA equivalent
of our “chessboard” NFA.

15
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4}
3 2,6 5 {5}
4 2,8 1,5,7
5 2,4,6,8 1,3,7,9
6 2,8 3,5,9
7 4,8 5
8 4,6 5,7,9
* 9 6,8 5 Alert: What we’re doing here is
the lazy form of DFA construction,
where we only construct a state
if we are forced to. 16
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5}
4 2,8 1,5,7 {2,4,6,8}
5 2,4,6,8 1,3,7,9 {1,3,5,7}
6 2,8 3,5,9
7 4,8 5
8 4,6 5,7,9
* 9 6,8 5

17
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5} {2,4,6,8} {1,3,7,9}
4 2,8 1,5,7 {2,4,6,8}
5 2,4,6,8 1,3,7,9 {1,3,5,7}
6 2,8 3,5,9 * {1,3,7,9}
7 4,8 5
8 4,6 5,7,9
* 9 6,8 5

18
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5} {2,4,6,8} {1,3,7,9}
4 2,8 1,5,7 {2,4,6,8} {2,4,6,8} {1,3,5,7,9}
5 2,4,6,8 1,3,7,9 {1,3,5,7}
6 2,8 3,5,9 * {1,3,7,9}
7 4,8 5 * {1,3,5,7,9}
8 4,6 5,7,9
* 9 6,8 5

19
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5} {2,4,6,8} {1,3,7,9}
4 2,8 1,5,7 {2,4,6,8} {2,4,6,8} {1,3,5,7,9}
5 2,4,6,8 1,3,7,9 {1,3,5,7} {2,4,6,8} {1,3,5,7,9}
6 2,8 3,5,9 * {1,3,7,9}
7 4,8 5 * {1,3,5,7,9}
8 4,6 5,7,9
* 9 6,8 5

20
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5} {2,4,6,8} {1,3,7,9}
4 2,8 1,5,7 {2,4,6,8} {2,4,6,8} {1,3,5,7,9}
5 2,4,6,8 1,3,7,9 {1,3,5,7} {2,4,6,8} {1,3,5,7,9}
6 2,8 3,5,9 * {1,3,7,9} {2,4,6,8} {5}
7 4,8 5 * {1,3,5,7,9}
8 4,6 5,7,9
* 9 6,8 5

21
 Example: Subset Construction
r b r b
1 2,4 5 {1} {2,4} {5}
2 4,6 1,3,5 {2,4} {2,4,6,8} {1,3,5,7}
3 2,6 5 {5} {2,4,6,8} {1,3,7,9}
4 2,8 1,5,7 {2,4,6,8} {2,4,6,8} {1,3,5,7,9}
5 2,4,6,8 1,3,7,9 {1,3,5,7} {2,4,6,8} {1,3,5,7,9}
6 2,8 3,5,9 * {1,3,7,9} {2,4,6,8} {5}
7 4,8 5 * {1,3,5,7,9} {2,4,6,8} {1,3,5,7,9}
8 4,6 5,7,9
* 9 6,8 5

22
 Proof of Equivalence: Subset
Construction
• The proof is almost a pun.
• Show by induction on |w| that
δN(q0, w) = δD({q0}, w)
• Basis: w = ε: δN(q0, ε) = δD({q0}, ε) = {q0}.

23
 NFA’s With ε-Transitions
• An NFA with null transition is allowed to make
transition not only on input from the alphabet but
also with null input, i.e. without any input symbol.
This transition without input is called null transition.
• An NFA with null transition is also denoted by a 5-
tuple
M = ( Q, Σ, δ, F, q0)
Where Q, Σ, q0, F have their usual meaning and
transition δ defines a mapping from Q x (Σ U {є})->2Q

24
 NFA’s With ε-Transitions
• We can allow state-to-state transitions on ε
input.
• These transitions are done spontaneously,
without looking at the input string.
• A convenience at times, but still only regular
languages are accepted.

25
 ε-NFA Example
Key idea:
 NFA pattern for each symbol and each operator.
 Join them with e-moves in precedence order.
a
s0 s1
NFA for a

a  b
s0 s1 s3 s4
NFA for ab 26
 ε-NFA Example

a
 s1 s2 
s0 s5
 b
s3 s4 

NFA for a | b
27
 ε-NFA Example


 s1
a 
s0 s2 s4


NFA for a*

28
 ε-NFA Example

b
   s4 s5 
a 
s0 s 1 s2 s3 s8 s9
 s c
6 s7 


NFA for a ( b|c )*
29
ε-NFA Example

30
 ε-NFA Example

ε 0 1 ε
A {E} {B} ∅
1 1 B ∅ {C} {D}
1 B C D
C ∅ {D} ∅
A ε ε 0 * D ∅ ∅ ∅
E {F} ∅ {B, C}
0 E F
0 F {D} ∅ ∅

31
 Closure of States

• CL(q) = set of states you can reach from

state q following only arcs labeled ε.
• Example: CL(A) = {A}; ε
B 1 C 1 D
CL(E) = {B, C, D, E}. 1
A ε ε 0
0 E F
0

• Closure of a set of states = union of the

closure of each state.
32
 Extended Delta

• Basis: δ˄ (q, ε) = CL(q).

• Induction: δ˄(q, xa) is computed as follows:
1. Start with ˄δ(q, x) = S.
2. Take the union of CL(δ(p, a)) for all p in S.
• Intuition: δ˄(q, w) is the set of states you can
reach from q following a path labeled w.

And notice that δ(q, a) is not

33
that set of states, for symbol a.
 ε
Example: Extended 1 B 1 C 1 D

Delta A ε ε 0
0 E F
˄ 0
• δ(A, ε) = CL(A) = {A}.
• ˄
δ(A, 0) = CL({E}) = {B, C, D, E}.
• ˄ (A, 01) = CL({C, D}) = {C, D}.
δ
• Language of an ε-NFA is the set of strings
˄
w such that δ(q0, w) contains a final state.

34
 ε-NFA  DFA Construction
• The algorithm is called subset construction.
• In the transition table of an NFA, each entry is a set of states.
• In DFA, each entry is a single state.
• The general idea behind NFA-to-DFA construction is that each
DFA state corresponds to a set of NFA states.
• The DFA uses its state to keep track of all possible states the
NFA can be in after reading each input symbol.
• We will use the following operations.
• -closure(T):
set of NFA states reachable from some NFA state s in T on -
transitions alone.

35
 ε-NFA  DFA Construction
• move(T,a):
set of NFA states to which there is a transition on input a
from some NFA state s in set of states T.
• Before it sees the first input symbol, NFA can be in
any of the state in the set -closure(s0), where s0 is the start
state of the NFA.
• Suppose that exactly the states in set T are reachable from
s0 on a given sequence of input symbols.
• Let a be the next input symbol.
• On seeing a, the NFA can move to any of the states in the
set move(T,a).

36
 ε-NFA  DFA Construction
• When we allow for
-transitions, NFA can be in any of the states in
-closure(move(T,a))
after seeing a.

37
 Subset Construction
Algorithm:
Input:
NFA N with state set S, alphabet , start state s0, final states F
Output:
DFA D with state set S’, alphabet , start states
s0’ = -closure(s0), final states F’, transition table: S’ x  → S’

// initially, e-closure(s0) is the only state in D states S’ and it is

unmarked
s0’ = -closure(s0)
S’ = {s0’ } (unmarked)

38
 Subset Construction
while there is some unmarked state T
in S’
mark state T
for all a in  do
U = -closure( move(T,a) );
if U not already in S’
add U as an unmarked state to S’
Dtran(T,a) = U;
end for
end while

F’:
for each DFA state S
if S contains an NFA final state
mark S as DFA final state
39
 Subset Construction

a
2 3
 
  a b b
0 1 6 7 8 9 10
 
b
4 5

NFA for (a | b )*abb 40
 Subset Construction
The start state of equivalent DFA is -closure(0), which is
A = {0,1,2,4,7}

A = {0,1,2,4,7}, these are exactly the states reachable from state 0

via -transition.

The input symbol alphabet here is {a,b}.

The algorithm tells us to mark A and then compute

-closure(move(A,a))

move(A,a)), is the set of states of NFA that have transition on ‘a’

from members of A.
41
 Subset Construction
Only 2 and 7 have such transition, to 3 and 8.

So, -closure(move(A,a)) = -closure({3,8}) = {1,2,3,4,6,7,8}

•Let B = {1,2,3,4,6,7,8}.
•Thus Dtran[A,a] = B

For input b, among states in A, only 4 has transition on b to 5

•C = -closure({5}) = {1,2,4,5,6,7} Thus, Dtran[A,b] = C

42
 Subset Construction
We continue this process with the unmarked sets B and C

i.e., -closure(move(B,a)),
-closure(move(B,b)),

Similarly,
-closure(move(C,a))
-closure(move(C,b))

Until all sets and states of DFA are marked.

43
 Subset Construction
Eventually, the 5 sets are:
A={0,1,2,4,7}
B={1,2,3,4,6,7,8}
C={1,2,4,5,6,7}
D={1,2,4,5,6,7,9}
E={1,2,4,5,6,7,10}

A is start state
A={0,1,2,4,7} D={1,2,4,5,6,7,9}
B={1,2,3,4,6,7,8} E={1,2,4,5,6,7,10}
C={1,2,4,5,6,7} 44
 Subset Construction
E is accepting state
A={0,1,2,4,7} D={1,2,4,5,6,7,9}
B={1,2,3,4,6,7,8} E={1,2,4,5,6,7,10}
C={1,2,4,5,6,7}

45
 Subset Construction

a a
a b b
A B D E
a
b a
b
C
DFA for (a | b )*abb
b 46
 Subset Construction
Input symbol
State a b

A B C
B B D
C B C
D B E
47
 Equivalence of NFA, ε-NFA
• Every NFA is an ε-NFA.
– It just has no transitions on ε.
• Converse requires us to take an ε-NFA and
construct an NFA that accepts the same
language.
• We do so by combining ε–transitions with the
next transition on a real input.

Warning: This treatment is a

bit different from that in the text. 48
Picture of ε-Transition Removal

a
Transitions
on ε
Transitions
on ε
49
Picture of ε-Transition Removal
Text goes To here, and performs
from here the subset construction

a
Transitions
on ε
Transitions
on ε
50
Picture of ε-Transition Removal
To here, with no
subset construction

a
We’ll go
from here a

a
Transitions
on ε
Transitions
on ε
51
 Equivalence – (2)
• Start with an ε-NFA with states Q, inputs Σ,
start state q0, final states F, and transition
function δE.
• Construct an “ordinary” NFA with states Q,
inputs Σ, start state q0, final states F’, and
transition function δN.

52
 Equivalence – (3)

• Compute δN(q, a) as follows:

1. Let S = CL(q).
2. δN(q, a) is the union over all p in S of δE(p, a).
• F’ = the set of states q such that CL(q)
contains a state of F.
• Intuition: δN incorporates ε–transitions
before using a but not after.

53
 Equivalence – (4)
• Prove by induction on |w| that

CL(δN(q0, w)) = E(q0, w).

˄
δ
• Thus, the ε-NFA accepts w if and only if the
“ordinary” NFA does.

54
 Interesting
closures: CL(B)
Example: ε-NFA-to-
= {B,D}; CL(E) NFA
= {B,C,D,E}
0 1 ε 0 1
A {E} {B} ∅ A {E} {B}
B ∅ {C} {D} * B ∅ {C}
C ∅ {D} ∅ C ∅ {D}
* D ∅ ∅ ∅ * D ∅ ∅
E {F} ∅ {B, C} * E {F} {C, D}
F {D} ∅ ∅ F {D} ∅
Since closure of
ε-NFA E includes B and
Since closures of C; which have
B and E include transitions on 1
final state D. to C and D. 55
 Summary
• DFA’s, NFA’s, and ε–NFA’s all accept exactly
the same set of languages: the regular
languages.
• The NFA types are easier to design and may
have exponentially fewer states than a DFA.
• But only a DFA can be implemented!

56