Formal Language and Automata Theor

y
Prof. Sachin Jain,Prof.Atul kumar,Prof. vaibhavi Patel
Assistant Professor
Computer Science & Engineering
 CHAPTER-2
Regular Language and Finite Automata
Regular Expression
What is Regular Expression?
• An expression which is constructed over alphabet ∑ using the operators * , . ,
+ is called as regular expression.
• For every regular language we can construct an equivalent R.E.

• Regular Operator
▪ The operators * , . , + are called regular operators
* is kleen closure operator
. is concatenation operator
+ is union operator
• Operator precedence

▪ Kleen closure > Concatenation > Union

Types of Regular Expression
• The Regular expression is classified in to thee types

1. Restricted Regular Expression(* , . , +) or standard R.E.

2. Semi Restricted Regular Expression (* , . , + , ∩)
3. Unrestricted (* , . , + , ∩ , ~)
Some identities of Regular Expression
• If r is any regular expression then following identities follows
1. r* is also an regular expression
2. r+ is also an regular expression
3. If r = ф then r* = € and r+ = ф
4. If r = € then r* = € and r+ = €
5. r* + r+ = r*
6. r* . r = r+
7. (r*)* = r*
8. Let r1 and r2 are two R.E. then (r1+r2)* is also a R.E.

1. (r1+r2)* = (r1* + r2*)*

2. (r1+r2)* = (r1 + r2*)*
3. (r1+r2)* = (r1* + r2)* = (r1* . r2*)* = (r2* . r1*)*
Regular Expression for some regular languages
• If r is any regular expression then L(r) is regular language generated byregular
expression r.

Regular Expression Regular Languages

r=ф L = {}
r=€ L = {€}
r=a L = {a}
r = ab L = {ab}
r = (a+b) L = {a,b}
r = (a+b+c) L = {a , b , c}
r = w1+w2+w3…+wn L = {w1,w2,w3,…wn}
Simplification of Regular Expression
• Simplification of Regular expression means simplify the given regular expressi
on using identities if possible.
• Example
1. (0+€)* = (0* + € )*
= (0*)*
= 0*
2. (0* + 0+ )* = (0*)*
= 0*
3. (1* 0)* + 0*1 + 1*(No simplification required)
Regular Expression for some regular languages
1. Construct the regular expression that generates all the string a’s and b’s

I. Including € r=(a+b)*
II. Excluding € r=(a+b)+
III. Where each string start with ab r=ab(a+b)*
IV. Where each string end with ab r=(a+b)* ab
V. Contain substring aba r=(a+b)* aba (a+b)*
VI. Start and end with a r=a (a+b)* a +a
VII. Third symbol from LHS is a r= (a+b) (a+b) (a+b)(a+b)*
Finite Automata
Definition (DFA)
• It is finite state automata which is used to accept or reject a sequence
of string
• In DFA, for each input symbol, there must be only one state. Hence, it is
called Deterministic Automaton. As it has a finite number of states, the m
achine is called Deterministic Finite Machine or Deterministic Finite Auto
maton.

Image source : Google

Formal definition of DFA
• A DFA can be represented by a 5-tuple (Q, ∑, δ, q0, F) where −
Q is a finite set of states.
∑ is a finite set of symbols called the alphabet.
δ is the transition function where δ: Q × ∑ → Q {only one state}
q0 is the initial state from where any input is processed (q0 ∈ Q).
F is a set of final state/states of Q (F ⊆ Q).

Image source : Google

Graphical Representation of a DFA

•A DFA is represented by digraphs called state diagram.

•The vertices represent the states.
•The arcs labeled with an input alphabet show the transitions.
•The initial state is denoted by an empty single incoming arc.
•The final state is indicated by double circles.

Graphical Representation of DFA

Equivalence of DFA and Regular Expression
• If a language is accepted by DFA then L is denoted by a Regular
expression.

• Regular expression and Finite Automata , both have the same co

mputational power.
• For each and every regular expression we can design DFA such tha
t L(R)=L(F).
L(R)= Language accepted by regular expression
L(F)= Language accepted by finite automata
Acceptability by DFA and NDFA
• A string is accepted by a DFA/NDFA iff the DFA/NDFA starting at the initial
state ends in an accepting state (any of the final states) after reading the s
tring wholly.
• A string S is accepted by a DFA/NDFA (Q, ∑, δ, q 0, F), iff
δ*(q0, S) ∈ F

• The language L accepted by DFA/NDFA is

{S | S ∈ ∑* and δ*(q0, S) ∈ F}

• A string S′ is not accepted by a DFA/NDFA (Q, ∑, δ, q 0, F), iff

δ*(q0, S′) ∉ F

• The language L′ not accepted by DFA/NDFA (Complement of accept

ed language L) is
{S | S ∈ ∑* and δ*(q0, S) ∉ F}
Construction of DFA
• Rules for construction of DFA

• Step 1:- Recognize language.

• Step 2:- Find Minimum string using language.
• Step 3:- Create a DFA which accepts or satisfy minimum string.
• Step 4:- Satisfy each input symbol at every state.
• NOTE:- Designed DFA will be minimal DFA
Construction of DFA (Example)
• Construct DFA which starts with “a”, given input symbol {a , b}
• DFA which must Start with “a” but necessarily not with “b” i.e. If string w
ill start from “b” then input symbol “b” will move to dead state or trap st
ate.
• So, L={a,ab,abb,aab….}
• Minimal string = a
Construction of DFA (Example)
• Construct DFA which starts with “a”, given input symbol {a , b}
(Continued)

• Now we will satisfy all state with every input and then we will get
Construction of DFA (Example)
• Construct DFA which ends with “b”, given input symbol {a , b}
• DFA will start with “a” and “b” both but necessarily end with “b” only.
• L={b,ab,bb,aab,abab….}
• Minimal string = b
• After following above all rules we will get below designed DFA
Construction of DFA (Example)
• Construct DFA which starts and ends with same character, give
n input symbol {a , b}

• L={a,b,aa,bb,aba,bab,abba,baab…..}
• Minimal string = {a,b}
• After following above all rules we will get below designed DFA
Construction of DFA (Example)
• Draw a DFA for the language accepting strings starting with ‘ab’ over inp
ut alphabets ∑ = {a, b}

• L={ab, abaab, abababab, abbbbbb…..}

• Minimal string = {ab}
• After following above all rules we will get below designed DFA
Construction of DFA (Example)
• Draw a DFA for the language accepting strings starting with ‘101’ over in
put alphabets ∑ = {0, 1}

• L={101,1010101,1010000,…..}
• Minimal string = {101}
• After following above all rules we will get below designed DFA
Construction of DFA (Example)
• Construct a DFA that accepts a language L over input alphabets ∑ = {a, b}
such that L is the set of all strings starting with ‘aa’ or ‘bb’.

• L={aa, bb, aab , bba , bbb , aaa…..}

• Minimal string = (aa + bb)
• After following above all rules we will get below designed DFA
Non-Deterministic Finite Automata (NFA or
NDFA)
Definition (NFA)
• It is non-deterministic finite state automata which is used to accept or
reject a sequence of string.
• In NFA, for each input symbol, there will be more than one state i.e. co
mbination of states. In other words the exact state to which the machine
moves cannot be determined. Hence, it is called Non Deterministic Auto
maton. As it has a finite number of states, the machine is called Non-Dete
rministic Finite Machine or Non-Deterministic Finite Automaton.

Image source : Google

Formal definition of NDFA or NFA
• A DFA can be represented by a 5-tuple (Q, ∑, δ, q0, F) where −
Q is a finite set of states.
∑ is a finite set of symbols called the alphabet.
δ is the transition function where δ: Q × ∑ → 2Q
(Here the power set of Q (2Q) has been taken because in case of n NDFA,
from a state, transition can occur to any combination of Q states)
q0 is the initial state from where any input is processed (q0 ∈ Q).
F is a set of final state/states of Q (F ⊆ Q).

Image source : Google

Graphical Representation of a NDFA or NFA

•A NDFA is represented by digraphs called state diagram.

•The vertices represent the states.
•The arcs labeled with an input alphabet show the transitions.
•The initial state is denoted by an empty single incoming arc.
•The final state is indicated by double circles.

Image source : Google

Graphical Representation of a NDFA or NFA

•A NDFA is represented by digraphs called state diagram.

•The vertices represent the states.
•The arcs labeled with an input alphabet show the transitions.
•The initial state is denoted by an empty single incoming arc.
•The final state is indicated by double circles.

Image source : Google

Difference between DFA and NFA
DFA NFA
The transition from a state is to a single p The transition from a state can be to mult
articular next state for each input symbol. iple next states for each input symbol. He
Hence it is called deterministic. nce it is called non-deterministic.

Empty string transitions are not seen in D NDFA permits empty string transitions.
FA.
Backtracking is allowed in DFA In NDFA, backtracking is not always possi
ble.
Requires more space Requires less space.
A string is accepted by a DFA, if it transits A string is accepted by a NDFA, if at least
to a final state. one of all possible transitions ends in a fin
al state.
Equivalence of NDFA and Regular Expression
• If a language is accepted by NDFA then L is denoted by a Regular
expression.

• Regular expression and Non-deterministic FA, both have the same com
putational power.
• For each and every regular expression we can design NDFA such that L(R
)=L(NF).
L(R)= Language accepted by regular expression
L(NF)= Language accepted by non-deterministic finite automat
a
Regular Grammar
Grammars

A grammar G can be formally written as a 4-tuple (N, T, S, P)

where −

• N or V is a set of variables or non-terminal symbols.

• T or ∑ is a set of Terminal symbols.
• S is a special variable called the Start symbol, S ∈ N
• P is Production rules for Terminals and Non-terminals. A pr
oduction rule has the form α → β, where α and β are string
s on V ∪ ∑ and least one symbol of α belongs to V
Grammars: example

(({S, A}, {a, b}, S,{S → aAb, aA → aaAb, A → ε } )

– Here,
• S and A are Non-terminal symbols.
• a and b are Terminal symbols.
• ε is an empty string.
• S is the Start symbol, S ∈ N
• Production P : S → aAb, aA → aaAb, A → ε
Derivations from a Grammar
• Strings may be derived using the productions in a grammar.
• For Example
Let us consider the grammar −
G2 = ({S, A}, {a, b}, S, {S → aAb, aA → aaAb, A → ε } )

• Derivation of the string: ‘aaabbb’

S ⇒ aAb using production S → aAb
⇒ aaAbb using production aA → aAb
⇒ aaaAbbb using production aA → aAb
⇒ aaabbb using production A → ε
Grammar: Regular grammars
• The set of all strings that can be derived from a grammar is said to be th
e language generated from that grammar.
• Regular Grammar generates Regular language which is accepted by Finit
e automata.
• Regular Grammar also known as Type 3 grammar.

Image source : Google

Grammar: Regular grammars
Equivalence with finite automata: FA → RLG
• Steps for Converting Finite Automata to Right linear grammar:
– Repeat the process for every state
– Begin the process from start state
– Write the production as the output followed by the state on
which the transition is going
– And at the last add ε because that's is required to end the de
rivation

Note: We have added ε because either you could continue the derivatio
n or would like to stop it. So to stop the derivation we have written
ε
Example: FA → RLG

Steps:
Pick start state a
nd output is on s
ymbol 'a' we are So we will write as :
going on state B
A -> aB
Example: FA → RLG

Steps:
And then we will
pick state B and t so we will get the bel
hen we will go for ow production.
each output.
B -> aB/bB/ε
• B is final state so we have t
o add ε
Example: FA → RLG

Final answer:
So final we got ri
ght linear gramm
ar as: A -> aB
B -> aB/bB/ε
Equivalence with finite automata: FA → LLG

• Steps for Converting Left linear grammar to Finite Automata

:

– Take reverse of the finite automata

– Then write right linear grammar following the previous
lesson
– Then take reverse of the right linear grammar
– And you will get the final left linear grammar
Example: FA → LLG

Step 1:Take reverse of the finite automata

Make final state as init
ial state and vice vers
a

Remove unreacha
ble states
Example: FA → LLG

Step 2:Then write right linear grammar following the previous lesson

we will write dow

n "right linear gra
mmar" for it. B -> aB/bB/aA
A -> ε
Example: FA → LLG
Step 3: Then take reverse of the right linear grammar
reverse the right linear
grammar to get the ou
put = left linear Gram
B -> aB/bB/aA B -> Ba/Bb/Aa
mar
A -> ε A -> ε

Formula
FA -> Reverse of FA -> Right Linear Grammar -> Reverse of Right Linear Gram
ar = Left Linear Grammar
Equivalence with finite automata: RLG → FA

• Conversion from RLG and LLG to FA is very simple.

• Steps for Converting right linear grammar to Finite Automata:

– Start from the first production
– Start State: It will be the first production's state
– And then for every left alphabet go to SYMBOL followed by it
– Final State: Take those states which end up with input alpha
bets.
Example: RLG → FA

• A -> aB/bA/b
• B -> aC/bB
• C -> aA/bC/a
Equivalence with finite automata: LLG → FA
• Steps for Converting left linear grammar to Finite Automata:

– Take reverse of LLG

– Create Finite automata using previous example
– Then again take reverse of the FA and that will be our final output
– Start State: It will be the first production's state
– Final State: Take those states which end up with input alphabets

Formula
Left Linear Grammar -> Reverse of left Linear Grammar-> FA -> Reverse of FA
Example: LLG → FA

Step 1:

left linear grammar Right linear grammar

A -> Ba/Ab/b Take the reverse of the LLG
A -> aB/bA/b
B -> Ca/Bb B -> aC/bB
C -> Aa/Cb C -> aA/bC/a
Example: LLG → FA

Step 2:

Right linear grammar

Now create the FA
A -> aB/bA/b
B -> aC/bB
C -> aA/bC/a
Only state A is final state in thi
s example
Example: LLG → FA

Step 3:
Now take revers
e of the above F
A (this is final out
put)

Here state A was itself a fina

l state that is why it again be
come start state as well as fi
nal state.
Properties of regular languages
Properties of regular languages

• Regular languages have two important kinds of properties:

•Closure properties.
1.Decision properties.
Properties of regular languages: Closure properties
• Closure properties on regular languages are defined as certain operatio
ns on regular language which are guaranteed to produce regular language
.

• Closure refers to some operation on a language, resulting in a new langu

age that is of same “type” as originally operated on i.e., regular.
Properties of regular languages: Closure properties

• Regular languages are closed under following operations.

• Let L and M be regular languages. Then the following languages ar
e all regular:
– Union: L ∪ M
– Intersection: L ∩ M
– Complement: L
– Difference: L - M = L ∩ M
– Concatenation: L . M
– Reversal: LR
– Kleen Closure: L∗
– Positive Closure: L+
Closure properties: Closure under Complementation

• If 𝐿 ⊆ Σ∗ , then the complement of 𝐿, denoted 𝐿, is Σ∗ − 𝐿.

• Theorem: If 𝐿 is a regular language over Σ, then 𝐿 is also a regular langua
ge.

• Proof:
– Construct a DFA for 𝐿
– This can be transformed into a DFA for 𝐿 by making all accepting st
ates non-accepting and vice versa.
Closure properties: Closure under Reversal

• The reversal of 𝐿, written 𝐿𝑅 is {𝑥 | 𝑥𝑅 ∈ 𝐿}

• The reversal of a language L, is the language consisting of the reve
rsals of all its strings
• Example: L = {001, 10, 111}
LR = {100, 01, 111}
• Theorem: If 𝐿 is regular, then so is 𝐿𝑅.
• Proof:
– Start with a DFA for 𝐿.
– Reverse all of the transitions in the DFA
– Make the DFA’s start state the only accepting state.
– Create a new start state with 𝜀-transitions to all of the original accepting s
tates.
Properties of regular languages : Decision properties

• A property is a yes/no question about languages.

• Some examples:
❖Is 𝐿 empty?
❖Is 𝐿 finite?
❖Are 𝐿1 and 𝐿2 equivalent?

• A property is a decision property for regular languages if an

algorithm exists that can answer the question (for regular lan
guages).
Properties of regular languages : Decision properties

• Following are the decision properties for FA:

1.Emptiness
2.Finiteness
3.Equivalence
4.Membership
Decision properties: Emptiness and Non-emptiness

• Does the language contain any string at all?

• Algorithm:
– Select the state that cannot be reached from the initial states & delete th
em (remove unreachable states).
– If the resulting machine contains at least one final states, so then the finit
e automata accepts the non-empty language.
– if the resulting machine is free from final state, then finite automata acce
pts empty language.
Decision properties: Finiteness and Infiniteness

• Is the language accepted by FA is finite or infinite?

• Algorithm:
– Select the state that cannot be reached from the initial state & delete the
m (remove unreachable states).
– Select the state from which we cannot reach the final state & delete them
(remove dead states).
– If the resulting machine contains loops or cycles then the finite automata
accepts infinite language.
– If the resulting machine do not contain loops or cycles then the finite auto
mata accepts finite language.
Decision properties: Equivalence

• We want an algorithm that takes two languages, 𝐿1 and 𝐿2, and de

termines if they are the same?

• Algorithm:
– Convert 𝐿1 and 𝐿2 to DFAs.
– Convert 𝐿1 and 𝐿2 to minimal DFAs. (Refer slide no 40 )
– Determine if the minimal DFAs are the same.
Decision properties: Membership

• Membership is a property to verify an arbitrary string is accepted by

a finite automaton or not? i.e. it is a member of the language or not?

• Let M is a finite automata that accepts some strings over an alphabet

, and let ‘w’ be any string defined over the alphabet,
– if there exist a transition path in M, which starts at initial state & ends in a
nyone of the final state, then string ‘w’ is a member of M, otherwise ‘w’ is
not a member of M.
Pumping lemma for regular languages
Pumping lemma for regular languages
We need a tool to prove that a language is NOT Regular
Important Ideas:
• Pumping:
– Repeating a section of the string an arbitrary number of times (≥0)
, with the resulting string remaining in the language.
• Pumping Length :
– “All string in the language can be pumped if they are at least as lon
g as a certain special value, called the pumping length.”
Pumping lemma for regular languages

Examples:
101 ⇒ 1(01)*
000001 ⇒ (0)*1
110100 ⇒ 1(1)*0100
Pumping lemma for regular languages

If A is a Regular Language, then there is a number p (the pum

ping length) where if s is any string in A of length at least p, th
en s may be divided into 3 pieces, s = xyz, satisfying the follow
ing conditions:

a. For each i ≥ 0, xyiz ∈ A,

b. |y| > 0, and
c. |xy| ≤ p.
Pumping lemma for regular languages

Application:
• The pumping lemma is extremely useful in proving that cert
ain sets are non-regular. The general methodology followed d
uring its applications is :

– Select a string s in the language L.

– Break the string s into x, y and z in accordance with the abov
e conditions imposed by the pumping lemma.
– Now check if there is any contradiction to the pumping lem
ma for any value of i.
Example: Use the pumping lemma to show that following language is not regul
ar: L = {0n1n|n>=1}

Assume L is Regular, then Pumping Lemma must hold.

p is the pumping length given by the PL.

Choose s to be 0p1p.

Because s ∈ L and |s| ≥ p, PL guarantees s can be split into 3 pieces

, s = xyz, where for any i ≥ 0, xyiz ∈ L.
Example: Use the pumping lemma to show that following language is not regul
ar: L = {0n1n|n>=1}
• Consider 3 cases:

1.y is only 0s. xyyz has more 0s than 1s, thus a contradiction via con
dition 1 of PL.
2.y is only 1s. Also a contradiction.
3.y is both 0s and 1s. xyyz may have same number of 1s and 0s, but
will be out of order, with some 1s before 0s, also a contradiction.

• Contradiction is unavoidable, thus L is not Regular.

Exercise

Q1: Use Pumping Lemma to show that following language is n

ot regular. L = { wwR / w ε {0,1}* }

Q2: Prove that the language L = {0n: n is a prime number} is n

ot regular.
Minimization of finite automata
Minimization of finite automata

•DFA minimization stands for converting a given DFA to its eq

uivalent DFA with minimum number of states.

•For ant regular language, more than one DFA can possible bu
t there is only one minimal DFA for that language.
Minimization of finite automata: Equivalence Theorem

• Step 1:
– Eliminate all the inaccessible(unreachable) states from the given DFA (if any).
• Step 2:
– Draw the transition table for all pair of states.
• Step 3:
– We will divide Q (set of states) into two sets. One set will contain all final states and
other set will contain non-final states. This partition is called P0 (0 equivalence)
• Step 4:
– Initialize k = 1
Minimization of finite automata: Equivalence Theorem

• Step 5:
– Find Pk by partitioning the different sets of Pk-1.
– In each set of Pk-1, we will take all possible pair of states.
– If two states of a set are distinguishable, we will split the sets into differen
t sets in Pk.
• Step 6:
– Stop when Pk = Pk-1 (No change in partition)
• Step 7:
– All states of one set are merged into one. No. of states in minimized DFA
will be equal to no. of sets in Pk.
Minimization of finite automata: Equivalence Theorem

• How to find whether two states in partition P k are

distinguishable ?

Two states ( qi, qj ) are distinguishable in partition P k if for an

y input symbol a, δ ( qi, a ) and δ ( qj, a ) are in different sets i
n partition Pk-1.
Equivalence Theorem: Example1

•The given DFA contains no dead states and inaccess

ible states
Equivalence Theorem: Example1

Draw a state transition table.

Equivalence Theorem: Example1

Now using Equivalence Theorem, we have-

P0 = { q0 , q1 , q2 , q3 } { q4 }

P1 = { q0 , q1 , q2 } { q3 } { q4 }

P2 = { q0 , q2 } { q1 } { q3 } { q4 }

P3 = { q0 , q2 } { q1 } { q3 } { q4 }

Since P3 = P2, so we stop.

From P3, we infer that states q0 and q2 are e
quivalent and can be merged together.
Equivalence Theorem: Example1

So, Our minimal DFA is-

Equivalence Theorem: Example2

Minimize following DFA

Equivalence Theorem: Example2

State q3 is inaccessible from the initial state. So, we

eliminate it and its associated edges from the DFA.
Equivalence Theorem: Example2

Draw a state transition tab

le
Equivalence Theorem: Example2

Now using Equivalence Theorem, we have-

P0 = { q0 } { q1 , q2 }
P1 = { q0 } { q1 , q2 }

Since P1 = P0, so we stop.

Equivalence Theorem: Example2

From P1, we infer that states q1 and q2 are equivalent and can be m
erged together.
So, Our minimal DFA is-
Exercise

Minimize the following DFAs

www.paruluniversity.ac.in