Module – 2 Artificial Intelligence & Machine Learning (18CS71)

MODULE – 2

1) Knowledge Representation Issues

[1.1] Representations & Mappings

➢ To solve a complex problem in Artificial Intelligence, we need to have both “Knowledge”

and “some mechanism to manipulate that knowledge in order to create solutions”.

➢ There are a variety of ways in which we can represent the Knowledge in Artificial
Intelligence.

➢ The two entities that we will be dealing here in AI are as follows;

a)FACTS: the things that we want to represent

b)REPRESENTATION of Facts

➢ These entities can be structured into two levels;

a)The Knowledge Level: here the facts are described

b)The Symbol Level: representation of the objects at the knowledge level are defined in
terms of symbols.

➢ There will be a two-way mapping done between FACTS and REPRESENTATIONS, and
we call them as “Representation Mappings”.

➢ The forward representation mappings will map from Facts -> Representations. Whereas,
the backward representation mappings will be the other way.

➢ Mappings between Facts & Representation can be shown as follows;

Department of CSE Page 2

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

➢ One representation of Facts is so common that it deserves special mention: i.e the natural
language sentences ( particularly English sentences).

➢ Let us look into an example that uses Mathematical Logic as the representational
formalism as follows;

a) Consider the English Sentence: Spot is a dog.

b) The fact represented above in English can be represented using Logic as follows;

dog ( Spot )

c) Suppose that we also have a logical representation of the Fact that “all dogs have tails”,
this we can represent it as shown below;

In Logic, we use the symbol

d) Then, using the deductive mechanism of logic, we may generate the new representation
object as below;

hastail ( Spot )

e) Them using an appropriate backward mapping function, we could then generate

another English sentence as below;

Spot has a tail.

[1.2] Approaches to Knowledge Representation

➢ A good system that we use to represent the knowledge in a particular domain should
possess the following four properties;

i) Representational Adequacy:- the ability to represent all of the kinds of knowledge that
are needed in that domain.
ii) Inferential Adequacy:- the ability to manipulate the representational structures in such
a way as to derive new structures corresponding to new knowledge inferred from
old.

Department of CSE Page 3

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

iii) Inferential Efficiency:- the ability to incorporate into the Knowledge structure additional
information that can be used to focus the attention of the inference mechanisms in the
most promising directions.
iv) Acquisitional Efficiency:- the ability to acquire new information easily.

➢ There is no single system that optimizes all of the capabilities to represent the Knowledge.
➢ Multiple techniques exist for representing Knowledge.
➢ Some of the examples of Knowledge representation techniques are as follows;
i) Simple Relational Knowledge

✓ The simplest way to represent declarative facts is, as a set of relations of the same
sort used in the database systems. Example shown below;

ii) Inheritable Knowledge

✓ The relational knowledge was able to describe the objects of the Knowledge Base.
✓ It is possible to augment the basic representation with inference mechanisms that
operate on the structure of the representation.
✓ By using the “Property Inheritance” method, we can have a group of classes, and
its objects associated to it.
iii) Inferential Knowledge
✓ Property Inheritance is not the only form of inference.
✓ Sometimes all the power of traditional logic is necessary to describe the inferences
that are needed.
✓ We can make use of the first-order predicate logic to represent additional
knowledge, which uses certain symbols as per the predicate logic.
iv) Procedural Knowledge
✓ Procedural knowledge specifies What to Do , When.

Department of CSE Page 4

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

✓ The most common way of presenting procedural knowledge is simply as a code

which uses certain programming languages like LISP.

[1.3] Issues in Knowledge Representation

i) Important Attributes

✓ There are two attributes that are of very general significance i.e instance & isa.
✓ Isa represents the class of an object, whereas instance represents the thing that
belongs to a class of a particular object.
✓ These attributes are important because they support Property Inheritance.
✓ The name doesn‟t matter, but it does matters about the classes and its inclusion.
ii) Relationship among attributes

✓ The attributes that we use to describe objects are themselves entities that we
represent.
✓ There are four properties that is independent of the specific knowledge and they
are;
❖ Inverses

▪ Entities in the world are related to each other in many different ways.
▪ In many cases, it is important to represent other views of
relationships.
▪ We can use attributes that focus on a single entity, but to use them in
pairs, one the inverse of the other.
▪ Example: team(Pee-Wee-Reese, Brooklyn-Dodgers) can represent
the team information in two ways;
One associated with Pee-Wee-Reese -> team = Brooklyn-Dodgers
One associated with Brooklyn-Dodgers-> team = Pee-Wee-Reese
❖ Existence in an isa hierarchy
▪ Just as there are Classes of Objects, specialized subsets of those
classes; there are Attributes and Specializations of Attributes.
▪ Example: the attribute “height” is actually a specialization of the more
general attribute “Physical-Size”.

Department of CSE Page 5

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

❖ Techniques for reasoning about values

▪ Sometimes values of attributes are specified explicitly when a
knowledge base is created.
▪ But the reasoning system must be prepared enough to reason about
the values that are not given explicitly, such as;

a)Information about the type of the value

b)Constraints on the value
c) Rules for computing the value when its needed
d) Rules that describe actions that should be taken if a value ever
becomes known.
❖ Single-valued attributes

▪ A specific but very useful kind of attribute is one that is guaranteed to

take a unique value.
▪ Example: A baseball player can, at any one time , have only a single
height and be a member of only one team.

iii) Choosing the Granularity of Representation

✓ Though we have various representation formalism, it is necessary to know “what

level of detail should the world is represented”.

✓ For this, we should know the following;

• What should be our primitives?
• Should there be small number of low-level ones?
• Should there be a larger number covering a range of granularities?

✓ There are 3 issues related to the use of low-level primitives.

Problem 1: High-level facts may require a lot of storage & much of that storage is
really wasted.

Problem 2: If the knowledge is initially presented to the system in a relatively high-

level form (such as English), then substantial work must be done to reduce the
knowledge into primitive form.

Problem 3: In many domains, it is not at all clear what the primitives should be.

Example: set of primitives: Mother, Father, Son, Daughter, Brother, Sister

If we say “Mary is Sue‟s Cousin”, then an attempt to describe the cousin

relationship in terms of the primitives could produce any of the following
interpretations;

Department of CSE Page 6

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Mary = daughter(brother(mother(Sue)))
Mary = daughter(sister(mother(Sue)))
Mary = daughter( brother(father(Sue)))
Mary=daughter(sister(father(Sue)))

If we don‟t know whether Mary is female, then we have another 4 set of primitives
with son.

Therefore the way to solve this problem is by changing set of primitives as;
Parent, Child, Sibling, Male, Female

Then the fact “Mary is Sue‟s cousin” could be represented as;

Mary = Child(Sibling(Parent(Sue)))

iv) Representing sets of objects

✓ It is important to be able to represent sets of objects as there are some properties

that are true of sets that are not true of the individual members of a set. i.e there
are sets that will be true as a Group (or) sets, but not true as an individual member
of a set.

Example: “There are many sheep than people in Australia”

And

“English speakers can be found all over the world”

Therefore, No single English speaker can be found all over the world.

v) Finding the right structures as Needed

✓ In order to have access to the right structure for describing a particular situation, it
is necessary to solve all of the following problems;

i) How to select an initial structure?

ii) How to fill in appropriate details from the current situation?

iii) How to find a better structure if the one chosen initially turns out to be not
appropriate?

iv) What to do if none of the available structures is appropriate?

v) When to create & remember a new structure?

Department of CSE Page 7

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

2) Predicate Logic

[2.1] Representing Simple facts in Logic

✓ This is appealing because it is simple to deal with & a decision procedure for it exists.

✓ The standard symbols used to represent facts are as follows;

✓ A real world fact can be represented as logical propositions.

✓ From a set of propositions, we can even conclude.

Example:
It is raining. RAINING
It is sunny SUNNY
It is windy. WINDY

If it is raining, then it is not sunny.

✓ In predicate logic, we can represent real world facts as statements written as wff’s(Well Formed
Formula)

Example: Consider some set of sentences given below. The facts described by these sentences can
be represented as a set of wff’s in predicate logic as follows;

1) Marcus was a man.

Man(Marcus)

- This representation captures the fact that Marcus is a man.

- But it fails to capture some of the information in English sentence, i.e the notion of past
tense.

Department of CSE Page 8

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

2) Marcus was a Pompeian.

Pompeian(Marcus)

3) All pompeians were Romans.

4) Caesar was a ruler.

Ruler(Caesar)

5) All romans were either loyal to caesar or hated him.

6) Everyone is loyal to someone.

7) People only try to assassinate rulers they are not loyal to.

8) Marcus tried to assassinate Caesar.

tryassassinate( Marcus, Caesar)

[2.2] Representing INSTANCE and ISA relationships

✓ Apart from normal predicate logic, we can also use INSTANCE and ISA predicates too.

✓ The predicate INSTANCE is a binary, where first argument is an object; second argument is a class
to which the object belongs.

Example:

Department of CSE Page 9

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

[2.3] Resolution

✓ Resolution is a procedure, which gains its efficiency from the fact that it operates on statements
that have been converted to a very convenient standard form.
✓ Resolution is used if various statements are given and need to prove a conclusion of those
statements.
✓ Resolution is a single inference rule which can efficiently operate on Conjunctive Normal Form.

Conversion to Clausal form

✓ Propositional resolution works only on expressions in clausal form.

✓ Fortunately, it is possible to convert any set of propositional calculus sentences into an
equivalent set of sentences in clausal form.
✓ A literal is either an atomic sentence (or) a negation of an atomic sentence.

,
✓ A clausal sentence is either a literal or a disjunction of literals.

, ,PV Q
✓ A clause is a set of literals.

✓ Empty clause { } is unsatisfiable.

Department of CSE Page 10

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Steps to convert FOL into CNF

1. Eliminate implication
2. Standardize variable
3. Move negation inwards
4. Skolemization
5. Drop universal quantifier

Example:
1) Eliminate implication

2) Standardize variable
- i.e instead of using same variable in all sentences, use different variables for each.

3) Move negation inwards

4) Skolemization
- i.e to remove existential quantifier and replace it by Skolem constant.

After Skolemization
Smile(A)
Graduating(B)

Department of CSE Page 11

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

5) Drop universal quantifier

-i.e to remove universal quantifier

After Dropping
Smile(x)
Graduating(y)

Basis of Resolution

✓ The resolution procedure is a simple iterative process: at each step two clauses , called the Parent
classes are compared and will yield a new Clause that has been inferred from them.
✓ The new clause represents ways that the two Parent clauses interact with each other.

Example: We have two clauses

Winter V Summer
¬Winter V Cold

- Both clauses must be True

- If Winter is True, then Cold must be True, to guarantee the truth of Clause-2
- If ¬Winter is True, then Summer must be True, to guarantee truth of Clause-1.
- Thus, from these two clauses we can deduce the following;
Summer V Cold

Resolution in Propositional Logic

Given Axioms Convert to Clause Form

P P
(P^Q) -> R ¬(P^Q)v R
= ¬P v ¬Q v ¬R
(SvT) -> Q (¬S ^ ¬T) v Q
=(¬SvQ) ^ (¬TvQ)
T T

To Prove: R

Step 1: Convert the given axioms into clause form, in such a way that it should be in disjunction form and
not in conjunction.

Example: A v B - is correct
A ^ B - is not correct, so here we need to make A,B as two separate clauses

Department of CSE Page 12

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Step 2: To prove the statement “R” , we need to prove its negation part. i.e to prove ¬R

Resolution in Predicate Logic

Suppose we have the axioms already in the clausal form as below;

Department of CSE Page 13

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Department of CSE Page 14

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

[2.4] Representing Knowledge using Rules

✓ It is about the use of rules to encode knowledge.

✓ A procedural representation is one in which, the control information that is necessary to use the
knowledge is embedded in the knowledge itself.

✓ A declarative representation is one in which knowledge is specified, but how to use that
knowledge is not given.

✓ Logic programming is a programming language paradigm in which logical assertions are viewed as
programs. Example: PROLOG

✓ A PROLOG program is described as a series of logical assertions.

✓ Forward Vs Backward Reasoning

- A search procedure is to discover a path through a problem space from a initial

configuration to a goal state.
- There are two directions in which such a search could proceed and they are;
i) Forward – from the start state
ii) Backward – from the goal state
- To reason forward, the left sides(i.e preconditions) are matched against the current state &
the right sides(results) are used to generate new nodes until goal is reached.
- To reason backward, the right sides(results) are matched against the current node & the
left sides (preconditions) are used to generate new nodes representing new goal states to
be achieved.

✓ Backward chaining rule systems

- They are good for goal-directed problem solving.

- Starts with goals & works backwards to determine what facts must be asserted , so that
goals can be achieved.

✓ Forward chaining rule systems

- Instead of being directed by goals, forward-chaining will be directed by incoming data.

- Starts with the known facts & asserts new facts.

✓ Matching

- If we compare rules by matching the current state & the pre-conditions of the rules.
Department of CSE Page 15
 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

- Indexing method is used to find the match by considering the current state as an index into
the rules & select as an index into the rules and select the matching one’s exactly.

3) Concept Learning

➢ Concept learning is a process of searching for the correct hypothesis that can be decided
from a large set of hypothesis.

Example:

Tablets Gadgets

Smart Phones

Each features of these gadgets are known as Binary Valued attribute.

If we can write the features of both Tablets and Smartphones as follows;

SIZE: Large, Small => x1

COLOR: Black, Blue => x2
SCREENTYPE: Flat, Folded => x3
SHAPE: Square, Rectangle => x4

Then we can write the concept here as;

Concept = < x1, x2, x3, x4 >

The hypothesis for a Tablet can be written as;

Tablet = < Large, Black, Flat, Square >

And, for the smartphone we can represent it as;

Smartphone = < Small, Blue, Folded, Rectangle >

The number of possible instances = 2d = 24 = 16

where d is the number of features

Total possible concepts = 2 (numberofpossibleinstances) = 216

Department of CSE Page 16

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Concept Space

Target Space/ Hypothesis space

Specific Hypothesis can be represented using „phi‟ symbol as follows;

< Φ, Φ, Φ, Φ > which is used to represent negative example which simply “Rejects All”

General hypothesis can be represented using „?‟ symbol as follows;

< ?, ?, ?, ? > which is used to represent positive example which “Accept All”

Main Goal: is to find all concept/hypothesis that are consistent.

[3.1] Concept Learning Task

Example: concept: Days on which Aldo enjoys his favorite water sport.

➢ The table above describes a set of example days, each represented by a set of attributes.

➢ The attribute “EnjoySport” indicates whether Aldo enjoys his favorite water sport on this
day.

➢ The task is to learn to predict the value of “EnjoySport” for an arbitrary day based on the
values of it‟s other attributes.

➢ Let each hypothesis be a vector of 6 constraints specifying the values of the 6 attributes
Sky, AirTemp, Humidity, Wind, Water, Forecast

Department of CSE Page 17

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

➢ For each attribute, the hypothesis will either;

 Indicate by a „?‟ that any value is acceptable for this attribute

 Specify a single required value (ex: warm) for the attribute
 Indicate by a „Φ‟ that no value is acceptable.

➢ To illustrate the hypothesis that Aldo enjoys his favorite sport only on Cold days with High
humidity is represented as;

( ? , Cold, High, ?, ?, ? )

[3.2] Find-S: Finding a maximally Specific Hypothesis

Algorithm

1. Initialize h to the most specific hypothesis in H

2. For each positive training instance x
For each attribute constraint a, in h
If the constraint a, is satisfied by x
Then do nothing
Else replace a, in h by the next more general constraint that is satisfied by x
3. Output hypothesis h

Steps to be performed in FIND-S

1) The first step of Find-S is to initialize „h‟ to the most Specific hypothesis in „H‟

h = { Φ,Φ,Φ,Φ,Φ,Φ }

2) The first example in table is a positive example, it becomes clear that our hypothesis is too
specific. Therefore, none of the „Φ‟ constraints in „h‟ are satisfied by this example, so each
is replaced by next more general constraint that fits the example such as;

h = { Sunny, Warm, Normal, Strong, Warm, Same }

3) Next example is considered, where that is also positive example. Therefore, we need to
further generalize „h‟ by substituting a „?‟ in place of any attribute value in „h‟ is not satisfied
by new example.
h = { Sunny, Warm, ?, Strong, Warm, Same }

4) Consider third example, which is a negative example & the algorithm makes o change to
„h‟. In fact, the Find-S algorithm simply ignores every negative example.

5) To complete our trace of Find-S, the 4th positive example leads to further generalization of
„h‟ as follows;

Department of CSE Page 18

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

h = { Sunny, Warm, ?, Strong, ?, ? }

6) The search moves from Most Specific to More General hypothesis.

[3.3] Candidate Elimination Algorithm

The candidate elimination algorithm incrementally builds the version space given a
hypothesis space H and a set E of examples. The examples are added one by one;
each example possibly shrinks the version space by removing the hypotheses that are
inconsistent with the example. The candidate elimination algorithm does this by
updating the general and specific boundary for each new example.
• You can consider this as an extended form of Find-S algorithm.
• Consider both positive and negative examples.
• Actually, positive examples are used here as Find-S algorithm (Basically they are
generalizing from the specification).
• While the negative example is specified from generalize form.

S = { ɸ, ɸ,ɸ,….ɸ }
G = { ?, ?, ?, …? }

Algorithm

Step1: Load Data set

Step2: Initialize General Hypothesis and Specific Hypothesis.
Step3: For each training example
Step4: If example is positive example
If attribute_value == hypothesis_value:
Do nothing
else:
replace attribute value with '?'(Basically generalizing it)
Step5: If example is Negative example
Make generalize hypothesis more specific.

Steps to be performed in Candidate Elimination Algorithm

For the data set given below;

SKY AIR TEMP HUMIDITY WIND WATER FORECAST ENJOYSPORT

Sunny Warm Normal Strong Warm Same Yes
Sunny Warm High Strong Warm Same Yes
Rainy Cold High Strong Warm Change No
Sunny Warm High Strong Cool Change Yes

Department of CSE Page 19

 Module – 2 Artificial Intelligence & Machine Learning (18CS71)

Initially : G =[[?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?],

[?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?]]
S = [Null, Null, Null, Null, Null, Null]

For instance 1 : <'sunny','warm','normal','strong','warm ','same'> and

positive output.
G1 = G
S1 = ['sunny','warm','normal','strong','warm ','same']

For instance 2 :<'sunny','warm','high','strong','warm ','same'> and positive

output.
G2 = G
S2 = ['sunny','warm',?,'strong','warm ','same']

For instance 3 :<'rainy','cold','high','strong','warm ','change'> and

negative output.
G3 = [['sunny', ?, ?, ?, ?, ?], [?, 'warm', ?, ?, ?, ?], [?, ?,
?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, ?], [?, ?, ?, ?, ?, 'same']]
S3 = S2

For instance 4 :<'sunny','warm','high','strong','cool','change'> and positive

output.
G4 = G3
S4 = ['sunny','warm',?,'strong', ?, ?]

At last, by synchronizing the G4 and S4 algorithm produce the output.

Output :

G = [['sunny', ?, ?, ?, ?, ?], [?, 'warm', ?, ?, ?, ?]]

S = ['sunny','warm',?,'strong', ?, ?]

Department of CSE Page 20