Reasoning & Agents: Prof. Dr. Abeer Mahmoud

AIN SHAMS UNIVERSITY- FACULTY OF COMPUTER AND INFORMATION SCIENCES
REASONING & AGENTS

(AIT-421 ‎)
Prof. Dr. Abeer Mahmoud
Professor of Computer Science -Head of computer science Department-Faculty of
Computer and Information Sciences- Ain Shams University
Abeer.mahmoud@cis.asu.edu.eg
(course coordinator)
Computer Science
Department
2 Prof. Dr. Abeer Mahmoud
1 Introduction on Nature of Agents, Performance
Measure and Environment
2 Planning as a search problem: in deterministic,

observable, static and known environment
3 Searching Using constrains
4 Informed Search and Exploration for Agents
5 Propositional Logic & First Order Logic(FOL)
Course 6 Unification and generalized Modus Ponens
Contents 7 Resolution Based Inference
8 Acting in uncertain (byes net, hidden Markov

models,..)
9 Game Theory
10 Rational Decision making
Planning as a
Lec-4 search problem
Search so fare
• Search problem:
• States (configurations of the world)
• Actions and costs
• Successor function (world dynamics)
• Start state and goal test
• Search tree:
• Nodes: represent plans for reaching states
• Plans have costs (sum of action costs)
• Search algorithm:
• Systematically builds a search tree
• Chooses an ordering of the fringe
(unexplored nodes)
• Optimal: finds least-cost plans
Classical Searching algorithm
Uninformed Search Algorithms( Blind Search)
1- Breadth first Search
2- Uniform Cost Search(UCS)
3- Depth First Search
4- Depth limited Search
5- Iterative Deeping Search
6 - Bidirectional Search
Informed Search (Heuristic Search)
Best First Search
Greedy Search
A* Search
6
Informed search
7
Informed search & Exploration
 Modified version from blind search algorithm
1. Greedy best first search

2. A* and its relatives
 The family of local search includes methods

1. inspired by statistical physics [simulated annealing ]
2. evolutionary biology [genetic algorithms]
3. online search [in which agent is faced with state space
that is completely unknown] (course coordinator)
8
Informed search & Exploration
Best first search
Main idea: use an evaluation function f(n) for each node
Implementation:
o Order the nodes in Frontier in decreasing order of
desirability (from low f(n) which means high desirability to
high f(n) which means low desirability. )
o There is a whole family of best-first search strategies,

each with a different evaluation function.
Special cases:
 Greedy best-first search.
 A* search.
9
Best-first search Algorithm
1-greedy best first search
 Tries to expand the node that is closest to the goal
o Use straight line distance ex : hSLD(IN(Arad))=366
[note that the values of fn hSLD cannot be computed

from the problem description itself]
10
Greedy Search Straight line distances between cities
Example 1 which are additionally provided
11
The greedy best first search using hSLD finds a
solution without ever expanding a node that is not on
solution path, hence its cost is minimal
This show why the algorithm is called greedy [at each

step it tries to get as close to goal as it can]
13
Greedy Search Straight line distances between cities
Example 2 which are additionally provided
14
 Consider the problem of getting from Iasi to Fagras
 The heuristic suggests that Neamt be expanded first because it is closes
to Fagaras but it is like dead end
 The solution is to go first to Vaslui a step that is actually farther from the
goal according to the heuristic & then continue to Urzicent, Bucharest and
Fagaras.
 In this case , then heuristic causes unnecessary needs to be expanded
Greedy best first search

 Resembles depth first search in the way it prefers to follow a
single path all the way to goal
15
Best-first search Algorithm
2-A* best first search

 Main idea: avoid expanding paths that are already expensive.
 Minimizing the total estimated solution cost
 It evaluate a node by
F(n) = g(n) + h(n)
Cost so far to reach n

Estimated Cost to get from n to
goal
 Path cost is g and heuristic function is h

– f(state) = g(state) + h(state)
– Choose smallest overall path cost (known + estimate)
16
A* Search
Example
17
A* Search
Example
18
A* Search
Example
19
Properties of A*
• Complete? Yes
• Time? Exponential
• Space? Keeps all nodes in memory
• Optimal? Yes
20
Test yourself
(greedy and A*)
21
Test yourself
(greedy and A*)
4 1
2 B E
1 1
1 2
2 A
D G 0
1
2 1 1 3
3
C F
22
Example 2: Implement the greedy algorithm on the following figure,
then write the its solution path.
4 1
2 B E
1 1
1 2
2 A
D G 0
1
2 1 1 3
3
C F
Solution path: A  C  D  F  G
1 3 1 3
Path cost : 8 Greedy is not suitable  Not optimal
:
Example 2 Implement the A* algorithm on the following figure, then
write the its solution path.
A
F(A)=0+2=2
B C
F(B)=2+4=6 F(C)=1+2=3
D
D
F(D)=4+1=5
F(D)=3+1=4
Solution path:
ABDEG
Path cost : 6 E E F
F
F(E)=5+1=6 F(E)=6+1=7 F(F)=5+1=6
F(F)=3+1+1=5
G G G G
F(G)=6+0=6 F(G)=4+3+0=7 F(G)=7+0=7 F(G)=8+0=8
Beyond classical search
25
LOCAL SEARCH STRATEGY
 Hill-Climbing Search.
 Simulated Annealing Search.
 Local Beam Search.
 Genetic Algorithms.
26
Classical search versus Local Search
Classical search Local Search
Examples of BFS,DFS,UCS,IDFS,LDF Simulated Annealing, Genetic
algorithms S, greedy, A* Algorithm, Local Beam, Hill-
Climbing
Search Order systematic Find best state according to
exploration of search some objective function h(s).
space. e.g., n-queens, h(s)= # of attacking
queens.
Solution The path to the goal the goal state itself is the
is a solution solution
What to keep Keeps one or more Keep a single "current" state,
in memory paths in memory and move to neighboring states
27 in order to try improve it.
Example: n-queens
 Put n queens on an n ×
n board with no two
queens on the same
row, column, or
diagonal.
 In the 8-queens problem, what matters

is the final configuration of queens, not
the order in which they are added.
28
Key idea:
1. Select (random) initial state (generate an initial guess).
2. Make local modification to improve current state

(evaluate current state and move to other states).
3. Repeat Step 2 until goal state found (or out of time).
29
Advantages Drawback:
 Use very little memory –  Local Search can get stuck in
usually a constant local maxima and not find the
amount. optimal solution.
 Can often find reasonable

solutions in large or
infinite state spaces (e.g.,
continuous). For which
systematic search is
unsuitable.
30
State Space Landscape
• A state space landscape: is a graph of states associated
with their costs.
31
Hill-Climbing Search
32
 Main Idea: Keep a single current node and move to a neighboring state to
improve it.
 Uses a loop that continuously moves in the direction of increasing value

(uphill):
 Choose the best successor, choose randomly if there is more than one.
 Terminate when a peak reached where no neighbor has a higher value.
 It also called greedy local search, steepest ascent/descent.
33
34
Hill-Climbing in‎Action‎…‎
Current best solution
cost
States
35
Current solution
cost
Local minima
Global minima States
36
Current
cost solution
Local
minima
Global States
minima
Drawback: Depending on initial state, it can get stuck in
local maxima/minimum or flat local maximum and not find
the solution.
Solution : Random restart.
37
Simulated Annealing Search
38
The Problem
 Most minimization strategies find the nearest local minimum
Standard strategy
 Generate trial point based on current estimates

 Evaluate function at proposed location
 Accept new value if it improves solution
 We need a strategy to find other minima

 This means, we must sometimes select new points that
do not improve solution How?
39
Simulated annealing Search
 Main Idea: escape local maxima by allowing some "bad"

moves but gradually decrease their frequency.
 Instead of picking the best move, it picks a random

move..
40
 say the change in objective function is d

 if d is positive, then move to that state
 otherwise:
• move to this state with probability proportional to d
• thus: worse moves (very large negative d) are executed less
often
41
• Lets say there are 3 moves available, with changes in the

objective function of d1 = -0.1, d2 = 0.5, d3 = -5. (Let T = 1).
• pick a move randomly:
• if d2 is picked, move there. (don’t calculate just accept cause it is
better)
• if d1 or d3 are picked, probability of move = exp(d/T)
• move 1: prob1 = exp(-0.1) = 0.9,
• i.e., 90% of the time we will accept this move
• move 3: prob3 = exp(-5) = 0.0067
• i.e., 0.6% of the time we will accept this move

42
Test yourself
if the Τ=3 , calculate the probability of move to the current state given the
change in objective function in the following table
ΔE Returned Solution
Next Current
d1 = 0.4 ___ _____ _ _____
d2 = - 0.2 ___ _____ ___ ____
d3= -3 __ ______ _______
43
Properties of Simulated Annealing
• Cooling Schedule: determines rate at which the
temperature T is lowered.
44
Local Beam Search
45
Local Beam Search
 Main Idea: Keep track of k states rather than just one.
 Start with k randomly generated states.
 At each iteration, all the successors of all k states are

generated.
 If any one is a goal state, stop; else select the k best

successors from the complete list and repeat.
 Drawback: the k states tend to regroup very quickly in

the same region  lack of diversity.
46
Local Beam Search
Cost
States
47
Local Beam Search
48
Thank you !

Reasoning & Agents: Prof. Dr. Abeer Mahmoud

Uploaded by

Copyright:

Available Formats

Reasoning & Agents: Prof. Dr. Abeer Mahmoud

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Reasoning & Agents: Prof. Dr. Abeer Mahmoud

Uploaded by

Copyright:

Available Formats

AIN SHAMS UNIVERSITY- FACULTY OF COMPUTER AND INFORMATION SCIENCES

REASONING & AGENTS

2 Planning as a search problem: in deterministic,

3 Searching Using constrains

4 Informed Search and Exploration for Agents

5 Propositional Logic & First Order Logic(FOL)

Course 6 Unification and generalized Modus Ponens

Contents 7 Resolution Based Inference

8 Acting in uncertain (byes net, hidden Markov

10 Rational Decision making

1. Greedy best first search

 The family of local search includes methods

o There is a whole family of best-first search strategies,

 Tries to expand the node that is closest to the goal

o Use straight line distance ex : hSLD(IN(Arad))=366

[note that the values of fn hSLD cannot be computed

This show why the algorithm is called greedy [at each

Greedy best first search

2-A* best first search

Cost so far to reach n

 Path cost is g and heuristic function is h

 In the 8-queens problem, what matters

2. Make local modification to improve current state

3. Repeat Step 2 until goal state found (or out of time).

 Can often find reasonable

 Uses a loop that continuously moves in the direction of increasing value

 Terminate when a peak reached where no neighbor has a higher value.

 It also called greedy local search, steepest ascent/descent.

Current best solution

Global minima States

 Generate trial point based on current estimates

 We need a strategy to find other minima

 Main Idea: escape local maxima by allowing some "bad"

 Instead of picking the best move, it picks a random

 say the change in objective function is d

• Lets say there are 3 moves available, with changes in the

• if d1 or d3 are picked, probability of move = exp(d/T)

• move 1: prob1 = exp(-0.1) = 0.9,

• i.e., 90% of the time we will accept this move

• move 3: prob3 = exp(-5) = 0.0067

• i.e., 0.6% of the time we will accept this move

 At each iteration, all the successors of all k states are

 If any one is a goal state, stop; else select the k best

 Drawback: the k states tend to regroup very quickly in

You might also like