Agents and Environments

Lecture 2
APT 3010: Introduction to AI
Prof. Audrey Mbogho
Fall 2020
USIU-Africa
 What is Artificial Intelligence?
●
Should a successful AI be an artificial human being?
●
Does it need to have emotions in order to be human?
●
Does it need to have a human form?
●
Is Intelligence to do with thinking clever thoughts or is it to do
with doing clever things?
●
Does successful AI need to think in the way we think?
●
When does a computer program turn into “an AI”?

Notes are taken from the course text (Russell & Norvig)
Chapters 1 & 2, and from John Levin, University of
Strathclyde.

2
 Four Perspectives of AI
●
Four different perspectives emerged among
researchers as to what artificial intelligence entails:
– AI systems should be designed to behave in exactly in
the same way that humans behave
– AI systems should seek to model human thought
processes precisely and solve problems in the same
way we do
– AI systems should be those which can make exact
logical deductions based on their knowledge of the world
– AI systems should make rational decisions about how to
act in the world to gain the best possible reward

3
 Acting Humanly – The Turing Test
●
If a computer can imitate a human being well
enough to fool an observer, then it must be
intelligent
●
Not the easiest thing to do – we’re nowhere near
at the moment
– But see https://www.theverge.com/21346343/gpt-3-
explainer-openai-examples-errors-agi-potential
●
Is the goal of artificial flight to make “machines
that fly so exactly like pigeons that they even fool
other pigeons”?

4
 Thinking Humanly – Cognitive
Science
●
Do experiments to find out how the human mind
works (psychology)
●
Build computational models to test those theories
●
Clearly related to the goals of artificial intelligence,
but not directly trying to make better computer
programs
●
Cognitive Science can clearly inform AI, but we may
want to ignore the advice it gives us
●
We may get better behaviour by not thinking
humanly

5
 Thinking Rationally – Laws of
Thought
●
Rational arguments and logical deductions:
– Socrates is a man.
– All men like football.
– Therefore, Socrates likes football.
●
Problem one: it’s really not at all easy to translate all
the knowledge in the world into logical formulae.
●
Problem two: even if you can, the problem of inference
turns out to be computationally nigh on impossible.
●
Problem three: even if you are the world’s greatest
logician, does this help you decide what to do next?

6
 Acting Rationally: The Rational
Agent
●
An agent is just something which lives in an
environment and performs actions
●
A rational agent is one that acts in order to achieve the
best outcome whenever it needs to choose one action
over another
●
As an observer of such an agent, we say that it is
acting in an intelligent way: it exhibits intelligent
behaviour
●
To act in a rational way requires aspects of the other
approaches mentioned, but it is more general, and it is
more amenable to scientific experiment

7
 What does Rational Mean?
●
A rational agent is one that acts in order to achieve
the best outcome whenever it needs to choose one
action over another
●
Rational means that, given all the things the agent
has perceived in its lifetime and its current goals, it
chooses its action to maximise its expected reward
●
Rationality is not the same as omniscience: the actual
reward may turn out to not as good as expected
●
Rationality means making an informed decision –
given what we know, what’s the best thing to do?

8
 What’s Needed for Rationality
●
Problem-solving skills: being able to formulate
problems and search for solutions to those problems
●
Game playing: knowing what action to do next in an
adversarial situation in order to win
●
Planning: deciding on a sequence of actions that will
achieve a goal or a set of goals
●
Knowledge representation: holding knowledge about
the way the world works
●
Learning: finding out how to perform better

9
An Intelligent Agent

10
Example: A Robot Vacuum Cleaner

11
 Percept and Percept Sequences
●
A percept is a complete set of readings from all of
the agent’s sensors at an instant in time
●
For the robot vacuum cleaner, this will consist of its
location and whether the floor is clean or dirty
●
Example percept: [A, dirty]
●
A percept sequence is a complete ordered list of the
percepts that the agent received since time began
●
Example: [ [A, dirty], [A, dirty], [A, clean], [B, dirty], ...
]

12
 An Agent Function
●
An agent function is a theoretical device which
maps from any possible percept sequence to
an action:

13
 An Agent Program
●
An agent program is what we run on the
architecture of the agent to implement the agent
function:

●
What is the right function?
●
Can it be implemented efficiently on the
architecture?

14
Evaluating the Agent: Performance
●
How can we evaluate an agent, such as our robotic
vacuum cleaner?
●
Look at its performance on the task in hand
●
In other words, give it a score: how would you do
this for the vacuum cleaner?
●
The more intelligent the agent, the higher its score
●
Simple agents can score well at simple tasks, but for
complex tasks we need more sophisticated agents:
– rational agents which reason to achieve a high score

15
 PEAS
●
To design a rational agent, we must specify the
“task environment” for the agent
●
We use the acronym PEAS for this:
– Performance: how well does the agent do?
– Environment: what does the agent’s environment
look like?
– Actuators: what actuators does the agent have to
perform its actions with?
– Sensors: what sensors does the agent have to
perceive the environment with?

16
 Properties of Environments
●
An unknown and cluttered room with other agents
in it is a more difficult place to operate in than an
empty room
●
A simple task in a simple environment means that
we can get top performance out of a simple agent
●
A complex task in a complex environment
requires a very sophisticated agent
●
Consider the task and environment for an
automated taxi driver!

17
 Dimensions of the Environment
●
We use six dimensions to define the
environment:
– Fully observable or partially observable?
– Single agent or multi-agent?
– Deterministic or stochastic?
– Episodic or sequential?
– Static or dynamic?
– Discrete or continuous?

18
 Fully vs. Partially Observable
●
If an agent’s sensors give it access to the complete
state of the environment at each point in time, then
we say that the task environment is fully observable.
●
A task environment is effectively fully observable if
the sensors detect all aspects that are relevant to
the choice of action; relevance, in turn, depends on
the performance measure.
●
Fully observable environments are convenient
because the agent need not maintain any internal
state to keep track of the world.

19
 Fully vs. Partially Observable
●
An environment might be partially observable
because of noisy and inaccurate sensors or
because parts of the state are simply missing
from the sensor data, e.g.,
– a vacuum agent with only a local dirt sensor cannot
tell whether there is dirt in other squares
– an automated taxi cannot see what otherdrivers are
thinking.

20
 Single Agent vs. Multiagent
●
An agent solving a crossword puzzle by itself is
in a single-agent environment, whereas an
agent playing chess is in a two-agent
environment.
●
But it is not always is to know which entities in
an agent’s environment are other agents or can
just be treated as part of the environment.

21
 Single Agent vs. Multiagent
●
The key distinction is whether B’s behavior is best described
as maximizing a performance measure whose value
depends on agent A’s behavior.
●
For example, in chess, the opponent entity B is trying to
maximize its performance measure, which, by the rules of
chess, minimizes agent A’s performance measure.
●
Thus, chess is a competitive multiagent environment.
●
In the taxi-driving environment, on the other hand, avoiding
collisions maximizes the performance measure of all agents,
so it is a partially cooperative multiagent environment. It is
also partially competitive because, for example, only one car
can occupy a parking space.

22
 Deterministic vs. Stochastic
●
If the next state of the environment is completely
determined by the current state and the action
executed by the agent, then we say the
environment is deterministic; otherwise, it is
stochastic.
●
Taxi driving is stochastic because one can never
predict the behavior of traffic exactly
●
A tyre can blow out and engine problems can arise
suddenly
●
The vacuum world as described is deterministic

23
 Episodic vs. Sequential
●
In an episodic task environment, the agent’s
experience is divided into atomic episodes.
●
In each episode the agent receives a percept and
then performs a single action. The next episode does
not depend on the actions taken in previous episodes.
●
An agent that has to spot defective parts on an
assembly line bases each decision on the current
part, regardless of previous decisions
●
The current decision doesn’t affect whether the next
part is defective

24
 Episodic vs. Sequential
●
In sequential environments, on the other hand,
the current decision could affect all future
decisions.
●
Chess and taxi driving are sequential: in both
cases, short-term actions can have long-term
consequences.
●
Episodic environments are much simpler than
sequential environments because the agent
does not need to think ahead.

25
 Static vs. Dynamic
●
If the environment can change while an agent is
deliberating, then we say the environment is dynamic
for that agent; otherwise, it is static.
●
Static environments are easy to deal with because the
agent need not keep looking at the world while it is
deciding on an action, nor need it worry about the
passage of time.
●
Dynamic environments, on the other hand, are
continuously asking the agent what it wants to do.
●
Taxi driving is dynamic. Crossword puzzles are static.

26
 Discrete vs. Continuous
●
The chess environment has a finite number of
distinct states (excluding the clock).
●
Chess also has a discrete set of percepts and
actions.
●
Taxi driving is a continuous-state and continuous-
time problem: the speed and location of the taxi
and of the other vehicles sweep through a range of
continuous values and do so smoothly over time.
●
Taxi-driving actions are also continuous (steering
angles, etc.).

27
 Task Environments and their
Characteristics

* After the cards have been dealt

28
 Reactive Agents
●
If an agent only “reacts” to its current percept,
we call it a reactive (or reflex) agent
●
Actions are chosen rules of the form “if
condition then action” (or something equivalent
to this)
●
Example: the simple vacuum cleaner controller
●
Good performance only at simple tasks - would
need an unfeasibly large number of rules for a
complicated task, such as playing chess

29
 Deliberative Agents
●
Deliberation is the process of thinking about how
your actions will affect the world before
committing to it
●
Given a choice of actions, choose the one that
gets you closer to your goals, or closer to a state
of high utility
●
Since the goal or state of high utility may be some
way off, need to think about sequences of actions
●
This leads to a search problem...

30