Chapter 2

Intelligent Agents
 Outline
• Introduction to Agents and Environments
• Acting of Intelligent Agents (Rationality)
• (Task Environment) PEAS
• Structure of Intelligent Agents
• Environment types
• Agent Types:
– Simple reflex agent
– Model-based reflex agent
– Goal-based agent
– Utility-based agent
– Learning agent
 Introduction: Agents
• An Agent is anything that can be viewed as
perceiving its environment through sensors
and acting upon that environment through
actuators.
– Human agent:
• eyes, ears, and other organs for sensors;
• hands, legs, mouth, and other body parts for actuators.
– Robotic agent:
• cameras and infrared range finders for sensors;
• various motors for actuators.
 Agents and environments

• Percept: agent’s perceptual input at any given

instant
• Percept sequence: complete history of
everything the agent has ever perceived
• An agent’s choice of action at any given instant
can depend on the entire percept sequence
observed to date
 Agents and environments
• An agent’s behavior is described by the
agent function which maps from percept
histories to actions:
[f: P*  A]
• Internally, the agent function will be
implemented by an agent program which
runs on the physical architecture to produce f
• agent = architecture + program
 Ex. Vacuum-cleaner world

• Percepts: location
and contents, e.g.,
[A,Dirty]
• Actions: Left, Right,
Suck, NoOp
 Rational agents
• An agent should strive to "do the right thing",
based on what it can perceive and the actions it
can perform.
• The right action is the one that will cause the
agent to be most successful.
 Rational agents….. (cont’d)
• Rational Agent: For each possible percept
sequence, a rational agent should select an
action that is expected to maximize its
performance measure,
• Performance measure: An objective criterion
for success of an agent's behavior
• E.g., performance measure of a vacuum-
cleaner agent could be amount of dirt cleaned
up, amount of time taken, amount of electricity
consumed, amount of noise generated, etc.
 Rational agents ….. (cont’d)
• Rationality is distinct from omniscience (all-
knowing with infinite knowledge)
• Agents can perform actions in order to modify
future percepts so as to obtain useful
information (information gathering, exploration)
• An agent is autonomous if its behavior is
determined by its own Percepts and
experience (with ability to learn and adapt),
without depending solely on build-in
knowledge.
 Task Environment
• Before we design an intelligent agent, we
must specify its “task environment”:
• PEAS is an acronym that summarises
the 4 components of any environment
that it is necessary to define:
– Performance measure,
– Environment,
– Actuators,
– Sensors
 PEAS …. (cont’d)
• Performance measure: A measure of how good
the behaviour of agents operating in the
environment is.
• Environment: What things are considered to be a
part of the environment and what things are
excluded?
• Actuators: How can an agent perform actions in
the environment?
• Sensors: How can the agent perceive the
environment?
 Goals
• An intelligent agent will generally exist for a reason
– to perform some task
• This is generally known as the goal
• Needn’t be a single action or outcome,
– Often used to measure how well the agent is performing
• Sometimes the goal is at the end of a long series of
actions, sometimes each action has a goal of its
own
• Goals can be outcomes to achieve or to avoid
(drive fast, don’t crash)
 Performance Measure
• The agent needs to know if it has reached
the goal or not
• An agent needs to know how to work
towards a goal, but also to be told how
well it is doing
• The performance measure can either be
determined from the sensors, or given as
an external input
 Environment
• An agent interacts with its environment
– It is situated in an environment.
• It collects data from the environment via
sensors
• It acts in/on its environment via actuators (the
output of something: movement, data, power,
etc.) towards some goal
• It might change its environment as it acts
• It might learn from its environment too
 Sensors
• Typical sensors include
– Camera, light detector, IR sensor, etc. for light
– Microphone for sound
– Keyboard / mouse / buttons for typed input
– Touch sensors for impact
– Heat sensor / stress gauge / pressure
sensor / accelerometer for physical
measurements
– Many other sensors are possible.
 Actuators
• Actions are controlled by actuators
• An actuator can be anything that causes
any kind of output
– Varying electrical current
– Pumping hydraulic fluid, moving gears
– Transmitting data, showing images / text
– Changing a temperature / flow rate / pressure
– Moving something!
 PEAS …. (cont’d)
• Agent: taxi driver:
– Performance measure: Safe, fast, legal,
comfortable trip, maximize profits
– Environment: Roads, other traffic,
pedestrians, customers
– Actuators: Steering wheel, accelerator,
brake, signal indicator, horn
– Sensors: Cameras, sonar, speedometer,
GPS, odometer, engine sensors,
keyboard
 PEAS …. (cont’d)
• Agent: Medical diagnosis system
• Performance measure: Healthy patient,
minimize costs, lawsuits
• Environment: Patient, hospital, staff
• Actuators: Screen display (questions,
tests, diagnoses, treatments, referrals)
• Sensors: Keyboard (entry of symptoms,
findings, patient's answers)
 PEAS …. (cont’d)
• Agent: Part-picking robot
• Performance measure: Percentage of
parts in correct bins
• Environment: Conveyor belt with parts,
bins
• Actuators: Jointed arm and hand
• Sensors: Camera, joint angle sensors
 Home Work
• Agent : Table Tennis player
Structure of Intelligent Agents
• Any agent consists of two parts: the agent
architecture and the agent program.
• The architecture is the hardware and the
program is the software.
• agent = architecture + program
• how can we best implement the agent program
on the available architecture?
– use a lookup-table. A lookup-table is simply a table
which contains every possible percept history as
inputs, and best actions as outputs.
 Environment types
Fully observable, partially observable, unobservable
• Fully observable: An agent's sensors give it access to the
complete state of the environment at each point in time. A
task environment is effectively fully observable if the sensors
detect all aspects that are relevant to the choice of action
• E.g Chess – the board is fully observable, as are opponent’s
moves. Driving – what is around the next bend is not
observable (yet).
• An environment might be partially observable because of
noisy and inaccurate sensors or because parts of the state
are simply missing from the sensor data
• If the agent has no sensors at all then the environment is
unobservable.
 Environment types ….. (cont’d)
• Deterministic, stochastic, Strategic
• Deterministic : The next state of the environment is
completely determined by the current state and the
action executed by the agent. Ex Crossword puzzle
• If there are apparently “random” events that can
make the next state unpredictable, lack of a good
environment model, or lack of complete sensor
coverage, the environment is stochastic.
• If the environment is deterministic except for the
actions of other agents, then the environment is
Strategic
 Environment types ….. (cont’d)
• Episodic, sequential
• Episodic: The agent's experience is divided into
atomic "episodes" (each episode consists of the
agent perceiving and then performing a single
action), and the choice of action in each episode
depends only on the episode itself.
• the next episode does not depend on the actions
taken in previous episodes. Ex part picking robot
• Otherwise it is sequential.
 Environment types ….. (cont’d)

• In sequential environments, on the other

hand, the current decision could affect all
future decisions.
• Chess and taxi driving are sequential: in
both cases, short-term actions can have
long-term consequences.
 Environment types ….. (cont’d)
• Static, dynamic, semi-dynamic:
• Static: The environment is unchanged while an agent is
deliberating. (The environment is semi-dynamic if the
environment itself does not change with the passage of
time but the agent's performance score does)
• If the environment can change while an agent is
deliberating, then we say the environment is dynamic for
that agent
• Taxi driving is clearly dynamic; Chess, when played with
a clock, is semi-dynamic; Crossword puzzles are static.
 Environment types ….. (cont’d)
• Discrete, continuous
• Discrete: A limited number of distinct, clearly
defined percepts and actions.
• If the number of actions and/or percepts are
effectively unlimited it is a continuous
environment.
• The discrete/continuous distinction applies to the
state of the environment, to the way time is
handled, and to the percepts and actions of the
agent
 Environment types ….. (cont’d)
• Single-agent, multi-agent
• If there are no other agents in the
environment we say it is a Single-agent
environment. If there are other agents it is
a multi-agent environment.
• multi-agent
– Competitive,
– cooperative
Environment types ….. Summary

The environment type largely

determines the agent design
The simplest type of environment is
fully-observable, deterministic, episodic,
static, discrete and single-agent.
The real world is (of course) partially
observable, stochastic, sequential,
dynamic, continuous, multi-agent
 Summary of types of env’ts
Task Environment Observable Deterministic Episodic Static Descrete Agents

Crossword puzzle Fully Deterministic Sequential Static Descrete Single

Chess with a Clock Fully Deterministic Sequential Static Descrete Multi

Poker Partially Stochastic Sequential Static Descrete Multi

Backgammon Fully Stochastic Sequential Static Descrete Multi

Taxi Driving Partially Stochastic Sequential Dynamic Contineous Multi

Medical Diagnos Partially Stochastic Sequential Dynamic Contineous Single

Image Analysis Fully Deterministic Episodic Semi Contineous Single

Part Picking Robot Partially Stochastic Episodic Dynamic Contineous Single

Refinery Control Partially Stochastic Sequential Dynamic Contineous Single

Interactive English
Tutor Partially Stochastic Sequential Dynamic Descrete Multi
Chess
Backgammon
Crossword
Part Picking robot
Table Tennis
Self Driving cars
Humanoid Robot
Human Sensors
Sensing
 Agent types
Agents can be grouped into five classes based
on their degree of perceived intelligence and
capability. All these agents can improve their
performance and generate better action over the
time. These are given below:
•Simple reflex agents
•Model-based reflex agents
•Goal-based agents
•Utility-based agents
•Learning Agents
 Simple reflex agents
• Uses condition-action rule
• These agents use only the current percept, so
have no memory of past percept.
• The rules are of the form “if this is the percept
then this is the best action”.
– Ex. if car-in-front-is-braking then initiate-
braking
• In particular, they cannot base decisions on
things that they cannot directly perceive, i.e. they
have no model of the state of the world.
• These agents only succeed in the fully
observable environment.
 Simple reflex agents
• Humans also have many such connections,
some of which are learned responses (as for
driving) and some of which are innate reflexes
(such as blinking when something approaches
the eye).
• Although many more complex agents may
contain some condition-action rules, an agent
that is only a simple reflex agent is clearly
limited in the type of rational behaviour it can
produce.
Simple reflex agents
 Cot….

• Problems for the simple reflex agent design

approach:
– They have very limited intelligence
– They do not have knowledge of non-perceptual
parts of the current state
– Mostly too big to generate and to store.
– Not adaptive to changes in the environment.
 Model-based reflex agents
• A reflex agent with internal state.
• The Model-based agent can work in a partially observable
environment, and track the situation.
• maintain an internal model of the world, which is updated by
percepts as they are received. In addition, they have built-in
knowledge (i.e. prior knowledge) of how the world tends to evolve.
• It works by finding a rule whose condition matches the current
situation (as defined by the percept and the stored internal state)
and then doing the action associated with that rule.
• However, model-based agents are still basically reflex agents –
they use a set of condition-action rules to determine their
behaviour. This makes it difficult for them to be able to plan to
achieve longer terms goals – they live in the present only and do
not think about the future.
 Model-based reflex agents
• For example, a taxi-driving system may contain the
knowledge that when it doesn’t perceive them, cars
tend to keep moving with roughly the same direction
and speed.
• They also contain knowledge about how their actions
affect the state of the world. For example, the taxi-
driving system may contain the knowledge that if it
turns the steering wheel it will change direction.
• This built-in knowledge, combined with new percept,
enables such agents to update its model of the state of
world, thus enabling it to make better decisions.
 Cot…
• A model-based agent has two important factors:
– Model: It is knowledge about "how things happen in the
world," so it is called a Model-based agent.
– Internal State: It is a representation of the current state
based on percept history.
• These agents have the model, "which is knowledge
of the world" and based on the model they perform
actions.
• Updating the agent state requires information about:
– How the world evolves
– How the agent's action affects the world.
Model-based reflex agents
 Goal-based agents
• The knowledge of the current state environment is not
always sufficient to decide for an agent to what to do.
• The agent needs to know its goal which describes
desirable situations.
• Goal-based agents expand the capabilities of the
model-based agent by having the "goal" information.
• They choose an action, so that they can achieve the
goal.
• These agents may have to consider a long sequence
of possible actions before deciding whether the goal is
achieved or not. Such considerations of different
scenario are called searching and planning, which
makes an agent proactive.
 Cot….
• provide a crude distinction between "happy" and
"unhappy" states
• Although the goal-based agent appears less
efficient, it is far more flexible
• condition-action rules contain implicit goals
• Goal based agents are the same as model based
agents, except they contain an explicit statement of
the goals of the agent.
• These goals are used to choose the next action at
any given time.
 Goal-based agents

•
 Utility-based agents
• Goals alone are not really enough to generate
high-quality behavior.
• For example, there are many action sequences
that will get the taxi to its destination, thereby
achieving the goal, but some are quicker, safer,
more reliable, or cheaper than others.
• Utility based agents deal with this by assigning
utility to each state of the world. This utility
defines how “happy” the agent will be in such a
state.
 Cot..
• These agents are similar to the goal-based agent but
provide an extra component of utility measurement
which makes them different by providing a measure of
success at a given state.
• Utility-based agent act based not only goals but also
the best way to achieve the goal.
• The Utility-based agent is useful when there are
multiple possible alternatives, and an agent has to
choose in order to perform the best action.
• The utility function maps each state to a real number to
check how efficiently each action achieves the goals.
Utility-based agents
 Learning agents
• A learning agent in AI is the type of agent which can learn from
its past experiences, or it has learning capabilities.
• It starts to act with basic knowledge and then able to act and
adapt automatically through learning.
• A learning agent has mainly four conceptual components, which
are:
– Learning element: It is responsible for making improvements by learning
from environment
– Critic: Learning element takes feedback from critic which describes that
how well the agent is doing with respect to a fixed performance standard.
– Performance element: It is responsible for selecting external action
– Problem generator: This component is responsible for suggesting
actions that will lead to new and informative experiences.
• Hence, learning agents are able to learn, analyze performance,
and look for new ways to improve the performance.
 Cot….

• Learning agents are not really an alternative

agent type to those described above.
• All of the above types can be learning agents
• In learning agents diagram, the box labelled
Performance element can be replaced with
any of the 4 types described above.
• The Learning Element is responsible for
suggesting improvements to any part of the
performance element.
 Learning agents…
• The input to the learning element comes
from the Critic.
– The critic analyses incoming percepts and
decides if the actions of the agent have been
good or not
• The Problem Generator is responsible for
suggesting actions that will result in new
knowledge about the world being
acquired.
Learning agents
 .