Dimensions of variation of task environment

The task environment can be characterized by the following

attributes:
I Fully observable vs partially observable.
I Deterministic vs non-deterministic. If s, s1 ∈ S are states and
a ∈ A is an action then if (s, a) → s1 , s1 completely
determined by s and a - except for other agents - then it is
deterministic. Called stochastic - if it is non-deterministic and
some aspects of the environment can be modelled via
probability distributions (e.g. games of chance).
I Episodic (indp. sense-act chunks) vs sequential (curr. action
affects future)
I Dynamic (environment can change - even during sense-act
cycle) vs static. Called semi-dynamic if environment does not
change with time but agents performance does.
I Discrete vs continuous - nature of time applied to state,
environment, actions.
I Single agent vs multiple agents. Cooperative vs competitive
within a multi-agent environment.
Examples: Dimensions of variation

Task Env Obs(F/P) Det.(D/S) Epis(E/S) Static(S/D) Disc.(D/C) No. of Ags(1/M)

Crossword F D S S D 1
Tournament chess F Str. S Semi D M
Self driving car P S S D C M
Med. diagnosis P S S D C 1
Part picking P S E D C 1
Image analysis F D E Semi D 1
Plant controller P S S D C 1
Tutor P S S D D M
Chem. struc. P S S S D 1
Agent types
Agents can be classified into some broad agent types. The
following agent types are common (from the simple to complex).
I Reactive or reflex agents. Modelled by table driven
representations.
I Rule driven agents. Modelled by situation-action rules.
I Model driven agents. Constructs models of the task
environment, effect of actions on the environment and
changes in the environment due to actions.
I Goal driven agents. Have models and also goals. Typically,
one goal.
I Utility based agents. Generalization of goal driven agents.
Optimize utility in the presence of multiple conflicting goals
using trade-offs.
I Learning agents. f can keep changing based on changing
dependencies. Even when dependencies are assumed constant
they are not known a priori so they may have to be learnt
from data.
Focus of the course: AI as agent design

Course goals: Design agents in different task environments.

Formally, construct function f : S × P × K → A, where S is set of
states, P is set of percepts, K is the knowledge base (history), A
set of actions.
f is the findAction function which takes as argument: current
state, percepts (from sensing), knowledge (past history, other
information) and outputs an action.
To successfully build agents we need appropriate representations
for S, P, K and, A and suitable algorithms that implement f .
Two main approaches

I Deductive or top-down. We have knowledge about the task

domain and it has to be implemented as an agent.
I Inductive or bottom-up. We do not have knowledge about the
domain and an agent has to learn how to act.
I A third hybrid situation is one where we have partial
knowledge about the task domain but learning is also needed
to be able to do the task.