Outline

Contents
1

Introduction
1.1 Vision of MAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Views of the Agent Systems . . . . . . . . . . . . . . . . . . . . . .

1
1
3

Understanding Agents
2.1 Agent Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 An Abstract Architecture for Agents. . . . . . . . . . . . . . . . . . .
2.3 An Abstract Architecture for Agents. . . . . . . . . . . . . . . . . . .

4
4
7
11

1
1.1

Introduction
The Vision of Multiagent Systems

The Vision of Multiagent Systems. Trends in Computer Science I.

Ubiquity: How to integrate computers everywhere in a cheapest way?
The question not only remains to integration, it is a fact that today computers
are cheaper to produce and are adapted to our life. Thinking about home and
how computeres are integrated in ovens, refrigerators, laundry machines, cars,
mobile phones and more items of common use. Such computers are more and
more communicated with networks to share and publish information. In this
context, the trend is to be more pervasive with computers in our life to simplify
and give more information.
Interconnection: How computers interact better in large distributed systems ?
With the boom of the Internet in 1994, the network was seen as a support for
people to interact among them, not to think in computers interacting. Today,
mostly of the use of the networks are to coordinate computers, where they work
together.
Intelligence: How to increase the complexity of tasks of automated or delegated processes?
Have intelligent computers is a long time searched goal where Artificial Intelligence proposed many strategies but today is not yet accomplished. However,
is a desired goal in computers and programs to take decisions based in some
knowledge and reasoning.

This figure of the Rosette stone represents a master example of how information
was kept among the time. It represents intelligence and interaction between humans
in behave of one goal. The important point is, if we are able to do programs to threat
the information surviving for many time as the roseta and something more than static
information only.
The Vision of Multiagent Systems. Trends in Computer Science II.
Delegation: How to trust and give more control (critical tasks) to computers?
Talking about reaching intelligent computers as a goal takes this point in advance
in how do we are going to trust that computers can take the good decisions.
Wooldrige proposes in his book to think in a critical task as an airplanes traffic
controller in an airport. The task should be very good to be automated (a hard
work for humans) but how to trust that we can delegate the responsibility giving
enough trusting for the safety of the taken decisions (the planes are critical and
no accidents can be in consequence of bad delegated decisions).
Human delegation: How computers represent our best interests while interacting with
other humans or systems?
The final goal is that the computers automates taks for humans and helps us in
the dairy activities. For that, the computers must understand our world, how do
we react, interact and create concepts.
The Vision of Multiagent Systems. The Emergence of a New Field.
Definitions
Agent: An agent is a computer system capable of independent action on behalf of its
user or owner.
MultiAgent System: A number of agents interacting over the network. They act on
behalf of owners goals and motivations.
Agents Abilities
Cooperate
Coordinate
Negotiate

The Vision of Multiagent Systems. Open Questions.

How cooperation can emerge in agent societies?
How agents can communicate in languages expressing their goals and beliefs?
How the agents can recognize their goals, beliefs and actions to negotiate and
reach agreements?
How autonomous agents can coordinate actions to cooperate in behalf of their
goals?

1.2

Different Approaches for Multiagent Systems

The Involved fields. A Highly Interdisciplinary Research Field

Agents as software engineering paradigm
From the past two decades research work has been focused in how to model,
understand and implement systems in which interaction is present. There is an
understanding in how software can became complex when interaction needs the
coordination of many components (units of software programs). Todays, modeling passive objects is not enough to describe an application and since the agents
can be seen as active components, the interest in the software engineering community has grown. So, Agents are a new paradigm in Software Engineering, a
new challenge for software development.
Agents as tool for understanding human societies
To understand interactions between humans, computer agents are a tool for simulation. Since many agents can have a goal, a behavior representing a member of
the society, simulations can be done to predict or understand behaviors. It takes
fields of knowledge as Psychology, Economics, and History as in the science
fiction novel of Issac Asimov "The Foundation".

The Involved fields. Objections to Multi Agent Systems

Is not all distributed and concurrent systems
In the community of distributed/concurrent systems interaction among components is highly related to agents but is not all. The difference is that agents
3

suppose to be autonomous capable of making their own decisions. In distributed

systems, the behaviors and processes of the system are modeled from the beginning, it is easier to think in models with finite state machines that could be
predictable in their state. The agents are autonomous and if they have strong
fundaments from distributed/concurrent systems. Other difference is that a distributed system usually have a common goal and agents can have different goals.
Not only maters Artificial Intelligence (AI)
The claim is that agent systems are not a subfield of AI where the computer
agents deals with many problems that AI have not been interested or fully solved
and also that AI. An interesting point is that the agents try to interact in society
doing cooperation and it covers more fields than traditional AI.
Is not only economics or game theory
Game theory, that seems to be a field from economics to study the relations
among economic entities in real world. In recent times, Game Theory was used
to analyze multi agent systems. We should only keep that is a tool to describe
multi agent systems and not all the field.
Not all just social sciences
Social sciences study the behaviors of human societies. Computer agents are
seen as a tool to model human societies. We must relay that social sciences and
computer agents have a lot of things to share each other, and they are distinct
disciplines.

2
2.1

Understanding Agents
An Abstract View of an Agent System

An Agent Model
An agent is a computer system situated in one environment, and it is capable of an
autonomous action in this environment in order to meet design objectives (Wooldridge
and Jennings 1995).

Agent
Sensor
input

Action
output

Environment

On the diagram we can see how an agent can interact with his environment and affect it. Some times in complex systems, the agent have only a partial control of the
environment. Also under this model, the agent may have a set of actions to affect
the environment and reach its own goal. These actions, can be defined according to
the possible situations that the agent faces. The important issue is that an agent can
be seen as a control system with decision structures and can also be compared to a
software daemon working as an autonomous process.
Kinds of Agents Environments
Accessible versus Inaccessible
The agent can or can not obtain accurate information of the environments state.
Deterministic versus non-deterministic
Any action of the agent can or can not be with a single guaranteed effect.
Static versus dynamic
The static environment is changed only by the agent actions. A dynamic have
other processes changing it as in our physical world.
Discrete versus continuos
We say if we have a finite number of actions that we have a discrete environment
or not.
Towards Intelligent Agents
Properties of an intelligent agent
It should be clear that an agent as an autonomous process is capable of taking its own
decisions to reach the goals for them it was designed. Under this scope, an intelligent
agent should cover the properties listed above.
Reactivity: perceive and respond to the environment conditions in function of goals.
A goal directed behavior is some kind of plan or recipe to reach the goal of
an agent and this recipe should include a perception of the environment, how
it changes and creates special conditions to take decisions related to the design
objectives. A simple example is related to an agent thermostat reacting to the
environment changes and if its main goal is to keep a room fresh, under certain
5

limits or changes of temperature it may react compensating with cold currents of

air.
Proactiveness: take initiative to satisfy the goals.
This property is a compromise because an agent reacting all the time againts the
environment changes is not the best solution. One would like to prevent some
situations before they arrive to avoid certain environment conditions. That is the
notion of proactiveness, and is related to the own initiative of the agent after some
experience to reach its designed goals. The complexity to achieve this property
is not trivial and is still an open problem.
Social ability: interact with other agents or humans to reach goals.
To say that an agent has social abilities is not only because it exchanges information with other agents, it is because the agent exchanges information and
negotiates and cooperate with others to reach its own goals. The cooperation and
negotiation ability represents a high complexity because it depends on different
contexts and environments.

Differences of Agents Against other Programing Paradigms.

Agents and Object Oriented Technology (OOT)
This part is important because sometimes, programmers do not see the difference
between agents paradigm and object oriented languages as Java, C++ or Smalltalk. In
objects there is the notions of actors that the claim is that is the same concept as agent,
but OO paradigm is different in many aspects than Agents paradigm and we will clarify
with the next points.
Objects encapsulates one state and communicate by message passing.
If objects communicates by message passing, the agents have in its base a communication also by message passing but they have an special language and they
can define an ontology to have complex communication to reach agreements and
negotiate. Even if the objects can do that, they have not standards like the agents
to do that and objects are in a lower level of abstraction towards communication.
Objects does not exhibit control over its behavior.
The base of an agent is related to its autonomy and the ability to take decisions
in order to accomplish its programmed goal. An object do not exhibit this autonomy, is a passive element that needs to be invoked to be executed. In the case of
a thread, the thread is more related to a machine state not necessarily pertaining
to support all the agent structure.
Agents embody Stronger notion of autonomy and they decides themselves what
to do.
Based in the past paragraph, the autonomy comes with more flexibility because
the agents working in a society can solve problems doing coordination, delegation, communication related to its goals and they do not have an explicit programming as the in object oriented paradigm.
6

 Agents are flexible (reactive, proactive, social) nothing of that in objects.

Also retaking the text above the flexibility that agents can offer to design a system
(specially if it is the case of a distributed system) can not be achieved in the same
manier with the OO paradigm.
Multiagent systems are multithread, each agent have at least one thread of control.
The agents can have multiple threads in order to sense its environment, to manage its communication, to exhibit its behaviors. So, threads is implicit on the
agents, a concept that not all the time is done in the OO paradigm.

Differences of Agents Against other Programing Paradigms

Agents and Expert Systems (ES)
The most famous agent system developed was MYCIN (Hayes-Roth et al. 1983), a
system to help physicians to diagnose blood infections in humans at it worked fine
in its specific domain. In the contrary, there should be taken in account that expert
systems take all the information from humans and do not sense or react or consider an
environment concept as agents. Also, is to remark that an expert system is usually a big
process not intending to cooperate with other programs or processes and this is another
difference. In summary, we can establish the following differences:
ES have not a environment and sensors.
ES are not reactive and proactive.
ES do not have social ability to coordinate, cooperate, and negotiate.

Differences of Agents Against other Programing Paradigms

Agents and Intentional Systems
This area of research establishes mental states in the programs as beliefs, desires,
wishes, hope, and so on. This systems where presented by the philosopher Daniel
Dennet and McCarthy in 1978. The main difference with agent systems is related to:
Intentional systems are nothing more than an abstraction tool.
They are considered a base for agent systems in terms of belief, desire, and
intention.

2.2

Main Agents formal definitions

Main Agents formal definitions

Let define an environment for agents as follows:
E = {e, e0 , }

and the agents actions as:

Ac = {, 0 , }
It is important to remark that the set of actions modify the environment. Also consider
that every action will be executed by an agent leading it to different states. and the r
run of an agent as:

u1

3
2
1
0
eu
e3
e2
e1
r : e0

The run of an agent in an environment has to be seen as a sequence of environment

states and actions together.
Let
R the set of all possible finite sequences (over E and Ac ).
RAc the subset of sequences that end with an action,
RE the subset of sequences that end with an environment state.
The r, r0 , defined runs of an agent will be members of R.
So, in order to represent the effect of actions from an agent in one environment let:
: RAc
(E).
From past definition we can observe that:
Environments are history dependent.
This observations drives to understand that actions played by an agent have an
effect in the current state of the environment. So, a set of actions will produce an
state in the environment. The environment is not independent of past and actions
achieved.
The environment allows non-determinism.
It refers to the uncertainty about the result of performing an action in some state.
A system ends when:
(r) = (there are no more runs)
And formally an environment Env is a triple:
Env = hE, e0 , i
where:
E is a set of environment states
e0 E is an initial state
is a state transformer function

An Agent Model in an Environment

Finally let define: Ag : RE Ac and let be AG the set of all agents. The
model presented relates how agents can inhabit a system if we remember that Ac is
the repertoire of possible actions and RE is the subset of all the runs ending with an
environment state.
Notice that agents are supposed to be deterministic.
We say that a system is a pair of an Agent and an Environment.
So, a run of an agent Ag in an environment Env is R(Ag, Env)
Formally a sequence (e0 , 0 , e1 , 1 , e2 , ) representes a run of an agent Ag in
the environment Env = hE, e0 , i (consider no infinite runs with (r) = ) if :
e0 is the initial state for Env
0 = Ag(e0 )
for u > 0
eu ((e0 , 0 , , u1 )) where u = Ag((e0 , 0 , , eu ))
Finally to agents behaviorally equivalent with respect to the environment are valid
only if:
R(Ag1 , Env) = R(Ag2 , Env)
Please remark that until this section there are no definitions about how the agents are
implemented.
Reactive Agents Definition
Reactive agents respond only to its environment so let:
Ag : E Ac
An example of an reactive agent is a thermostat defined by:
(
heatterof f /if e = temperatureOK
Ag(e) =
heatteron / otherwise
Looking at the past definition is not hard to think in an agent purely reactive that
can have another similar agent. It is compared to animals reacting to one stimulus.

Agents Perception
An agent is composed of data structures where in the figure the see function captures an observation of the environment and the action is the process of decisionmaking. see : E P er and
action : P er? Ac

see

action
Agent

Environment

The past functions maps sequences of percepts to

actions and it is consiedered that Ag = hsee, actioni.
Agents Perception Example
1. Supose two environments e1 E and e2 E such that e1 6= e2 but see(e1 ) =
see(e2 ).
2. Let the set statements y="today is holiday" and x="the room temperature is OK".
3. Then the set of environment E = {{e
x, ye}, {e
x, y}, {x, ye}, {x, y}} = {e1 , e2 , e3 , e4 }.
(
p1 if e = e1 or e = e2 ,
4. So the see function of thermostat only support see(e) =
p2 if e = e3 or e = e4 .
The enviromnent x and y are not really related, a temperature sensor can not know if is
a holiday or not so the final function of see only takes account the different states of x
if what we want for the agent is to control the room temperature.
Agents State
1. Each agent have an internal structure and let I be the set of all internal states of
the agent.
2. So, let perception function from past example see : E P er.
3. And the action-selection function action : I Ac
4. From internal states to actions let define the function next : I P er I.
The agent starts in an initial state i0 . Then an observation of the environment e generates a percep see(e). Then the internal state of the agent is updated via the next
function as next(i0 , see(e)). Then the agent selects an action and it is described by
action(next(i0 , see(e))). When the action is performed the agent repeat the cycle as
in the next figure presented.
10

Agent
see

action

State

next

Environment

2.3

Telling an Agent What to Do

Utility functions
Statement
We want to tell to an agent what to do, without telling how to do it.

Let define an utility function to define how good is an agent state as: u : E R
To avoid only viewing local states let define the function for all the run states as:
u:RR
The measure depends upon the kind of task to carry out by the agent.
Assume that the utility function u has somme uper bound that there exist a k R
such that for all r R we have u(r)k then we can talk about optimal agents.

11

Let define P (r | Ag, Env) to denote the probability that run r occurs when Ag is
placed in Env or :
X
P (r|Ag, Env) = 1
rR(Ag,Env)

Then the optimal agent is :

Definition 1.
max

Agopt = arg
AgAG

P (r|Ag, Env) = 1

rR(Ag,Env)

This experssion es based in decision theory to maximize the expected utility.

Finally let define (Rusel and Subramamian, 1995) a bounded optimal agent to be
implemented in a computer m with certain characteristics as follows:
AG m = {Ag|Ag AG and Ag can be implemented on m}
And a m computer results in the equation for an agent considering the utility function
u : R R in the following expression:
Definition 2.
max

Agopt = arg
AgAG

u(r)P (r|Ag, Env)

rR(Ag,Env)

The importance on this last equation is the fact that we are not considering al the
subset of agents able to have an action in the environment, but the subset of agents able
to act and to fit in the capacity of an m computer with processor and memory specified.
The Utility concept is more to understand agents quoting that is more common to talk
about the achieved goals of an agent rather that its utility.
Definition 3. Is one where the utility function acts as a predicate over runs.
The function (r) where r R will denote a predicate specification.
Formally, the utility function u : R R is a predicate if the range of u is set to
{0,1}. That means that u assing a run either 1(true) or 0(false).
A task environment is defined as a pair of hEnv, i as:
: R 0, 1 and is a predicate over runs.
Let define T E the set of all task environments where it specifies:
The properties of the system the agent will inhabit.
The criteria by which the agent will be judged either for a successful or failed
task.
Given a task environment hEnv, i, we write R (Ag, Env) to denote the set of
runs of Agent Ag in environment Env that satisfy . Formally:
Definition 4. R (Ag, Env) = {r|r R(Ag, Env)and(r)}
12

An agent succeeds in task environment if

R (Ag, Env) = R(Ag, Env)
Definition 5. Ag succeeds in hEnv, i if every run Ag in Env satisfies specification
if:
r R(Ag, Env) we have (r)
Notice that this definition is a pessimistic one because the agent have a success only if
all the possible runs satisfies the specification.
Definition 6. An optimistic function rater than the past one can be defined by:
r R(Ag, Env) such that (r)
A success definition to include a probability distribution over possible outcomes
including the transform sequence can be defined as:
Definition 7.
P (|Ag, Env) =

P (r|Ag, Env)

rR (Ag,env)

Where P (|Ag, Env) is the probability that run r occurs if Ag is placed in Ev.
Two most common kinds of tasks are:
Achievement tasks. Those to achieve state of affairs .
Maintenance tasks. Those to maintain state of affairs .
An achievement of a task is specified by a number of goal states.
Task achievement is one of the most studied problems in AI.
Definition 8. The task environment hEnv, i specifies an achievement task there
is some set G E such that r R(Ag, Env) : (r) is true e G : e r.
G is the set of an achievement task environment as the goal states of the task.
hEnv, Gi is the achievement task environment with G goals over Env environment.
A task environment can be seen as a game where the agent is playing against the
environment or a called game against nature.
Definition 9. If we can identify a subset B of environment states |(r) is false if B
ocurrs in r and true otherwise. So, hEnv, i is a maintenance task if B E such
that (r) e B we have e 3 rr R(Ag, Env).
B is refered as the failure set.
A maintainance task environment is defined by hEnv, Bi.
Maintenance tasks can be seen as a game where the agent manages to avoid all states
in B. The environment as oponent forces the agent to fall in the B states.
13

Synthesizing Agents
How do we obtain an agent to succeed in a given task environment?
1. We can develop an algorithm to synthesize such agents from task environment
specifications.
2. We can develop an algorithm that will directly execute agent specifications in
order to produce appropriate behavior.
Agent Synthesizing
Is Automatic Programming where the goal is a program taking an input from the environment and from task environment automatically generate a successful agent in the
Env.
Definition 10. SYN : T E (AG {})
Think for as a null in Java
SYN can output an agent or
The synthesizing algorithm can be:
sound. whenever it returns an agent and success the task environment that is passed as
input. Formally SYN (hEnv, i) = Ag implies R(Ag, Env) = RP si(Ag, Env)
complete. if it is guaranteed to return an agent whenever there exists an agent that will
succeed in the task environment given as an input. Formally Ag AG s. t.
R(Ag, Env) = R (Ag, Env) implies SYB(hEnv, i) 6=

Summary
Summary
The first main message of your talk in one or two lines.
The second main message of your talk in one or two lines.
Perhaps a third message, but not more than that.

Outlook
Something you havent solved.
Something else you havent solved.

14