MSC COMPUTER SCIENCE

Agent interactions- Game theory, agent

coordination
SESSION TOPICS
1. Definitions

2. The structure of a Game

3. Representing Games

4. Various types of Games

5. Applications of Game Theory
6. Agent Coordination

October, 2009 School of Computing and

Informatics, University of Nairobi
 MSC COMPUTER SCIENCE
Agent interactions- Game theory, agent
coordination
OVERVIEW OF GAME THEORY (Wikipedia, http://en.wikipedia.org/wiki/Game_theory, )

Game theory (Wikipedia) is a branch of applied mathematics with

applications in economics that studies situations where players
choose different actions in an attempt to maximize their returns.
Game theory (Microsoft ® Encarta ® 2006) is Mathematical analysis of
any situation involving a conflict of interest, with the intent of
indicating the optimal choices that, under given conditions, will lead
to a desired outcome.
Fields: economics; biology; psychology; sociology; philosophy;
political science; ethics; computer science (artificial intelligence and
cybernetics); military.

Contributors: include Émile Borel; John von Neumann; John F. Nash.

October, 2009 School of Computing and

Informatics, University of Nairobi
 ICS 806 - MULTI-AGENT SYSTEMS

Agent interactions- Game theory, agent coordination

What is a game? (James Miller(2003))

A game is a structure consisting of three parts:

1. A set of players
2. Moves the players can make
3. Payoffs the players might receive

The players choose their moves to maximize their

payoff. Each player always assumes that the other
players are also trying to maximize their score.

October, 2009 School of Computing and

Informatics, University of Nairobi
 ICS 806 - MULTI-AGENT SYSTEMS

Representation of games

Games, under game theory, are well-defined

mathematical objects.
A game consists of a set of players, a set of moves (or
strategies) available to those players, and a
specification of payoffs for each combination of
strategies.

There are two ways of representing games:

the normal form
the extensive form.

October, 2009 School of Computing and

Informatics, University of Nairobi
 ICS 806 - MULTI-AGENT SYSTEMS

Normal form
The normal (or strategic form) game representation uses a matrix
which shows the players, strategies, and payoffs (see the example
below). Here there are two players; one chooses the row and the
other chooses the column.
When a game is presented in normal form, it is presumed that each
player acts simultaneously or, at least, without knowing the actions
of the other.

Player 2 chooses Player 2 chooses

left right

Player 1 chooses top 4, 3 -1, -1

Player 1 chooses
0, 0 3, 4
bottom

A normal form game

October, 2009 School of Computing and

Informatics, University of Nairobi
Extensive form
If players have some information about the choices of other players, the
game is usually presented in extensive form. The extensive form can be
used to represent games where some order is important. Games here
are presented as trees. Here each vertex (or node) represents a point of
choice for a player. The player is specified by a number listed by the
vertex. The lines out of the vertex represent a possible action for that
player. The payoffs are specified at the bottom of the tree.

October, 2009 School of Computing and

Informatics, University of Nairobi
Symmetric and asymmetric games
A symmetric game is one in which the payoffs for playing a particular
strategy depend only on the other strategies employed, not on who is
playing them. The identities of the players can be changed without
changing the payoff to the strategies. Most of the commonly studied
2×2 games such as the prisoner’s dilemma are symmetric.

Asymmetric games are the ones where there are no identical strategy sets
for both players. There are different strategies for each player.

E F

E 1, 2 0, 0

F 0, 0 1, 2

An asymmetric game

October, 2009 School of Computing and

Informatics, University of Nairobi
Zero sum and non-zero sum games
These are games where the total benefit to all players in the game, for
every combination of strategies, always adds to zero (or more informally
put, a player benefits only at the expense of others). An example is the
game of chess.

A B

A 2, −2 −1, 1

B −1, 1 3, −3

A Zero-Sum Game

October, 2009 School of Computing and

Informatics, University of Nairobi
Simultaneous and sequential
Simultaneous games - both players move
simultaneously, or if they do not move simultaneously,
the later players are unaware of the earlier players'
actions (making them effectively simultaneous).

Sequential games (or dynamic games) are games where

players move in turns and later players have some
knowledge about earlier actions. This need not be
perfect knowledge about every action of earlier players;
it might be very little information.

Normal form is used to represent simultaneous games,

and extensive form is used to represent sequential ones.

October, 2009 School of Computing and

Informatics, University of Nairobi
Perfect information
Occurs if all players know the moves previously made
by all other players. Sequential games can be
games of perfect information.

Complete information
Occurs where every player knows the strategies and
payoffs of the other players but not necessarily their
actions.

Infinitely long games

Arise where there are no constraints and the games
may last for ever. In such games the focus of
attention is simply on whether one or the other
player has a winning strategy.
October, 2009 School of Computing and
Informatics, University of Nairobi
Applications of game theory
Economics and business – analyze phenomena such as auctions,
bargaining, and voting systems. The focus is on equilibria in games such
as Nash equilibrium where each player’s strategy represents a best
response to the other players strategies.
Biology. The payoffs for games is corresponding to fitness. There is less
focus on equilibria more on evolutionary forces.
Computer science and logic. Game theory is used to model interactive
computations.
Political science. Game theory has been used to explain democratic peace
where the public and open debate in democracies send clear and reliable
information regarding the intentions to other states.
Philosophy. Game theory has been develop a philosophical account of
convention.
Ethics. Some attempts have been made to derive morality from self-interest
given that games like the Prisoner's Dilemma present an apparent
conflict between morality and self-interest, explaining why cooperation is
required despite self-interests.
Sociology. Game theoretic analysis of interactions among prisoners have
been conducted.
Military. It is used to analyze the military options and their implications.
October, 2009 School of Computing and
Informatics, University of Nairobi
AGENT COORDINATION
Basic concepts and definitions
Coordination may be defined as:
• Managing dependencies between activities (Barbuceanu and Fox (1996)).
• Managing interaction among agents in areas such as creation,
destruction, mobility, communication and distribution (Paolo et al. (1999)).
• The process of building programs by gluing together active pieces” =>
“A coordination model is the glue that binds together activities into an
ensemble” (Zambonelli et al. (1999)).

Coordination activity examples: applying mutual exclusion policies;

controlling access to shared resources; using some distributed artificial
intelligence algorithms for collective problem solving.

Coordination model- necessary in guiding Design and Development of

complex Distributed Applications. :– a methodology -: (Marco and
Zambonelli (2005)).

October, 2009 School of Computing and

Informatics, University of Nairobi
Role of coordination (Marco and Zambonelli (2005)):

Micro scale(single agent)

Enable the agents to cooperate to accomplish tasks
that exceed their capabilities as single individuals.

Medium scale (several agents)

Orchestration of distributed activities to improve their
interactions with the surrounding environment.

Global scale (many agents- widely distributed)

Dynamically discover communication partners and to
autonomously engage direct or third-party
interaction patterns.

October, 2009 School of Computing and

Informatics, University of Nairobi
Implementing coordination
• Use communication through some structured language such as KQML
(Barbuceanu and Fox (1996)).
• Use social constraints such as obligations (Barbuceanu et al.(1998)).
• Use reinforcement learning as a means of coordinating agents ( Sandip
and Mahendra (1999)) - agents learn action policies that they in turn use to
optimize their strategies.
• Use a coordination model (Paolo et al.(1999)).

A coordination model consists of the following:

1. Coordinables - entities that interact such as Unix-like processes,
threads, concurrent objects, agents and users.

2. Coordination media- abstractions that enable agent interactions such

as semaphores, monitors, or channels, or more complex media like
tuple spaces, blackboards, pipes, etc.

3. Coordination laws- define the behaviour of the coordination media in

response to interaction events for example communication language,
expresses and exchanges data structures.

October, 2009 School of Computing and

Informatics, University of Nairobi
Practical implementations of agent coordination (Marconi and
Zambonelli(2005), Zambonelli(1999)):
Direct Coordination Models: - agents directly and explicitly communicate
with each other. Based on look up services. service registration,
service discovery and service lookup.
Examples: Client-server middleware systems like Jini, and message-
oriented middleware like JADE.
Limitation: costly communication overheads.
Shared Data Space Models: - shared data spaces support agent
interactions; agents can read and write shared data structures.
Examples include: tuple spaces, JavaSpaces.
Limitation: agents are no longer coupled in their interactions since they
communicate through an intermediary.
Event-Based Models: - agents interact with each other by generating
events and by reacting to events of interest; agents do not interact
explicitly with each other. Agents subscribe, trigger reaction or fire
events to a common pool supported by event dispatcher, subscription
manager and match maker.
Limitation: non coupling of agent that only interact through events.

October, 2009 School of Computing and

Informatics, University of Nairobi
Practical implementations continue (Marcos and Zambonelli(2005)
Field based coordination model- agents operate using an uncoupled and
local (i.e., scalable) interactions. Field-based coordination models can
be summarized as follows:
1. Agents interact through virtual “computational fields,” propagated in
the system by the agents themselves or by the environment in which
agents operate. These fields are types of distributed data structures
conveying information about other agents and about the context in
general, and can provide agents with strong coordination task tailored
context-awareness.

2. The local value of propagated fields can be locally accessed by

agents. This can be used to acquire contextual information and also to
have agents coordinate in a very simple way with other agents, i.e., by
having agents act and move following the waveform” of these fields.
3. Openness and environmental dynamics are adaptively taken into
account in that any change in the system may induce changes in the
fields’ waveform that, consequently, tend to influence agents’
behaviors.
4. The above feedback cycle lets the system self-organize so that a
global coordination pattern is eventually achieved.
October, 2009 School of Computing and
Informatics, University of Nairobi
End of Session (Week7) Exercises
1. What is game theory?

2. How is a game structured?

3. Describe various types of games

4. What is coordination?
5. What is the importance of coordination?
6. How can coordination among agents be
realized?
7. How can coordination be implemented?

October, 2009 School of Computing and

Informatics, University of Nairobi