Extensive form games

The normal form representation is unable to capture games that unfold over time.

→ Fails to capture situations in which the order of moves might be important. (In
terms of information sets of each player when they move, not in chronological
terms)

As an example, consider the familiar Battle of the Sexes game:

Suppose players move sequentially with Alex moving first. Here Chris makes a move after
learning what Alex did, hence the difference is in what Chris knows when Chris makes a
move. Furthermore Alex knows, by common knowledge of rationality, that Chris will choose
to follow Alex because it is Chris’s best response to do so. As a result Alex can get what
Alex wants → “first-mover advantage.”

Note : “first-mover” is not always an advantage. Example : Matching Pennies game.

Here we want to represent formally such sequential strategic situations and apply strategic
reasoning to these new representations.→ Solution concepts capturing idea of sequential
rationality.

Section 1 : The Extensive-Form Game

→ Representation for games that unfold over time and in which some players move after
they learn the actions of other players. The innovation → is to allow the knowledge of
some players, when it is their turn to move, to depend on the previously made choices of
other players.

As with the normal form, two elements must be part of any extensive- form game’s
representation:
1. Set of players, N .
2. Players’ payoffs as a function of outcomes, {vi(.)}i∈N .

We need two other components to capture sequential play:

3. Order of moves.
 4. Actions of players when they can move.

Because some players move after choices are made by other players, we need to be able to
describe the knowledge that players have about the history of the game when it is their
turn to move.

→ More precisely it is not the chronological order of play that matters, but what
players know when they make their choices.

→ To represent the way in which information and knowledge unfold in a game, we

add a fifth component to the description of an extensive-form game:

5. The knowledge that players have when they can move.

Also, we must account for the possibility that some random event can happen during the
course of the game.
→ Stages in a game in which some uncertainty is resolved are called moves of Nature.
→ Nature moves according to some fixed and exogenous probability distribution to the
game.

6. Probability distributions over exogenous events.

Finally, to be able to analyze these situations with the methods and concepts to which we
have already been introduced, we add a final and familiar requirement:

7. The structure of the extensive-form game represented by 1–6 is

common knowledge among all the players.

As a formal notation, we will use decision trees to capture multiplayer strategic situations.

GAME TREES

A game tree will offer a diagrammatic description that is suitable to represent extensive-
form games.
Consider, for example, a game that is very common and falls under the general category of a
“trust game.” (Driver-Mechanic)
Player 1 first chooses whether to ask for the services of player 2.
→ He can trust player 2 (T ) or not trust him (N ), the latter choice giving both players a
payoff of 0.
→ If player 1 plays T , then player 2 can choose
- to cooperate (C), which represents offering player 1 some fair level of service,
- or defect (D), by which he basically cheats player 1 with an inferior, less costly to
provide service.
Assume that if player 2 cooperates then both players get a payoff of 1, while if player 2
chooses to defect then player 1 gets a payoff of -1 and player 2 gets a payoff of 2.

This very simple structure includes most of the elements we described, but it still lacks
how we capture knowledge?

For example : Battle of the Sexes game, in which Alex moved first,

The sequence of choices will result in one of the outcomes at the bottom of the tree.
However, how can we distinguish between the sequential case in which player 2 knows the
move of player 1 and the simultaneous case in which player 2 moves after player 1 but is
ignorant about player 1’s move?
Some necessary notation first,

How do we describe the knowledge of each player when it is his turn to move? We now
proceed to put structure on the information that a player has when it is his turn to move.

Example:

Player 1 chooses from the action set A1 = {O, F}, and player 2 chooses from A2 = {o, f} ,
without observing the choice of player 1.
→ On the right side of Figure 7.4, the dashed line connecting x1 with x2 denotes that both are
in the same information set.

→ In this example player 2 cannot distinguish between x1 and x2, so that h2 = { x1, x2}
Imperfect versus Perfect Information
We defined games of complete information in Chapter 3 as the situation in which each
player i knows the action set and the payoff function of each and every player j ∈N, and
this itself is common knowledge.
→ This definition sufficed for the normal- form representation.
→ For extensive-form games, however, it is useful to distinguish between two different
types of complete-information games:
Definition 7.3 A game of complete information in which every information set is a
singleton and there are no moves of Nature is called a game of perfect information. A
game in which some information sets contain several nodes or in which there are moves of
Nature is called a game of imperfect information.

Note that any simultaneous-move game is a game of imperfect information. Examples

include the simultaneous-move Battle of the Sexes → endogenous uncertainty

Games of imperfect information are also useful to capture the uncertainty a player may
have about acts of Nature → exogenous uncertainty (see book pp137)

Notice that both situations share a common feature: occurrences that some player does not
know are captured by uncertainty over where he is in the game, be it from exogenous or
endogenous uncertainty. In either case a player must form beliefs about the unobserved
actions, of Nature or of other players, in order to analyze his situation.

Section 2 : Strategies and Nash equilibrium

Pure Strategies
Consider the sequential-move Battle of the Sexes game described again in Figure 7.7.

Player 1 has a single information set with one node, so for him a pure strategy is as simple
as “play O” or “play F .”
For player 2, however, things are a bit more involved. Player 2 has two information sets,
each associated with a different action of player 1.
This simple example demonstrates that when a player’s move follows after the realization
of previous events in the game, and if the player can distinguish between these previous
events (they result in different information sets), then he can condition his behavior on the
events that happened.
A strategy is therefore no longer a simple statement of what a player will do, as in the
normal-form simultaneous-move game. Instead we have
Pure Strategies in Extensive-Form Games A pure strategy for player i is a complete plan
of play that describes which pure action player i will choose at each of his information
sets.

→ If we consider the simultaneous-move Battle of the Sexes game in Figure 7.8, the pure
strategies for player 1 are S1 ={O, F} , and those for player 2 are S2 = {o, f} because
each player has only one information set.

→ In contrast, in the sequential-move Battle of the Sexes game in Figure 7.7, player 2 has
two distinct information sets in which he can choose o or f . That is, player 2’s choice of
action from the set {o, f} can be made contingent on what player 1 does → “complete
plan of play”

Thus, we can describe the set of pure strategies for player 2 as follows:

S2 = {oo, of, fo, ff },

For player 1 the pure strategy set remains S1 = {O, F} .

Let Hi be the collection of all information sets at which player i plays, and let hi ∈ Hi be one
of i’s information sets. Let Ai(hi) be the actions that player i can take at hi , and let Ai be the
set of all actions of player i, (i.e., the union of all the elements in all the sets
Ai(hi)). We can now define a pure strategy as follows:
Thus, referring to the Battle of the Sexes game in Figure 7.7, even though player 2 has only
two actions from which to choose, the two actions translate into four pure strategies because
he has two information sets.

NOTE : a potentially small set of moves can translate into a much larger set of strategies
when sequential moves are possible, and when players have knowledge of what preceded
their play. In general assume that player i has k > 1 information sets, the first with m1
actions from which to choose, the second with m2, and so on until mk. Letting |Si |denote
the number of elements in Si , the total number of pure strategies player i has is

|Si |= m1 × m2 × . . . × mk.

Mixed versus Behavioral Strategies

The definition of mixed strategies follows immediately, just like in the normal-form game:
Definition 7.5 A mixed strategy for player i is a probability distribution over his pure
strategies si ∈ Si .

→ a player randomly chooses between all his pure strategies—in this case all the
complete plans of play (before the game is played) —and once a particular plan is
selected the player follows it.

→ In the sequential-move Battle of the Sexes game in Figure 7.7, the previous
definition of a mixed strategy implies that player 2 can randomize among any of his
four pure strategies in the set S2 ={ oo, of, fo, ff}

You may notice that this interpretation takes away some of the dynamic flavor that we set
out to capture with extensive-form games. More precisely, when a mixed strategy is used,
the player selects a plan randomly before the game is played and then follows a particular
pure strategy.

→ To allow for strategies that let players randomize as the game unfolds we define a new
concept as follows:

In a behavioral strategy, a player mixes among his actions whenever he is called to play.
This differs from a mixed strategy, in which a player mixes before playing the game but
then remains loyal to the selected pure strategy.

Consider the sequential-move Battle of the Sexes game in Figure 7.9. Player 2 has two
information sets associated with the nodes x1 and x2, which we denote hO and hF ,
respectively, and in each he can choose between two actions in A2 = {o, f} .
 → A pure strategy would be an element from S2 = { oo, of, fo, ff}

→ A mixed strategy would be a probability distribution (poo, pof , pfo, pff ), where
ps2 ≥ 0 and poo + pof + pfo + pff = 1.

→ Denote a behavioral strategy as four probabilities, σ2(o(h2O)), σ2(f(h2O)),

σ2(o(h2F )), and σ2(f(h2F )), where σ2(o(h2O)) + σ2(f(h2O)) = σ2(o(h2F )) + σ2(f(h2F
)=1
In Figure 7.9 we have used σ2(o(hO)) =1/3, σ2(f(h2O))= 2/3, σ2(o(h2F )) = 1/2, and σ2(f(h2F
))= 1/2

Important note: Any randomization over play can be represented by either mixed or
behavioral strategies provided the game is one of perfect recall
1) Given the mixed strategy (poo, pof , pfo, pff ) can we define a behavioral strat
egy σ2(a2(h)) = Pr{a2│h} that yields the same randomization over outcomes?
→ Conditional on reaching x1 the probability of playing o is Pr{o│O} = poo +
pof , and the probability of playing f is Pr{ f│O} =pfo + pff , so that
Pr{o│O}+ Pr{o│O} =1.
→ Similarly, conditional on reaching x2 the probability of playing o is Pr{o│F} =
poo + pfo , and the probability of playing f is Pr{ f│F} =pof + pff , so that
Pr{o│O}+ Pr{o│O} =1.
→ Thus we can define a behavioral strategy σ2(a2(h)) = Pr{a2│h} that yields the same
randomization over outcomes as the mixed strategy(poo, pof , pfo, pff ).

2) Given a behavioral strategy, we can find a mixed strategy that leads to the same
outcomes?
Again consider the behavioral strategy σ2(o(hO)) =1/3, σ2(f(h2O))= 2/3, σ2(o(h2F )) =
1/2, and σ2(f(h2F ))= 1/2
We know that if the player uses a mixed strategy (poo, pof , pfo, pff ),
 → then conditional on player 1 choosing O, action o will be chosen with probability
poo + pof , and action f will be chosen with probability pfo + pff .

→ similarly, conditional on player 1 choosing F, action o will be chosen with

probability poo + pfo, and action f will be chosen with probability pof + pff .

Thus to replicate the behavioral strategy, four equalities must be satisfied by the
mixed strategy (poo, pof , pfo, pff ):

It can be shown that this particular behavioral strategy can be generated by an infinite
number of different mixed strategies (poo, pof , pfo, pff ). You may verify that (1/3, 0, 1/6,
1/2) as well as (1/6, 1/6, 1/3, 1/3) will lead to equivalent outcomes as as the behavioral
strategy. (See text p 142)

Conclusion : This example suggests that any randomization over play can be represented
by either mixed or behavioral strategies. As it turns out, this is true under a rather mild
condition.

Definition 7.7 A game of perfect recall is one in which no player ever forgets
information that he previously knew.

That is, a game of perfect recall is one in which, if a player is called upon to move more
than once in a game, then he must remember the moves that he chose in his previous
information sets. Kuhn (1953) proved that mixed and behavioral strategies are equivalent, in
the sense that given strategies of i’s opponents, the same distribution over outcomes can be
generated by either a mixed or a behavioral strategy of player i.

Normal-Form Representation of Extensive-Form Games

Consider the two variants of the Battle of the Sexes game presented in Figure 7.11.
The matrix form for the simultaneous-move version in the right panel is :

Now consider the sequential-move Battle of the Sexes game depicted in the left panel.
Recall that S1 = {O, F} and S2 = {oo, of, fo, ff}. This game can be represented by a 2 ×
4 matrix as follows:

Transforming extensive-form games into the normal form → loose the sequence of the
game.

Still a useful exercise since the concept of a Nash equilibrium is static in nature, Therefore
the normal-form representation of an extensive form will suffice to find all the Nash
equilibria of the game.

This is particularly useful if the extensive form is a two-player game with a finite number
of strategies for each player, because we can write its normal form as a matrix and solve it
with the simple techniques developed earlier. As we will now see, this approach has some
useful implications.

Remark Though every extensive form has a unique normal-form representation, it is not
true that every normal form can be represented by a unique extensive-form game. Consider
the following matrix for a normal-form game:

This normal form is consistent with either of the two game trees depicted in Figure 7.12.
Notice that the extensive form on the left is a game of perfect information while the game
on the right is one of imperfect information.

Section 3 : Nash Equilibrium and Paths of Play

We are now ready to analyze the equilibrium predictions in extensive-form games using the
concept of Nash equilibrium.

By transforming an extensive form into its normal-form representation, we can use the
normal form to find all the Nash equilibria of the game.

Consider the sequential-move Battle of the Sexes game depicted in the left panel of Figure
7.11.

Consider it’s normal-form representation,

→ It is easy to see that we can “replicate” the outcomes of the Nash equilibria we
found for the simultaneous version of this game with the strategies (O, oo) and (F,
ff).
→ underlining the best responses of player 1 to each column and overlining the best
responses of player 2 to each row → a third pure-strategy Nash equilibrium
exists: (O, of ).

Notice that in the sequential-move Battle of the Sexes example there are two Nash
equilibria that result in the exact same outcome of both players going to the opera: (O, of )
and (O, oo).

→ the difference between these two equilibria is not in what the players actually play
in equilibrium, but instead what player 2 plans to play in an information set that is
not reached in equilibrium.
This leads us to the following point:

In the extensive form, every outcome of the game is associated with a unique path of play

We also know that players will play a Nash equilibrium because, given their correct beliefs
about the strategies of their opponents, it is in their best interest to stick with their
equilibrium strategies.

→ This implies that if a strategy profile s∗ is a Nash equilibrium, each player i

prefers sticking to the predicted path of play over “leaving” it and choosing some
other path in the game, given his belief that the other players will stick to s∗ . It is
therefore useful
−i to define the following:

Using this definition we can reinterpret the forces that make a Nash equilibrium prediction
self-enforcing : In a Nash equilibrium players choose to proceed on the equilibrium
path because of their beliefs about what the other players are doing both on and off the
equilibrium path.

In Figure 7.13 the Nash equilibrium (F, ff) of the sequential-move Battle of the Sexes
game is shown with the highlighted edges in the tree.
→ This equilibrium is supported by player 1’s correct belief that if he would deviate
from the equilibrium path and play O then he would receive 0 because player 2 will
proceed to play f in the information set x1. In other words, the “threat” imposed by
player 2’s strategy of how he will proceed off the equilibrium path is supporting the
actions of player 1 on the equilibrium path.
Notice, however, that the Nash equilibrium (F, ff) has a rather curious feature.

→ The reason that player 1 is not willing to deviate from F to O is his belief that
player 2 will choose f in information set x1.

→ However, choosing f after O is an irrational choice of player 2 given the payoffs

of the game, implying that this “threat” is somewhat incredible.

This observation sheds some light on a weakness of the normal-form representation in that
it treats all choices as “once-and-for-all” simultaneous choices, and thus beliefs can never
be challenged. We address this weakness in the next chapter with Subgame perfection

Summary

• In addition to the set of players, their possible actions, and their payoffs from
outcomes, the extensive-form representation captures the order in which they play
as well as what each player knows when it is his turn to move. Game trees provide
a useful tool to describe extensive-form games.
• If a player cannot distinguish between two or more nodes in a game tree then they
belong to the same information set. Care has to be taken to specify correctly each
player’s information sets.
• In games of perfect information, every player knows exactly what transpired before
each of his turns to move, so each information set is a singleton. If this is not the
case then the players are playing a game of imperfect information.
• A pure strategy defines a player’s deterministic plan of action at each of his
information sets. A mixed strategy is a probability distribution over pure strategies,
while a behavioral strategy is a plan of probability distributions over actions in
every information set.
• Every extensive-form game has a unique normal-form representation, but the reverse
is
not necessarily true.