Lecture notes: second set 1

1 Introduction to structure of dynamic oligopoly models

Consider a simple two-rm model, and assume that all the dynamics are deter-
ministic.
Let x
1t
, x
2t
, denote the state variables for each rm in each period. Let q
1t
, q
2t
denote the control variables. Example: xs are capacity levels, and qs are
incremental changes to capacity in each period.
Assume (for now) that x
it+1
= g (x
it
, q
it
), i = 1, 2, so that next periods state is
a deterministic function of this periods state and control variable. (Can allow
for cross-eects with no problem.)
Firm i (=1,2) chooses a sequence q
i1
, q
i2
, q
i3
, . . . to maximize its discounted
prots:

t=0

t
(x
1t
, x
2t
, q
1t
, q
2t
)
where ( ) denotes single-period prots.
Because the two rms are duopolists, and they must make these choices recog-
nizing that their choices can aect their rivals choices. We want to consider
a dynamic equilibrium of such a model, when (roughly speaking) each rms
sequence of qs is a best-response to its rivals sequence.
A rms strategy in period t, q
it
, can potentially depend on the whole history
of the game (H
t1
{x
1t
, x
2t
, q
1t
, q
2t
}
t

=0,... ,t1
), and well as on the time period
t itself. This becomes quickly intractable, so we usually make some simplifying
regularity conditions:
Firms employ stationary strategies: so that strategies are not explicitly a
function of time t (i.e. they depend on time only indirectly, through the
history H
t1
). Given stationarity, we will drop the t subscript, and use
primes

to denote next-period values.
A dimension-reducing assumption is usually made: for example, we might
assume that q
it
depends only on x
1t
, x
2t
, which are the payo-relevant
Lecture notes: second set 2
state variables which directly aect rm is prots in period i. This is usu-
ally called a Markov assumption. With this assumption q
it
= q
i
(x
1t
, x
2t
),
for all t.
Furthermore, we usually make a symmetry assumption, that each rm
employs an identical strategy assumption. This implies that q
1
(x
1t
, x
2t
) =
q
2
(x
2t
, x
1t
).
To characterize the equilibrium further, assume we have an equilibrium strategy
function q

(, ). For each rm i, then, and at each state vector x

1
, x
2
, this
optimal policy must satisfy Bellmans equation, in order for the strategy to
constitute subgame-perfect behavior:
q

(x
1
, x
2
) = argmax
q
{(x
1
, x
2
, q, q

(x
2
, x
1
)) + V (x

1
= g (x
1
, q) , x

2
= g (x
2
, q

(x
2
, x
1
)))}
(1)
from rm 1s perspective, and similarly for rm 2. V (, ) is the value function,
dened recursively at all possible state vectors x
1
, x
2
via the Bellman equation:
V (x
1
, x
2
) = max
q
{(x
1
, x
2
, q, q

(x
2
, x
1
)) + V (x

1
= g (x
1
, q) , x

2
= g (x
2
, q

(x
2
, x
1
)))} .
(2)
I have described the simplest case; given this structure, it is clear that the
following extensions are straightforward:
Cross-eects: x

i
= g (x
i
, x
i
, q
i
, q
i
)
Stochastic evolution: x

i
|x
i
, q
i
is a random variable. In this case, replace
last term of Bellman eq. by E [V (x

1
, x

2
) |x
1
, x
2
, q, q
2
= q

(x
2
, x
1
)].
This expectation denotes player 1s equilibrium beliefs about the evolution
of x
1
and x
2
(equilibrium in the sense that he assumes that player 2 plays
the equilibrium strategy q

(x
2
, x
1
)).
> 2 rms
Firms employ asymmetric strategies, so that q
1
(x
1
, x
2
) = q
2
(x
2
, x
1
)
. . .
Lecture notes: second set 3
Computing the equilibrium strategy q

( ) consists in iterating over the Bell-

man equation (1). However, the problem is more complicated than the single-
agent case for several reasons:
The value function itself depends on the optimal strategy function q

( ),
via the assumption that the rival rm is always using the optimal strategy.
So value iteration procedure is more complicated:
1. Start with initial guess V
0
(x
1
, x
2
)
2. If qs are continuous controls, we must solve for q
0
1
q
0
(x
1
, x
2
) and
q
0
2
q
0
(x
2
, x
1
) to satisfy the system of rst-order conditions (here
subscripts denotes partial derivatives)
0 =
3
_
x
1
, x
2
, q
0
1
, q
0
2
_
+ V
0
1
_
g
_
x
1
, q
0
1
_
, g
_
x
2
, q
0
2
__
g
2
_
x
1
, q
0
1
_
0 =
3
_
x
2
, x
1
, q
0
2
, q
0
1
_
+ V
0
1
_
g
_
x
2
, q
0
2
_
, g
_
x
1
, q
0
1
__
g
2
_
x
2
, q
0
2
_
.
(3)
For the discrete control case:
q
0
= argmax
qQ
_

_
x
1
, x
2
, q, q
0
2
_
+ V
0
_
g (x
1
, q) , g
_
x
2
, q
0
2
___
q
0
2
= argmax
qQ
_

_
x
2
, x
1
, q, q
0
1
_
+ V
0
_
g (x
2
, q) , g
_
x
1
, q
0
1
___
.
(4)
3. Update the next iteration of the value function:
V
1
(x
1
, x
2
) =
_

_
x
1
, x
2
, q
0
1
, q
0
2
_
+ V
0
_
g
_
x
1
, q
0
1
_
, g
_
x
2
, q
0
2
___
. (5)
Note: this and the previous step must be done at all points (x
1
, x
2
) in
the discretized grid. As usual, use interpolation or approximation to
obtain V
1
( ) at points not on the grid.
4. Stop when sup
x
1
,x
2
||V
i+1
(x
1
, x
2
) V
i
(x
1
, x
2
)|| .
In principle, one could estimate a dynamic games model, given time series
of {x
t
, q
t
} for both rms, by using a nested xed-point estimation algo-
rithm. In the outer loop, loop over dierent values of the parameters ,
and then in the inner loop, you compute the equilibrium of the dynamic
game (in the way described above) for each value of .
However, there is an inherent Curse of dimensionality with dynamic
games, because the dimensionality of the state vector (x
1
, x
2
) is equal to
Lecture notes: second set 4
the number of rms. (For instance, if you want to discretize 1000 pts in one
dimension, you have to discretize at 1,000,000 pts to maintain the same
neness in two dimensions!)
Some papers provide computational methods to circumvent this problem
(Keane and Wolpin (1994), Pakes and McGuire (2001), Imai, Jain, and
Ching (2005)). Generally, these papers advocate only computing the value
function at a (small) subset of the state points each iteration, and then
approximating the value function at the rest of the state points using values
calculated during previous iterations.
1.1 Dynamic games with incomplete information
Clearly, it is possible to extend the Hotz-Miller insights to facilitate estimation of
dynamic oligopoly models, in the case where q is a discrete control. Advantage,
as before, is that you can avoid numerically solving for the value function.
Data directly tell you: the choice probabilities (distribution of q
1
, q
2
|x
1
, x
2
);
state transitions: (joint distribution of x

1
x

2
|x
1
, x
2
, q
2
, q
2
).
Papers which develop these ideas include: Bajari, Benkard, and Levin (2008),
Pesendorfer and Schmidt-Dengler (2003), Aguirregabiria and Mira (2007), Pakes,
Ostrovsky, and Berry (2007). Applications include: Collard-Wexler (2006),
Ryan (2006), and Dunne, Klimer, Roberts, and Xu (2006). See survey in Pakes
(2008), section 3.
Here I outline the general setting, which can be considered a multi-agent version
of Rust setup. (Nitty-gritty details are omitted: you should be able to gure it
out!)
First-order Markov
Let

X and q denote the N-vector of rms state variables, and actions.
Symmetric model: rms are identical, up to the current values of their state
variables, and idiosyncratic utility shocks
Lecture notes: second set 5
Utilities: rm is current utility, given her action d
i
and her utility shocks
i,d
i
is:
U(

X, d
i
,

d
i
; ) +
i,d
i
.
All utility shocks are private information of rm i. Also, they are i.i.d. across
rms, and across actions (exactly like the logit errors in Rusts framework).
Because of these assumptions: each rms period t shocks aect only his period
t actions; they do not aect opponents actions, nor do they aect actions in
other periods t

= t.
In single-agent case, this distinction is nonsensical. But in multi-agent context,
it is crucial. Consider several alternatives:
All the shocks are publicly observed each period, but not observed by
econometrician: essentially, we are in the unobserved state variable
world of the last set of lecture notes, except that in multi-agent context,
each rms actions each period depend on all the shocks of all the rms in
that period. Likelihood function gets very complicated.
Shocks are private information, but serially correlated over time: now rms
can learn about their opponents shocks through their actions, and their
beliefs may evolve in complicated fashion. Also, rms may strategically
try to inuence their rivals beliefs about their shocks (signalling). Com-
plicated even from a theoretical point of view.
Shocks are unobserved by both rms and econometrician. In this case,
they are non-structural errors and are useless from the point of view of
generating more randomness in the model relative to the data.
In symmetric equilibrium, rms optimal policy function takes the form d
i
=
d

(

X,
i
). Corresponding choice probability: P(d
i
|

X) (randomness in
i
). These
can be estimated from data.
State variables evolve via Markovian law-of-motion

X

X,

d. This can also be
estimated from the data.
Lecture notes: second set 6
Hence, choice-specic value functions for rm i (identical for all rms) can be
forward-simulated:

V (

X, d
i
= 1; ) =E
vecd
i
|

X
u(

X, d
i
= 1,

d
i
; ) + E

X,

d=(d
i
,

d
i
)
E

|d

i
,

X

_
u(

X

,

d

; ) +

+E

|d

i
,

X

_
u(

X

,

d

; ) +

+
__
(6)
When

X is nite, the matrix representation of value function can also be used
here.
Predicted choice probabilities:

P(d
i
= 1|

X) =
exp(

V (

X, d
i
= 1; ))
exp(

V (

X, d
i
= 1; )) + exp(

V (

X, d
i
= 0; ))
.

2 An example of dynamic oligopoly: automobile market with

secondary markets
We go over Esteban and Shum (2007).
In durable goods industries (like car market), secondary markets leads to intertem-
poral linkages between primary and secondary markets. Used goods of today were
new goods of yesterday.
Interesting economic question: Does this harm or benet producers?
Intuition dierent from static markets:
Benchmark in DG setting is Coase outcome (rms inability to commit to low
levels of production can erode market power)
Vs. this benchmark, 2-mkts can benet producers:
1. 2-mkt oers substitutes for rms new production curtails Coasian ten-
dency to overproduce (commitment benet)
Lecture notes: second set 7
2. With heterogeneous consumers, 2-mkts segment market, allow rms to
target new goods to high-valuation consumers (sorting benet)

Economic Model: Car market

Multiproduct rms producing cars which dier in quality, durability and depreciation
schedule.
Empirical model accommodates cost/demand shocks; for simplicity, describe deter-
ministic model.
Firms j = 1, . . . , N. (e.g. Ford, GM, Honda)
L is total number of brands/models (e.g. Taurus, Accord, Escort).
Firm j produces L
j
models; set of products denoted L
j
.
Model i lasts T
i
periods. There are K

L
i=1
T
i
model-years actively traded
during any given period.
Each model year diers in one-dimensional quality quality ladder
[
1
,
2
, . . . ,
K
,
K+1
= 0]
where
K+1
is quality of outside option.
Notation: depreciation schedules for dierent models
Dene: (i) is ranking of model i when new.
Dene: v((i)) is ranking of 1-yr old; v
2
((i)) v (v((i)) is ranking of
2-yr old, etc.
Depreciation schedule of model i described by sequence
_
(i) , ( (i)) , ... ,
T
i
1
( (i))
_
.
Note: each model has its own depreciation schedule.
Lecture notes: second set 8
Note that rms are asymmetric. Hence equilibrium is characterized by set
of L Bellman equations (as will be derived below).

Economic model: Dynamic demand

Derive from individual-level optimizing behavior.
A continuum of innitely-lived consumers who dier in their preference for
quality (one dimension)
Quasilinear per-period utility: U
t
=
k
+ m p
k
t
, where m is total income.
Assume no liquidity constraints.
Choice set: model-years k = 1, . . . , K, plus outside option (utility normalized
=0)
Consumers incur no transactions costs: abstract away from timing issues.
Implies simple form of dynamic decision rule: in period t, consumer chooses
model-year k yielding maximal rental utility:
k
t
= argmax
k
_
0,
k
p
k
t
+ E
t
p
v(k)
t+1
, k = 1, . . . , K
_
where is discount factor and expected rental price is p
k
t
E
t
p
v(k)
t+1
.
(Drop E
t
for convenience: assume perfect foresight, so E
t
p
t+1
= p
t+1
.)

Demand functions
Given prices p
k
t
, p
(k)
t+1
for all k = 1, . . . , K, each period there are K indierent
consumers

1
t

2
t

3
t
. . .

K
t
0, s.t.
Lecture notes: second set 9
The indierent consumers solve

k
t
p
k
t
+ p
(k)
t+1
=
k+1

k
t
p
k+1
t
+ p
(k+1)
t+1
, for k = 1, . . . , K 1
Consumer heterogeneity: is uniformly distributed.
Derive inverse demand function (subject to non-negativity constraints)
p
k
t
= (
k

k+1
)

_
1
1
M
k

r=1
x
r
t
_
+ p
(k)
t+1
+ p
k+1
t
p
(k+1)
t+1

Supply side
y
t
= [1, x
1
t
, . . . , x
K
t
]

: vector of all cars transacted in period t.

d
t

_
x
(1)
t
, x
(2)
t
, . . . , x
(L)
t
_

: vector of all cars produced in period t.

Dene matrices A and B, to get law of motion for y
t
:
y
t
= Ay
t1
+Bd
t
.
Marginal costs constant; no (dis-)economies of scope: C
jt
=

iL
c
i
x
(i)
t
.
Period t prots for car i is

i
t
_
y
t
, y
t+1
, . . . , y
t+T
i
1
_
=
_
p
i
t
(y
t
, y
t+1
, . . . , y
t+T
i
1
) c
i
t
_
q
1
t
:
depends on past, current, future prodn of car i.
Important: dependence of current prots on future actions leads to a time-
consistency problem, which is absent from usual dynamic problems. Very
roughly, time-inconsistency implies that an agents optimal action in period t
diers depending on whether the agent is deciding in period t, or period t 1,
or period t 2, etc.
Think of durable goods monopoly: in period 1, his optimal period 2 price is the
monopoly price (because that would raise his prots in period 1). But when
period 2 comes, his optimal period 2 price is actually a lower price (since he
wants to sell to people who did not buy in period 1).
Lecture notes: second set 10
For individual rm: t, j N, i L
j
, period-t production x
(i)
t
maximizes
() max
x
(i)
t
,iL
j

=0

iL
j

i
t+
_
y
t+
, y
t++1
, . . . , y
t++T
i
1
_
. .
period t + prots
, s.t.
y
t+
= Ay
t+1
+Bd
t+
, for = 1, . . . , .
Note: obj fxn dierent in each period t: usual problem is
max
n
x
(i)
t
, iL
j
o

t=0

t=0

iL
j

i
t
_
y
t
, y
t+1
, . . . , y
t+T
i
1
_
.
FOC for x
(i)
t
contains derivative (say)

i
t1
x
(i)
t
: in choosing period-t prodn, rec-
ognize that it aects period-(t + 1) prots time-inconsistent.

Time-consistent Equilibrium production

Restrict attention to Markov strategies: Ay
t1
is the payo-relevant state
vector for period t (stocks of cars produced prior to period t which are still
actively traded in period t) =
Therefore consider production rules x
(i)
t
= g
i
(Ay
t1
), i L
j
, j N.
In Markov-perfect equilibrium, g
1
(.), . . . , g
L
(.) satisfy Bellman equation
V
j
(Ay
t1
) = max
x
(i)
t
, iL
j

iL
j

j
(y
t
, y
t+1
, . . . , y
t+T
i
1
) +V
j
(Ay
t
)
for all rms j, and the Markov decision rules
x
i
t
= g
i
(Ay
t1
), for all i L
j
.
Value function V
j
(Ay
t1
)=(*) (optimal prots from t onwards).

Linear Quadratic (LQ) Specication

Lecture notes: second set 11
We focus on linear equilibrium decision rule x
t+h
= GAy
t+h1
No trigger strategies (step functions)
Quadratic value function V (Ay
t1
) = y

t
A

SAy
t
.
Bellman equation can be rewritten in matrix notation
y

t1
A

SAy
t1
=
max
x
i
t
, iJ
_

iJ
_
T
i
1

z=0
y

t+z

z
R

z
(i)
y
t
__
y

t
C
j
y
t
+y

_
A

S
j
A

y
t
Recursive substitution yields
y

t1
A

SAy
t1
=
max
x
i
t
, iJ
y

t
__

iJ
T
i
1

z=0
(A

)
z
[(I +BG)

]
z

z
R
v
z
(i)
_
C
j
+
_
A

S
j
A

_
y
t
max
x
(i)
t
, iJ
y

t
Q
j
y
t
.

Deriving equilibrium production rules

Value iteration: solve for S and G by iterating over Bellman equation.
For each S, derive corresponding G via FOC of right-hand side:
B

j
_
Q
j
+Q

j
_
y
t
= B

j
_
Q
j
+Q

j
_
Ay
t1
+B

j
_
Q
j
+Q

j
_
Bd
t
= 0.
This system of FOCs corresponds to Eq. (3) in dynamic games handout.
Rearranging, we get:
d
t+h
= (WB)
1
(WA) y
t1
,
where W
j
B

j
_
Q
j
+Q

j
_
for each rm j and W [W
1
, . . . , W
N
]

.
Basis for estimating supply side of model.
Lecture notes: second set 12

Estimation
is not identied, set to constant.
To generate estimating equations, introduce shocks to rms marginal costs:
C(x
(i)
t
) = x
(i)
t
(c
i
+
it
) .
Assumptions:
t
[
1t
, . . . ,
Lt
]

has zero-mean, i.i.d. across t. The vector

t
is known
to all rms when they make their period t choices (no asymmetric information).
From certainty equivalence properties of linear-quadratic model, optimal rm strate-
gies are the same as in deterministic model, but with an additive shock:
d
t
= GAy
t1
+w
t
, where E(w
t
) = 0.
where E(w
t
) = 0 and independent of y
t1
.
With the cost shocks, then, production (and also prices) will be random over time.
However, due to independence of shocks across time, the innovations in prices will
have mean zero at time t:
p
t+1
= E
t
p
t+1
+
t+1
where E
t

t+1
= 0.
This implies that the realized demand function residual will also have mean zero,
conditional on formation known at time t:
0 = E
_
_
p
(i)
t

_

(i)

(i)+1
_
_
_
1
1
M
(i)

r=1
x
r
t
_
_
p
((i))
t+1
p
(i)+1
t
+p
((i)+1)
t+1

t
_
_
= E
_
_
_
1 L
1
_
_
p
(i)
t
p
(i)+1
t
_

(i)

(i)+1
_
_
_
1
1
M
(i)

r=1
x
r
t
_
_

t
_
_
These form the basis for the moment conditions which we use for estimation, which
we denote
T
().
Lecture notes: second set 13

GMM Estimation
Source of identication: co-movements time series of prices and production.

GMM estimator argmin

T
()

1
T

T
().
Nested GMM procedure: for each value of parameters , solve LQ dynamic
programming problem for coecients G() of optimal production rules. LQ
dynamic programming problem is very easy and quick to solve.
Results
Lecture notes: second set 14
References
Aguirregabiria, V., and P. Mira (2007): Sequential Estimation of Dynamic Discrete
Games, Econometrica.
Bajari, P., L. Benkard, and J. Levin (2008): Estimating Dynamic Models of Imper-
fect Competition, Econometrica.
Collard-Wexler, A. (2006): Demand Fluctuations and Plant Turnover in the Ready-
to-Mix Concrete Industry, manuscript, New York University.
Dunne, T., S. Klimer, M. Roberts, and D. Xu (2006): Entry and Exit in Geographic
Markets, manuscript, Penn State University.
Esteban, S., and M. Shum (2007): Durable Goods Oligopoly with Secondary Markets:
the Case of Automobiles, Forthcoming, RAND Journal of Economics.
Imai, S., N. Jain, and A. Ching (2005): Bayesian Estimation of Dynamic Discrete
Choice Models, Queens University, mimeo.
Keane, M., and K. Wolpin (1994): The Solution and Estimation of Discrete Choice
Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence,
Review of Economics and Statistics, 76, 648672.
Pakes, A. (2008): Theory and Empirical Work in Imperfectly Competitive Markets,
Fisher-Schultz Lecture at 2005 Econometric Society World Congress (London, England).
Pakes, A., and P. McGuire (2001): Stochastic Algorithms, Symmetric Markov Perfect
Equilibrium, and the Curse of Dimensionality, Econometrica, 69, 12611282.
Pakes, A., M. Ostrovsky, and S. Berry (2007): Simple Estimators for the Parameters
of Discrete Dynamic Games (with Entry/Exit Examples), RAND Journal of Economics.
Pesendorfer, M., and P. Schmidt-Dengler (2003): Identication and Estimation of
Dynamic Games, NBER Working Paper, #9726.
Ryan, S. (2006): The Costs of Environmental Regulation in a Concentrated Industry,
manuscript, MIT.