Economics 703: Microeconomics II

Modelling Strategic Behavior1

George J. Mailath
Department of Economics
University of Pennsylvania

January 18, 2016

1
Copyright January 18, 2016 by George J. Mailath.
Contents

Contents i

1 Normal and Extensive Form Games 1

1.1 Normal Form Games . . . . . . . . . . . . . . . . . . . . . 1
1.2 Iterated Deletion of Dominated Strategies . . . . . . . . 8
1.3 Extensive Form Games . . . . . . . . . . . . . . . . . . . . 10
1.3.1 The Reduced Normal Form . . . . . . . . . . . . . 12
1.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 A First Look at Equilibrium 17

2.1 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . 17
2.1.1 Why Study Nash Equilibrium? . . . . . . . . . . . . 22
2.2 Credible Threats and Backward Induction . . . . . . . . 22
2.2.1 Backward Induction and Iterated Weak Domi-
nance . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 Subgame Perfection . . . . . . . . . . . . . . . . . . . . . . 28
2.4 Mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4.1 Mixed Strategies and Security Levels . . . . . . . 32
2.4.2 Domination and Optimality . . . . . . . . . . . . . 34
2.4.3 Equilibrium in Mixed Strategies . . . . . . . . . . . 37
2.4.4 Behavior Strategies . . . . . . . . . . . . . . . . . . 39
2.5 Dealing with Multiplicity . . . . . . . . . . . . . . . . . . . 41
2.5.1 I: Refinements . . . . . . . . . . . . . . . . . . . . . 41
2.5.2 II: Selection . . . . . . . . . . . . . . . . . . . . . . . 42
2.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3 Games with Nature 53

3.1 An Introductory Example . . . . . . . . . . . . . . . . . . . 53

i
ii CONTENTS

3.2 Purification . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3 Auctions and Related Games . . . . . . . . . . . . . . . . 57
3.4 Games of Incomplete Information . . . . . . . . . . . . . 70
3.5 Higher Order Beliefs and Global Games . . . . . . . . . . 74
3.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

4 Nash Equilibrium 85
4.1 Existence of Nash Equilibria . . . . . . . . . . . . . . . . . 85
4.2 Foundations for Nash Equilibrium . . . . . . . . . . . . . 92
4.2.1 Boundedly Rational Learning . . . . . . . . . . . . 93
4.2.2 Social Learning (Evolution) . . . . . . . . . . . . . . 94
4.2.3 Individual learning . . . . . . . . . . . . . . . . . . 100
4.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5 Dynamic Games 113

5.1 Sequential Rationality . . . . . . . . . . . . . . . . . . . . . 113
5.2 Perfect Bayesian Equilibrium . . . . . . . . . . . . . . . . 119
5.3 Sequential Equilibrium . . . . . . . . . . . . . . . . . . . . 125
5.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

6 Signaling 135
6.1 General Theory . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2 Job Market Signaling . . . . . . . . . . . . . . . . . . . . . 138
6.2.1 Full Information . . . . . . . . . . . . . . . . . . . . 139
6.2.2 Incomplete Information . . . . . . . . . . . . . . . 139
6.2.3 Refining to Separation . . . . . . . . . . . . . . . . 144
6.2.4 Continuum of Types . . . . . . . . . . . . . . . . . 145
6.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

7 Repeated Games 155

7.1 Basic Structure . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.2 Modeling Competitive Agents . . . . . . . . . . . . . . . . 166
7.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.3.1 Efficiency Wages I . . . . . . . . . . . . . . . . . . . 172
7.3.2 Collusion Under Demand Uncertainty . . . . . . . 176
7.4 Enforceability and Decomposability . . . . . . . . . . . . 178
7.5 Imperfect Public Monitoring . . . . . . . . . . . . . . . . . 183
7.5.1 Efficiency Wages II . . . . . . . . . . . . . . . . . . 183
January 18, 2016 iii

7.5.2 Basic Structure . . . . . . . . . . . . . . . . . . . . . 185

7.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

8 Topics in Dynamic Games 201

8.1 Dynamic Games and Markov Perfect Equilibria . . . . . 201
8.2 Coase Conjecture . . . . . . . . . . . . . . . . . . . . . . . 206
8.2.1 One and Two Period Example . . . . . . . . . . . . 207
8.2.2 Infinite Horizon . . . . . . . . . . . . . . . . . . . . 208
8.3 Reputations . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
8.3.1 Infinite Horizon . . . . . . . . . . . . . . . . . . . . 212
8.3.2 Infinite Horizon with Behavioral Types . . . . . . 215
8.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

9 Bargaining 223
9.1 Axiomatic Nash Bargaining . . . . . . . . . . . . . . . . . 223
9.1.1 The Axioms . . . . . . . . . . . . . . . . . . . . . . . 223
9.1.2 Nashs Theorem . . . . . . . . . . . . . . . . . . . . 224
9.2 Rubinstein Bargaining . . . . . . . . . . . . . . . . . . . . . 225
9.2.1 The Stationary Equilibrium . . . . . . . . . . . . . 226
9.2.2 All Equilibria . . . . . . . . . . . . . . . . . . . . . . 228
9.2.3 Impatience . . . . . . . . . . . . . . . . . . . . . . . 230
9.3 Outside Options . . . . . . . . . . . . . . . . . . . . . . . . 230
9.3.1 Version I . . . . . . . . . . . . . . . . . . . . . . . . . 231
9.3.2 Version II . . . . . . . . . . . . . . . . . . . . . . . . 233
9.4 Exogenous Risk of Breakdown . . . . . . . . . . . . . . . 236
9.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

10 Appendices 243
10.1 Proof of Theorem 2.4.1 . . . . . . . . . . . . . . . . . . . . 243
10.2 Trembling Hand Perfection . . . . . . . . . . . . . . . . . 246
10.2.1 Existence and Characterization . . . . . . . . . . . 246
10.2.2 Extensive form trembling hand perfection . . . . 249
10.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

References 251

Index 257
iv CONTENTS
Chapter 1

Normal and Extensive Form

Games1

1.1 Normal Form Games

Example 1.1.1 (Prisoners Dilemma).

II
Confess Dont confess
I Confess 6, 6 0, 9
Dont confess 9, 0 1, 1

Often interpreted as a partnership game: effort E produces an

output of 6 at a cost of 4, with output shared equally, and S denotes
shirking.

E S
E 2, 2 1, 3
S 3, 1 0, 0

1
Copyright January 18, 2016 by George J. Mailath

1
2 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

Definition 1.1.1. An n-player normal (or strategic) form game G is

an n-tuple
{(S1 , U1 ), . . . , (Sn , Un )}, where for each i,
Si is a nonempty set, called is strategy space, and
Qn
Ui : k=1 Sk R is called is payoff function.
Equivalently,
Qn a normal form game is simply a vector-valued function
u : i=1 Si Rn .
Qn
Notation: S k=1 Sk ,
s (s1 , . . . , sn ) S, Q
si (s1 , . . . , si1 , si+1 , . . . , sn ) Si ki Sk .
(si0 , si ) (s1 , . . . , si1 , si0 , si+1 , . . . , sn ) S.
Example 1.1.2 (Sealed bid second price auction). 2 bidders, bi = is
bid, vi = is reservation price (willingness to pay).
Then, n = 2, Si = R+ , and


v bj , if bi > bj ,

i

1
Ui (b1 , b2 ) = (vi bj ), if bi = bj ,

2


0, if bi < bj .

Example 1.1.3 (Sealed bid first price auction). 2 bidders, bi = is

bid, vi = is reservation price (willingness to pay).
Then, n = 2, Si = R+ , and


vi bi ,
if bi > bj ,
1
Ui (b1 , b2 ) = 2 (vi bi ), if bi = bj ,


0, if bi < bj .

Example 1.1.4 (Cournot duopoly). Perfect substitutes, so that mar-

ket clearing price is given by P (Q) = max{a Q, 0}, Q = q1 + q2 ,
C(qi ) = cqi , 0 < c < a, and n = 2
Quantity competition: Si = R+ , and Ui (q1 , q2 ) = (P (q1 + q2 )
c)qi .
 January 18, 2016 3

2
(yxx) (yxy) (yzx) (yzy) (zxx) (zxy) (zzx) (zzy)
x 0,0 0,0 0,0 0,0 1,2 1,2 1,2 1,2
1 y 0,0 0,0 2,1 2,1 0,0 0,0 2,1 2,1
z 1,2 2,1 1,2 2,1 1,2 2,1 1,2 2,1

Figure 1.1.1: The normal form of the voting by veto game in Example
1.1.6.

Example 1.1.5 (Bertrand duopoly). Economic environment is as for

example 1.1.4, but price competition. Since perfect substitutes,
lowest pricing firm gets the whole market (with the market split
in the event of a tie). We again have Si = R+ , but now


(p1 c) max{a p1 , 0}, if p1 < p2 ,

max{(ap1 ),0}
U1 (p1 , p2 ) = (p1 c) , if p1 = p2 ,

2
0, if p1 > p2 .

Example 1.1.6 (Voting by veto). Three outcomes: x, y, and z. Player

1 first vetoes an outcome, and then player 2 vetoes one of the re-
maining outcomes. The non-vetoed outcome results. Suppose 1
ranks outcomes: x  y  z (i.e., u1 (x) = 2, u1 (y) = 1, u1 (z) = 0),
and 2 ranks outcomes as: y  x  z (i.e., u2 (x) = 1, u2 (y) =
2, u2 (z) = 0).
1s strategy is an uncontingent veto, so S1 = {x, y, z}.
2s strategy is a contingent veto, so S2 = {(abc) : a {y, z}, b
{x, z}, c {x, y}}.
The normal form is given in Figure 1.1.1.

4 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

Definition 1.1.2. si0 strictly dominates si00 if si Si ,

Ui (si0 , si ) > Ui (si00 , si ).

si is a strictly dominant strategy if si strictly dominates every

strategy si00 si , si00 Si .
If i has a strictly dominant strategy, then
( )
arg max Ui (si, , si ) = si : Ui (si, , si ) = max
0
Ui (si,0 , si )
si si

is a singleton and is independent of si .

Remark 1.1.1. The definition per se of a normal form game (or
strict dominance for that matter) makes no assumption about the
knowledge that players have about the game. We will, however,
typically assume (at least) that players know the strategy spaces,
and their own payoffs as a function of strategy profiles. However,
as the large literature on evolutionary game theory in biology sug-
gests (see also Section 4.2), this is not necessary.
The assertion that players will not play a strictly dominated
strategy is compelling when players know the strategy spaces and
their own payoffs. But (again, see Section 4.2), this is not necessary
for the plausibility of the assertion.

Definition 1.1.3. si0 (weakly) dominates si00 if si Si ,

Ui (si0 , si ) Ui (si00 , si ),
0
and si Si ,
Ui (si0 , si
0
) > Ui (si00 , si
0
).
A strategy is said to be strictly or weakly undominated if it is not
strictly or weakly dominated by some other strategy. If the adjec-
tive is omitted from dominated (or undominated), weak is typically
meant (but not always, unfortunately). A strategy is weakly domi-
nant if it weakly dominates every other strategy.
January 18, 2016 5

Lemma 1.1.1. If a weakly dominant strategy exists, it is unique.

Proof. Suppose si is a weakly dominant strategy. Then for all si0
Si , there exists si Si , such that

Ui (si , si ) > Ui (si0 , si ).

But this implies that si0 cannot weakly dominate si , and so si is the
only weakly dominant strategy.
Remark 1.1.2 (Warning). There is also a notion of dominant strat-
egy:
Definition 1.1.4. si0 is a dominant strategy if si00 Si , si Si ,

Ui(si0 , si ) Ui (si00 , si ).

If si0 is a dominant strategy for i, then si0 arg maxsi Ui (si, , si ),

for all si ; but arg maxsi Ui (si, , si ) need not be a singleton and it
need not be independent of si [example?]. If i has only one dom-
inant strategy, then that strategy weakly dominates every other
strategy, and so is weakly dominant.
Dominant strategies have played an important role in mecha-
nism design and implementation (see Remark 1.1.3), but not other-
wise (since a dominant strategywhen it existswill typically weakly
dominate every other strategy, as in Example 1.1.7).

Remark 1.1.3 (Strategic behavior is ubiquitous). Consider a society

consisting of a finite number n of members and a finite set of out-
comes X. Suppose each member of society has a strict preference
ordering of X, and let be the set of all possible strict orderings
on X. A profile (1 , . . . , n ) n describes a particular society (a
preference ordering for each member).
A social choice rule or function is a mapping f : n X. For
any i , let t(i ) be the top-ranked outcome in X under i . A
social choice rule f is dictatorial if there is some i such that for all
(1 , . . . , n ) n , f (1 , . . . , n ) = t(i ). A social choice rule f is
unanimous if f (1 , . . . , n ) = x whenever x = t(j ) for all j.
6 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

The direct mechanism is the normal form game in which all

members of society simultaneously announce a preference order-
ing and the outcome is determined by the social choice rule as a
function of the announced preferences.

Theorem 1.1.1 (Gibbard-Satterthwaite). Suppose |X| 3 and f is

unanimous. Then, announcing truthfully in the direct mechanism
is a dominant strategy for all preference profiles if, and only if, the
social choice rule is dictatorial.

A social choice rule is said to be strategy proof if announcing

truthfully in the direct mechanism is a dominant strategy for all
preference profiles. It is trivial that for any dictatorial social choice
rule, it is a dominant strategy to always truthfully report in the
direct mechanism. The surprising result is the converse.

Example 1.1.7 (Continuation of example 1.1.2). In the 2nd price auc-

tion, each player has a weakly dominant strategy, given by b1 = v1 .
Sufficient to show this for 1. First argue that bidding v1 is a best
response for 1, no matter what bid 2 makes (i.e., it is a dominant
strategy). Recall that payoffs are given by


v1 b2 ,
if b1 > b2 ,
1
U1 (b1 , b2 ) = 2 (v1 b2 ), if b2 = b1 ,


0, if b1 < b2 .

Two cases:

1. b2 < v1 : Then U1 (v1 , b2 ) = v1 b2 U1 (b1 , b2 ).

2. b2 v1 : Then, U1 (v1 , b2 ) = 0 U1 (b1 , b2 ).

Thus, bidding v1 is optimal.

Bidding v1 also weakly dominates every other bid (and so v1
is weakly dominant). Suppose b1 < v1 and b1 < b2 < v1 . Then
U1 (b1 , b2 ) = 0 < v1 b2 = U1 (v1 , b2 ). If b1 > v1 and b1 > b2 > v1 ,
then U1 (b1 , b2 ) = v1 b2 < 0 = U1 (v1 , b2 ).
January 18, 2016 7

Example 1.1.8 (Provision of public goods). n people. Agent i values

public good at ri , total cost of public good is C.
Suppose costs are shared uniformly and utility is linear, so agent
1
is net utility is vi ri n C. P
P Efficient provision: Public good provided iff 0 vi , i.e., C
ri . P
Eliciting preferences: Agents announce vi and provide if vi
0? Gives incentive to overstate if vi > 0 and understate if vi < 0.
Groves-Clarke
P mechanism: if public good provided, pay agent i
amount ji vj (tax if negative) .

P P
vi + ji vj , if vi + ji vj 0,
Agent is payoff(vi ) = P
0, if vi + ji vj < 0.
P
Dominant strategy to announce vi = vi : If vi + ji v Pj > 0, an-
nouncing vi = vi ensures good is provided, while if vi + ji vj < 0,
announcing vi = vi ensures good is not provided. Moreover, con-
ditional on provision, announcement does not affect payoffnote
similarity to second price auction.
No payments if no provision, but payments large if provision:
P P 
Total payments to agents when provision = i ji vj = (n
P
1) i vi . Taxing agent i by an amount independent
P of is behavior
has no impact, so tax i the amount max{ ji vj , 0}. Result is
P P

vi , if j vj 0 and ji vj 0,



P P P

vi + ji vj , if j vj 0 and ji vj < 0,
payoff to i = P P P

ji vj , if j vj < 0 and ji vj 0,



P P

0, if j vj < 0 and ji vj < 0.

This is the pivotal mechanism. Note that i only pays a tax if i

changes
P social decision. Moreover, total taxes are no P larger than
i max{ v i , 0} if the good is provided and no larger than i max{vi , 0}
if the good is not provided.
8 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

Example 1.1.9 (Continuation of example 1.1.4, Cournot). There are

no weakly dominating quantities in the Cournot duopoly: Suppose
q2 < a. Then arg maxq1 U1 (q1 , q2 ) = arg max(a c q1 q2 )q1 . First
order condition implies a c 2q1 q2 = 0 or
a c q2
q1 (q2 ) = .
2
Since arg max U1 is unique and a nontrivial function of q2 , there is
no weakly dominating quantity.

1.2 Iterated Deletion of Dominated Strate-

gies
Example 1.2.1.

L M R
T 1, 0 1, 2 0, 1
B 0, 3 0, 1 2, 0

Delete R and then B, and then L to get (T , M).

Example 1.2.2 (Continuation of example 1.1.6). Apply iterative dele-

tion of weakly dominated strategies to veto game. After one round
of deletions,

2
(z, z, x)
1 y 2,1
z 1,2
January 18, 2016 9

and so 1 vetoes y, and not z!

Remark 1.2.1. For finite games, the order of removal of strictly

dominated strategies is irrelevant (see Problem 1.4.4(a)). This is not
true for weakly dominated strategies:
L M R
T 1, 1 1, 1 0, 0
B 1, 1 0, 0 1, 1
Both TL and BL can be obtained as the singleton profile that remains
from the iterative deletion of weakly dominated strategies. In ad-
dition, {T L, T M} results from a different sequence, and {BL, BR}
from yet another sequence. (The order does not matter for games
like the veto game of example 1.1.6, see Theorem 2.2.2.)
Similarly, the order of elimination may matter for infinite games
(see Problem 1.4.4(b)).
Because of this, the procedure of the iterative deletion of weakly
(or strictly) dominated strategies is often understood to require that
at each stage, all weakly (or strictly) dominated strategies be deleted.
We will follow that understanding in this class (unless explicitly
stated otherwise). With that understanding, the iterated deletion of
weakly dominated strategies in this example leads to {T L, BL}.

Remark 1.2.2. The plausibility of the iterated deletion of domi-

nated strategies requires something like players knowing the struc-
ture of the game (including that other players know the game), not
just their own payoffs. For example in Example 1.2.1, in order for
the column player to delete L at the third round, then the column
player needs to know that the row player will not play B, which re-
quires the column player to know that the row player knows that
the column player will not play R.
As illustrated in Section 4.2, this kind of iterated knowledge is
not necessary for the plausibility of the procedure (though in many
contexts it provides the most plausible foundation).

10 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

1.3 Extensive Form Games

Game trees look like decision trees. Role of definition is to make
clear who does what, when, knowing what.

Definition 1.3.1. A finite extensive form game consists of :

1. A set of players {1, . . . , n} and nature, denoted player 0.

2. A game tree (T , ), where (T , ) is an arborescence: T is a

finite set of nodes and is a binary relation on T denoting
precedence satisfying

(a) is asymmetric (t t 0 t 0 6 t),2

(b) transitive (t, t 0 , t 00 T , t t 0 , t 0 t 00 t t 00 ),3
(c) if t t 00 and t 0 t 00 then either t t 0 or t 0 t, and finally,
(d) there is a unique initial node, t0 T , i.e., {t0 } = {t T :
t 0 T , t 0 t}.

Let p(t) T denote the immediate predecessor of t, i.e., p(t)

t and t 0 such that p(t) t 0 t. Every noninitial node
has a unique immediate predecessor.4 The path to a node t
is the sequence t0 = p k (t), p k1 (t), . . . , p(t), t, where p ` (t) =
p(p `1 (t)) for ` 2.5 For every noninitial node t, there is a
unique path from the initial node to t.
Define s(t) := {t 0 T : t t 0 and t 00 , t t 00 t 0 } = {t 0 T :
p(t 0 ) = t}, the set of immediate successors of t.
Let Z {t T : t 0 T , t t 0 }, Z is the set of terminal nodes.

3. Assignment of players to nodes, : T \Z {0, 1, . . . n}. Define

Tj 1 (j) = {t T \Z : (t) = j}, j {0, 1, . . . , n}.
2
Note that this implies that is irreflexive: t 6 t for all t T .
3
A binary relation satisfying 2(a) and 2(b) is called a strict partial order.
4
See Problem 1.4.5.
5
Note that the equality t0 = p k (t) defines k, that is, the path has k steps from
t to the initial node.
January 18, 2016 11

4. Actions: Actions lead to (label) immediate successors, i.e., there

is a set A and a mapping

: T \{t0 } A,

such that (t 0 ) (t 00 ) for all t 0 , t 00 s(t). Define A(t)

(s(t)), the set of actions available at t T \Z.

5. Information sets: Hi is a partition of Ti for all i 0 (Hi is a

collection of subsets of Ti such that (i) t Ti , h Hi , t h,
and (ii) h, h0 Hi , h h0 h h0 = ). Assume t, t 0 h,

(a) t 6 t 0 , t 0 6 t,
(b) A(t) = A(t 0 ) A(h), and
(c) perfect recall (every player knows whatever he knew pre-
viously, including own previous actions).6

6. Payoffs, ui : Z R.

7. Prob dsn for nature, : T0 tT0 (A(t)) such that (t)

(A(t)).

Definition 1.3.2. An extensive form strategy for player i is a func-

tion

si : Hi h A(h) such that si (h) A(h), h Hi . (1.3.1)

The set of player is strategies is is strategy space, denoted Si .

Strategy profile is (s1 , . . . , sn ).

Definition 1.3.3. Suppose there are no moves of nature.

The outcome path is the sequence of nodes reached by strategy
profile, or equivalently, the sequence of specified actions.
The outcome is the unique terminal node reached by the strategy
profile s, denoted z(s). In this case, the normal form representation
6
Denote the information set containing t Ti by h(t) Hi . The formal
requirement is:
If t, t 0 , t 00 Ti , t t 0 , and t 00 h(t 0 ), then t h(t) with t t 00 . More-
over, defining ` and m by t = p ` (t 0 ) and t = p m (t 00 ), we have (p `1 (t 0 )) =
(p m1 (t 00 )).
12 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

is given by {(S1 , U1 ), . . . , (Sn , Un )}, where Si is the set of is extensive

form strategies, and
Ui (s) = ui (z(s)).
Definition 1.3.4. If there are moves of nature, the outcome is the
implied probability distribution over terminal nodes, denoted s
(Z). In this case, in the normal form representation given by
{(S1 , U1 ), . . . , (Sn , Un )}, where Si is the set of is extensive form strate-
gies, we have X
Ui (s) = s (z)ui (z).
z

For an example, see Example 3.1.1.

Example 1.3.1. The extensive form for example 1.1.6 is described
as follows: The game starts at the node t0 , owned by player 1,
((t0 ) = 1), and t0 has three immediate successors, t1 , t2 , t3 reached
by vetoing x, y, and z (so that (t1 ) = x, and so on). These im-
mediate successors are all owned by player 2 (i.e., (tj ) = 2 for
j = 1, 2, 3). At each of these nodes, player 2 vetoes one of the re-
maining outcomes, which each veto leading to a distinct node. See
Figure 1.3.1 for a graphical representation.
The result of the iterative deletion of weakly dominated strate-
gies is (y, zzx), implying the outcome (terminal node) t7 .
Note that this outcome also results from the profiles (y, yzx)
and (y, zzy), and (y, yzy).

Definition 1.3.5. A game has perfect information if all information

sets are singletons.
Example 1.3.2 (Simultaneous moves). The extensive form for the
prisoners dilemma is illustrated in Figure 1.3.2.

1.3.1 The Reduced Normal Form

Example 1.3.3. Consider the extensive form in Figure 1.3.3. Note
that payoffs have not been specified, just the terminal nodes. The
normal form is (where u(z) = (u1 (z), u2 (z))):
January 18, 2016 13

1
t0

x
y z

2 2
2
t1 t2 t3
y z x z x y

t4 t5 t6 t7 t8 t9
0 1 0 2 1 2
0 2 0 1 2 1

Figure 1.3.1: The extensive form for Example 1.1.6. An arrow connects a
node with its immediate predecessor, and is labelled by the
action associated with that node.

I
t0
S E

II
t1 t2
S E S E

t3 t4 t5 t6
0 3 1 2
0 1 3 2

Figure 1.3.2: The extensive form for the prisoners dilemma. Player IIs
information set is indicated by the dashed line.
14 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

Go II Go I Go1 II Go1 z5
I

Stop Stop Stop1 Stop1

z1 z2 z3 z4

Figure 1.3.3: A simple extensive form game.

Stop, Stop1 Stop, Go1 Go, Stop1 Go, Go1

Stop,Stop1 u(z1 ) u(z1 ) u(z1 ) u(z1 )
Stop,Go1 u(z1 ) u(z1 ) u(z1 ) u(z1 )
Go,Stop1 u(z2 ) u(z2 ) u(z3 ) u(z3 )
Go,Go1 u(z2 ) u(z2 ) u(z4 ) u(z5 )

Is strategies of Stop,Stop1 and Stop,Go1 are equivalent, as are

IIs strategies of Stop,Stop1 and Stop,Go1 .

Definition 1.3.6. Two strategies si , si0 Si are strategically equiva-

lent if uj (si , si ) = uj (si0 , si ) for all si Si and all j.
In the (pure strategy) reduced normal form of a game, every set
of strategically equivalent strategies is replaced by a single repre-
sentative.
Example 1.3.3 (continued). The reduced normal form is

Stop Go, Stop1 Go, Go1

Stop u(z1 ) u(z1 ) u(z1 )
Go,Stop1 u(z2 ) u(z3 ) u(z3 )
Go,Go1 u(z2 ) u(z4 ) u(z5 )

The strategy Stop for player I, for example, in the reduced normal
form should be interpreted as the equivalence class of extensive
form strategies {Stop,Stop1 , Stop,Go1 }, where the equivalence rela-
tion is given by strategic equivalence.
January 18, 2016 15

Notice that this reduction occurs for any specification of payoffs

at the terminal nodes.

The importance of the reduced normal form arises from the re-
duction illustrated in Example 1.3.3: Whenever a players strategy
specifies an action that precludes another information set owned
by the same player, that strategy is strategically equivalent to other
strategies that only differ at such information sets. As for Exam-
ple 1.3.3, this is true for arbitrary specifications of payoffs at the
terminal nodes.7
When describing the normal form representation of an extensive
form, it is common to (and we will typically) use the reduced normal
form.
An extensive form strategy always has the form given by (1.3.1),
while a normal form strategy may represent an equivalence class
of extensive form strategies (and a reduced normal form strategy
always does).

1.4 Problems
1.4.1. Describe a social choice rule that is unanimous, nondictatorial, and
strategy proof when there are two alternatives.

1.4.2. Consider the following social choice rule over the set X = {x, y, z}.
There is an exogenously specified order x  y  z, and define
S(X 0 ) {a X 0 : a  a0 a0 X 0 \ {a}}. Then,

f () = S({a X : a = t(i ) for some i}).

Prove that f is unanimous and nondictatorial, but not strategy proof.

7
When developing intuition about the relationship between the normal and
extensive form, it is often helpful to assume that extensive form payoffs have no
ties, i.e., for all players i and all pairs of terminal nodes, z, z0 Z, ui (z) ui (z0 ).
Under that restriction, changing payoffs at terminal nodes does not change the
set of reduced normal form strategies for any player.
However, if for example, ui (z) = ui (z0 ) for all players i and all pairs of termi-
nal nodes, z, z0 Z, then the reduced normal form consists of one strategy for
each player. Changing payoffs now does change the set of reduced normal form
strategies.
 16 CHAPTER 1. NORMAL AND EXTENSIVE FORM GAMES

1.4.3. Consider the Cournot duopoly example (Example 1.1.4).

(a) Characterize the set of strategies that survive iterated deletion

of strictly dominated strategies.
(b) Formulate the game when there are n 3 firms and identify
the set of strategies surviving iterated deletion of strictly dom-
inated strategies.

1.4.4. (a) Prove that the order of deletion does not matter for the process
of iterated deletion of strictly dominated strategies in a finite
game (Remark 1.2.1 shows that strictly cannot be replaced by
weakly).
(b) Show that the order of deletion matters for the process of iter-
ated deletion of strictly dominated strategies for the following
infinite game: S1 = S2 = [0, 1] and payoffs


si , if si < 1,

ui (s1 , s2 ) = 0, if si = 1, sj < 1,


s , if s = s = 1.
i i j

1.4.5. Suppose (T , ) is an arborescence (recall Definition 1.3.1). Prove

that every noninital node has a unique immediate predecessor.

1.4.6. Define the follows relation on information sets by: h0 h if

there exists x h and x 0 h0 such that x 0 x.

(a) Prove that each player is information sets Hi are strictly par-
tially ordered by the follows relation (recall footnote 3).
(b) Perfect recall implies that the set of is information sets satisfy
Property 2(c) of Definition 1.3.1, i.e., for all h, h0 , h00 Hi , if
h h00 and h0 h00 then either h h0 or h0 h. Give an
intuitive argument.
(c) Give an example showing that the set of all information sets is
not similarly strictly partially ordered by .
(d) Prove that if h0 h for h, h0 Hi , then for all x h, there
exists x 0 h0 such that x 0 x. (In other words, an individual
players information is refined through play in the game. Prove
should really be in quotes, since this is trivial.)
Chapter 2

A First Look at Equilibrium1

2.1 Nash Equilibrium

Example 2.1.1 (Battle of the sexes).
Sheila
Opera Ballet
Bruce Opera 2, 1 0, 0
Ballet 0, 0 1, 2

Definition 2.1.1. s S is a Nash equilibrium of G = {(S1 , u1 ), . . . , (Sn , un )}

if for all i and for all si Si ,

ui (si , si

) ui (si , si ).

Example 2.1.2. Consider the simple extensive form in Figure 2.1.1.

The strategy spaces are S1 = {L, R}, S2 = {``0 , `r 0 , r `0 , r r 0 }.
Payoffs are Uj (s1 , s2 ) = uj (z), where z is terminal node reached
by (s1 , s2 ).
The normal form is given in Figure 2.1.2.
Two Nash equilibria: (L, ``0 ) and (R, `r 0 ). Though `r 0 is a best
reply to L, (L, `r 0 ) is not a Nash equilibrium.
Note that the equilibria are strategy profiles, not outcomes. The
outcome path for (L, ``0 ) is L`, while the outcome path for (R, `r 0 )
1
Copyright January 18, 2016 by George J. Mailath

17
18 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

I
t0
L R

II II
t1 t2
` r `0 r0

t3 t4 t5 t6
2 4 1 3
3 2 0 1

Figure 2.1.1: A simple extensive form.

II
``0 `r 0 r `0 rr0
I L 2,3 2,3 4,2 4,2
R 1,0 3,1 1,0 3,1

Figure 2.1.2: The normal form of the extensive form in Figure 2.1.1.
January 18, 2016 19

is Rr 0 . In examples where the terminal nodes are not separately

labeled, it is common to also refer to the outcome path as simply
the outcomerecall that every outcome path reaches a unique ter-
minal node, and conversely, every terminal node is reached by a
unique sequence of actions (and moves of nature).
NOTE: (R, r r 0 ) is not a Nash eq, even though the outcome path
associated with it, Rr 0 , is a Nash outcome path.

(Y ) collection of all subsets of Y , the power set of Y {Y 0

Y }.
A function : X (Y )\{} is a correspondence from X to Y ,
sometimes written : X Y .
Note that (x) is simply a nonempty subset of Y . If f : X
Y is a function, then (x) = {f (x)} is a correspondence and a
singleton-valued correspondence can be naturally viewed as a func-
tion.

Definition 2.1.2. The best reply correspondence for player i is

i (si ) := arg max ui (si , si )

si Si

= {si Si : ui (si , si ) ui (si0 , si ), si0 Si }.

Note: without assumptions on Si and ui , i need not be well

defined. When it is well defined everywhere, i : Si Si .
If i (si ) is a singleton for all si , then i is is reaction function.

Remark 2.1.1. Defining : S S by

Y
(s) := i (si ) = 1 (s1 ) n (sn ),
i

we have that s is a Nash equilibrium if, and only if,

s (s ).

20 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

q2

ac

(q1 , q2 )

(a c)/2 1

(a c)/2 ac q1

Figure 2.1.3: The reaction (or best reply) functions for the Cournot game.

Example 2.1.3 (Continuation of example 1.1.9, Cournot). Recall that,

if q2 < a c, arg max ui (q1 , q2 ) is unique and given by

a c q2
q1 (q2 ) = .
2

More generally, is reaction function is

1
i (qj ) = max{ (a c qj ), 0}.
2

Nash eq (q1 , q2 ) solves

q1 =1 (q2 ),
q2 =2 (q1 ).
January 18, 2016 21

So (ignoring the boundary conditions for a second),

1
q1 = (a c q2 )
2
1 1
= (a c (a c q1 ))
2 2
1 1 q
= (a c) (a c) + 1
2 4 4
1 q1
= (a c) +
4 4
and so
1
q1 = (a c).
3
Thus,
1 1 1
q2 = (a c) (a c) = (a c).
2 6 3
Thus the boundary condition is not binding.
Price is given by
2 1 2
p = a q1 q2 = a (a c) = a + c > 0.
3 3 3
Note also that there is no equilibrium with zero prices.

Example 2.1.4 (Example 1.1.7 cont., sealed bid 2nd price auction).
Suppose v1 < v2 , and the valuations are commonly known. There
are many Nash equilibria: Bidding vi for each i is a Nash equilib-
rium (of course?). But so is any bidding profile (b1 , b2 ) satisfying
b1 < b2 , b1 v2 and v1 b2 (Why? Make sure you understand
why some inequalities are weak and some are strict). Are there any
other equilibria?

Example 2.1.5 (Example 1.1.3 cont., sealed bid 1st price auction).
Suppose v1 = v2 = v, and the valuations are commonly known.
The unique Nash equilibrium is for both bidders to bid bi = v. But
this eq is in weakly dominated strategies. But what if bids are in
pennies?
22 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

2.1.1 Why Study Nash Equilibrium?

Nash equilibrium is based on two principles:
1. each player is optimizing given beliefs/predictions about the
behavior of the other players; and
2. these beliefs/predictions are correct.
While optimization is not in principle troubling (it is true almost
by definition), the consistency of beliefs with the actual behavior is
a strong assumption. Where does this consistency come from?
Several arguments have been suggested:
1. preplay communication (but see Section 2.5.2),
2. self-fulfilling prophecy (if a theory did not specify a Nash equi-
libria, it would invalidate itself),
3. focal points (natural way to play),
4. introspection more generally (currently not viewed as persua-
sive),
5. learning (either individual or social), see Section 4.2, and
6. provides important discipline on modelling.

2.2 Credible Threats and Backward Induc-

tion
Example 2.2.1 (Entry deterrence). The entry game illustrated in Fig-
ure 2.2.1 has two Nash equilibria: (In, Accommodate) and (Out,
Fight). The latter violates backward induction.

Example 2.2.2 (The case of the reluctant kidnapper). Kidnapper has

two choices after receiving ransom: release or kill victim. After
release, victim has two choices: whether or not to reveal identity of
kidnapper. Payoffs are illustrated in Figure 2.2.2. Victim is killed in
only outcome satisfying backward induction.
January 18, 2016 23

Entrant

Out In

Incumbent
0
4 Fight Accommodate

1 1
1 2

Figure 2.2.1: An entry game.

Kidnapper

kill release

Victim
1
100 dont reveal reveal

10 5
1 2

Figure 2.2.2: The Reluctant Kidnapper

24 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

Go II Go I Go1 10
I 1, 000

Stop Stop Stop1

1 0 100
0 10 1

Figure 2.2.3: A short centipede game.

Definition 2.2.1. Fix a finite extensive form game of perfect infor-

mation. The backward induction solution is the strategy profile s
obtained as follows:
1. Since the game is finite, there is at least one last decision node,
t . Since the game has perfect information, the information set
containing t is the singleton {t }. With only a slight abuse
of notation, denote the information set by t . Player (t )s
action choice under s at t , s(t ) (t ), is an action reaching a
terminal node that maximizes (t )s payoff (over all terminal
nodes reachable from t ).2
2. With player (t )s action at t fixed, we now have a finite ex-
tensive form game with one less decision node. Now apply step
1 again. Continue till an action has been specified at every
decision node.
A backward induction outcome is the terminal node reached by
a backward induction solution.
Example 2.2.3 (Rosenthals centipede game). The perils of back-
ward induction are illustrated in Figure 2.2.3, with reduced normal
form given in Figure 2.2.4.
A longer (and more dramatic version of the centipede is given in
Figure 2.2.5.
2
This action is often unique, but need not to be. If it is not unique, each choice
leads to a distinct backward induction solution.
January 18, 2016 25

Stop Go
Stop 1, 0 1, 0
Go,Stop1 0, 10 100, 1
Go,Go1 0, 10 10, 1000

Figure 2.2.4: The reduced normal form for the short centipede in Figure
2.2.3

Go II Go I Go1 II Go1 I Go2 1, 000

I 100, 000
Stop1 Stop1 Stop2
Stop Stop
1 0 100 10 10, 000
0 10 1 1, 000 100

Figure 2.2.5: A long centipede game.

26 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

R1 II r I R2 10
I 1, 000

D1 d D2
15 20 9
0 10 1

Figure 2.2.6: Another cautionary example.

Backward induction solution in both cases is that both players

choose to stop the game at each decision point.

Example 2.2.4 (Another cautionary example). The backward solu-

tion for the game in Figure 2.2.6 is (D1 R2 , r ).
But if II is asked to make a move, how confident should she be
that I will play R2 after r ? After all, she already has evidence that
I is not following the backward induction solution (i.e., she already
has evidence that I may not be rational, or perhaps said better,
that her beliefs about Is payoffs may be incorrect). And 9 is close
to 10! But then playing d may be sensible. Of course, that action
seems to justify Is original irrational action.

Theorem 2.2.1. (Zermelo, Kuhn): A finite game of perfect informa-

tion has a pure strategy Nash equilibrium.

2.2.1 Backward Induction and Iterated Weak Domi-

nance
Generically in extensive form payoffs, the terminal node reached
by the backward induction solution agrees with the terminal node
reached by any strategy profile left after the iterated deletion of
dominated strategies. Given the distinction between extensive form
and reduced normal form strategies, there is of course no hope for
anything stronger.
January 18, 2016 27

L R
II
2
2 ` r

0 2
1 1

Figure 2.2.7: An example with ties. The backward induction solutions are
L`, Lr , and Rr . But R is weakly dominated for I.

Theorem 2.2.2. Suppose is a finite extensive form game of per-

fect information with no ties, i.e., for all players i and all pairs of
terminal nodes z and z0 , ui (z) ui (z0 ).3 The game has a unique
backward induction outcome. Let s be a strategy profile that re-
mains after some maximal sequence of iterated deletions of weakly
dominated strategies (i.e., the order of deletions is arbitrary and no
further deletions are possible). The outcome reached by s is the back-
ward induction outcome.

Example 2.2.5 (An example with ties). If the game has ties, then
iteration deletion of weakly dominated strategies can be stronger
than backward induction (see Figure 2.2.7).

3
This property is generic in extensive form payoffs in the following sense: Fix
an extensive game tree (i.e., everything except the assignment of payoffs to ter-
minal nodes z Z). The space of games (with that tree) is then the space of
all payoff assignments to the terminal nodes, Rn|Z| . Note that, viewed this way,
the space of games is a finite dimensional Euclidean space. The set of payoff
assigments that violate the property of no ties is a subset of a closed Lebesgue
measure zero subset of Rn|Z| .
28 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

I
L
R
2
1 II
1
T B

III
` r ` r

0 0 3 0
0 0 1 1
0 1 1 0

Figure 2.3.1: (L, T , r ) is Nash. Is it plausible?

2.3 Subgame Perfection

Example 2.3.1. In the game illustrated in Figure 2.3.1, the profile
(L, T , r ) is Nash. Is it plausible?

Define S(t) {t 0 T : t t 0 }.

Definition 2.3.1. The subset T t {t} S(t), of T , together with

payoffs, etc. appropriately restricted to T t , is a subgame if for all
information sets h,

h T t h T t.

The information set containing the initial node of a subgame is

necessarily a singleton (see Problem 2.6.6).

Definition 2.3.2. The strategy profile s is a subgame perfect equi-

librium if s prescribes a Nash equilibrium in every subgame.
January 18, 2016 29

Example 2.3.2 (augmented PD).

E S P
E 2, 2 1, 3 1, 1
S 3, 1 0, 0 1, 1
P 1, 1 1, 1 2, 2

Play game twice and add payoffs.

Nash strategy profile: E in first period, and S in second period as
long as opponent also cooperated in first period, and P if opponent
didnt exert effort in first period. Every first period action profile
describes an information set for each player. Player is strategy is
si1 = E,
(
S, if aj = E,
si2 (ai , aj ) =
P , if aj E.
Not subgame perfect: Every first period action profile induces a
subgame, on which SS must be played. But the profile prescribes
P S after ES, for example. Only subgame perfect equilibrium is al-
ways S.

Example 2.3.3 (A different repeated game). The stage game is

L C R
T 4, 6 0, 0 9, 0
.
M 0, 0 6, 4 0, 0
B 0, 0 0, 0 8, 8

(T , L) and (M, C) are both Nash eq of the stage game. The follow-
ing profile (s1 , s2 ) of the once-repeated game with payoffs added is
subgame perfect:
s11 = B,
(
M, if x = B,
s12 (x, y) =
T, if x B,
30 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

I
L
R
2
0 I

T B

II
` r ` r

1 4 0 5
1 0 0 1

Figure 2.3.2: A game with nice subgames.

s21 = R,
(
2 C, if x = B, and
s2 (x, y) =
L, if x B.

The outcome path induced by (s1 , s2 ) is (BR, MC).

These are strategies of the extensive form of the repeated game,
not of the reduced normal form. The reduced form strategies cor-
responding to (s1 , s2 ) are

s11 = B,
s12 (y) = M, for all y,
s21 = R,
(
C, if x = B, and
s22 (x) =
L, if x B.

Example 2.3.4. Consider the extensive form in Figure 2.3.2.

January 18, 2016 31

I
L

2
0 T B

II
` r ` r

1 4 0 5
1 0 0 1

Figure 2.3.3: An equivalent game with no nicesubgames.

The game has three Nash eq: (RB, r ), (LT , `), and (LB, `). Note
that (LT , `), and (LB, `) are distinct extensive form strategy pro-
files.
The only subgame perfect equilibrium is (RB, r ).
But, (L, `) also subgame perfect in the extensive form in Figure
2.3.3.
Both games have the same reduced normal form, given in Figure
2.3.4.

` r
L 2, 0 2, 0
T 1, 1 4, 0
B 0, 0 5, 1

Figure 2.3.4: The reduced form for the games in Figures 2.3.2 and 2.3.3.
32 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

Remark 2.3.1 (Equivalent representations?). A given strategic set-

ting has both a normal form and an extensive form representation.
Moreover, the extensive form apparently contains more informa-
tion (since it in particular contains information about dynamics and
information). For example, the application of weak domination to
rule out the (Stay out, Fight) equilibrium can be argued to be less
compelling than the backward induction (ex post) argument in the
extensive form: faced with the fait accompli of Enter, the incum-
bent must Accommodate. As Kreps and Wilson (1982b, p. 886)
write: analysis based on normal form representation inherently ig-
nore the role of anticipated actions off the equilibrium path...and in
the extreme yields Nash equilibria that are patently implausible.
But, backward induction and iterated deletion of weakly domi-
nated strategies lead to the same outcomes in finite games of per-
fect information (Theorem 2.2.2). Motivated by this and other con-
siderations, a classical argument holds that all extensive forms
with the same reduced normal form representation are strategically
equivalent (Kohlberg and Mertens, 1986, is a well known statement
of this position; see also Elmes and Reny, 1994). Such a view im-
plies that good extensive form solutions should not depend on
the extensive form in the way illustrated in Example 2.3.4. For more
on this issue, see van Damme (1984) and Mailath, Samuelson, and
Swinkels (1993, 1997).

2.4 Mixing
2.4.1 Mixed Strategies and Security Levels
Example 2.4.1 (Matching Pennies). A game with no Nash eq:

H T
H 1, 1 1, 1
T 1, 1 1, 1
January 18, 2016 33

The greatest payoff that player 1 can guarantee himself may ap-
pear to be 1 (the unfortunate result of player 2 correctly antici-
pating 1s choice).
But suppose that player 1 flips a fair coin so that player 2 cannot
anticipate 1s choice. Then, 1 should be able to do better.

Definition 2.4.1. Suppose {(S1 , u1 ), . . . , (Sn , un )} is an n-player nor-

mal form game. A mixed strategy for player i is a probability distri-
bution over Si , denoted i . Strategies in Si are called pure strategies.
A strategy i is completely mixed if i (si ) > 0 for all si Si .

In order for the set of mixed strategies to have a nice mathe-

matical structure (such as being metrizable or compact), we need
the set of pure strategies to also have a nice structure (often com-
plete separable metric, i.e., Polish). For our purposes here, it will
suffice to consider finite sets, or nice subsets of Rk . More gener-
ally, a mixed strategy is a probability measure over the set of pure
strategies. The set of probability measures over a set A is denoted
(A).
If Si is countable, think P of a mixed strategy i as a mapping
i : Si [0, 1] such that si Si i (si ) = 1.
Qn
Extend ui to j=1 (Sj ) by taking expected values, so that ui is
is expected payoff under randomization.
If Si is countable,
X X
ui (1 , . . . , n ) = ui (s1 , . . . , sn )1 (s1 ) n (sn ).
s1 S1 sn Sn

Writing X Y
ui (si , i ) = ui (si , si ) j (sj ),
si Si ji

we then have
X
ui (i , i ) = ui (si , i )i (si ).
si Si

In general, payoffs from a mixed strategy profile are calculated as

expected payoffs.
34 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

Definition 2.4.2. Player is security level (or payoff) is the greatest

payoff that i can guarantee himself:

v i = sup Q
inf ui (i , i ).
i (Si ) i ji (Sj )

If i achieves the sup, then i is a security strategy for i.

In matching pennies, each players security level is 0, guaranteed
1 1
by the security strategy 2 H + 2 T .

2.4.2 Domination and Optimality

Example 2.4.2. In the following game (payoffs are for the row player),
M is not dominated by any strategy (pure or mixed) and it is the
1 1
unique best reply to 2 L + 2 R:

L R
T 3 0
M 2 2
B 0 3

In the following game (again, payoffs are for row player), M is

not dominated by T or B, it is never a best reply, and it is strictly
1 1
dominated by 2 T + 2 B:

L R
T 5 0
M 2 2
B 0 5

Definition 2.4.3. The strategy si0 Si is strictly dominated by the

mixed strategy i (Si ) if

ui (i , si ) > ui (si0 , si ) si Si .
January 18, 2016 35

Henceforth, a strategy is strictly (or weakly) undominated if there

is no pure or mixed strategy that strictly (or weakly, respectively)
dominates it.
It is immediate that Definition 2.4.3 is equivalent to Definition
8.B.4 in MWG.

Lemma 2.4.1. Suppose n = 2. The strategy s10 S1 is not strictly

dominated by any other pure or mixed strategy if, and only if, s10
arg max u1 (s1 , 2 ) for some 2 (S2 ).

For more than two players, it is possible that a strategy can fail
to be a best reply to any mixed profile for the other players, and
yet that strategy is not strictly dominated (see Problem 2.6.17 for
an example).

Proof. We present the proof for finite Si .

(= ) If there exists 2 (S2 ) such that s10 arg max u1 (s1 , 2 ),
then it is straightforward to show that s10 is not strictly dominated
by any other pure or mixed strategy (left as exercise).
( =) We prove this direction by proving the contrapositive.4 So,
suppose s10 is a player 1 strategy satisfying

s10 6 arg max u1 (s1 , 2 ) 2 (S2 ). (2.4.1)

Define x(s1 , s2 ) = u1 (s1 , s2 ) u1 (s10 , s2 ), and observe that for fixed

s2 , we can represent the vector of payoff differences {x(s1 , s2 ) : s1
s10 } as a point in R|S1 |1 . Define

X conv{x R|S1 |1 : s2 S2 , xs1 = x(s1 , s2 )s1 s10 }.

|S |1
Denote the closed negative orthant by R 1 {x R|S1 |1 :
0
xs1 0, s1 s1 }. Equation (2.4.1) implies that for all 2 )(S2 ),
P |S |1
there exists s1 such that x(s1 , s2 )2 (s2 ) > 0, and so R 1 X =
. Moreover, X is closed, since it is the convex hull of a finite
number of vectors. See Figure 2.4.1.
4
The contrapositive of the conditional statement (A B) is (B A), and
has the same truth value as the original conditional statement.
36 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

s1

x(, s2 )
x(, s2 )

x(, s20 )
X

|S |1 s1
R 1
x(, s2 )

|S |1
Figure 2.4.1: The sets X and R 1 .

By an appropriate strict separating hyperplane theorem (see, for

example, Vohra (2005, Theorem 3.7)), R|S1 |1 \ {0} such that
|S |1 |S |1
x > x 0 for all x X and all x 0 R 1 . Since R 1 is
unbounded below, (s1 ) 0 s1 (otherwise making |x 0 (s1 )| large
enough for s1 satisfying (s1 ) < 0 ensures x 0 > x). Define
.
(s 00 ) P 00 0
1 s1 s10 (s1 ) , if s1 s1 ,
1 (s100 ) =
0, if s 00 = s 0 . 1 1

We now argue that 1 is a mixed strategy for 1 strictly dominating

|S |1
s10 : Since 0 R 1 , we have x > 0 for all x X, and so
X X
1 (s1 ) x(s1 , s2 )2 (s2 ) > 0, 2 ,
si s10 s2

i.e., for all 2 ,

X
u1 (1 , 2 ) = u1 (s1 , s2 )1 (s1 )2 (s2 )
s1 s10 ,s2
X
> u1 (s10 , s2 )1 (s1 )2 (s2 ) = u1 (s10 , 2 ).
s1 s10 ,s2
January 18, 2016 37

Remark 2.4.1. This proof requires us to strictly separate two dis-

joint closed convex sets (one bounded), rather than a point from
a closed convex set (the standard separating hyperplane theorem).
To apply the standard theorem, define Y {y R|S1 |1 : x
X, y` x` `}. Clearly Y is closed, convex and 0 Y . We can
proceed as in the proof, since the normal for the separating hyper-
plane must again have only nonnegative coordinates (use now the
unboundedness of Y ).

At least for two players, iterated strict dominance is thus it-

erated non-best repliesthe rationalizability notion of Bernheim
(1984) and Pearce (1984). For more than two players, as illustrated
in Problem 2.6.17, there is an issue of correlation. The rational-
izability notion of of Bernheim (1984) and Pearce (1984) does not
allow correlation, since it focuses on deleted strategies that are not
best replies to any mixed profile of the other players.
Lemma 2.4.1 holds for mixed strategies (see Problem 2.6.15).

2.4.3 Equilibrium in Mixed Strategies

Definition 2.4.4. Suppose {(S1 , u1 ), . . . , (Sn , un )} is an n-player nor-
mal form game. A Nash eq in mixed strategies is a profile (1 , . . . , n )
such that, for all i, for all i (Si ),

ui (i , i

) ui (i , i ). (2.4.2)

Equivalently, for all si Si ,

ui (i , i

) ui (si , i ),

since X

ui (i , i )= ui (si , i )i (si ) ui (si0 , i

),
si Si

where si0 arg max ui (si , i

).
38 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

Lemma 2.4.2. Suppose Si is countable. A strategy i is a best reply

to i (i.e., satisfies (2.4.2)) if, and only if,

i (si0 ) > 0 = si0 arg max ui (si , i

).
si

Proof. Left as an exercise (Problem 2.6.18).

Corollary 2.4.1. A strategy i is a best reply to i

(i.e., satisfies
(2.4.2)) if, and only if, for all si Si ,

ui (i , i

) ui (si , i ).

Example 2.4.3.

L R
T 2, 1 0, 0
B 0, 0 1, 1

(S1 ) = (S2 ) = [0, 1], p = Pr(T ), q = Pr(L).

is best replies in mixed strategies:


{1},
1
if q > 3 ,



1 (q) = [0, 1], if q = 13 ,




{0}, if q < 13 .

{1}, if p > 12 ,



2 (p) = [0, 1], if p = 12 ,



1
{0}, if p < 2 .

The best replies are graphed in Figure 2.4.2.

January 18, 2016 39

1
2

1
3

0 1 1 p
2

Figure 2.4.2: The best reply mappings for Example 2.4.3.

2.4.4 Behavior Strategies

What does mixing involve in extensive form games? Recall Defini-
tion 1.3.2:

Definition 2.4.5. A pure strategy for player i is a function

si : Hi h A(h) such that si (h) A(h) h Hi .

Denote is set of pure strategies by Si . Note that |Si | < for

finite extensive form games.
As above:

Definition 2.4.6. A mixed strategy for player i, i , is a probability

distribution over Si , i.e., i (Si ).

Definition 2.4.7. A behavior strategy for player i is a function

bi : Hi h (A(h)) such that bi (h) (A(h)), h Hi .

40 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

Write bi (h)(a) for the probability assigned to the action a

A(h) by the probability distribution bi (h).
Note that if |Hi | = 3 and |A(h)| = 2 h Hi , then |Si | = 8
and so (Si ) is a 7-dimensional simplex. On the hand, a behavior
strategy in this case requires only 3 numbers (the probability on
the first action in each information set).
The behavior strategy corresponding to a pure strategy si is
given by
bi (h) = si (h) , h Hi ,
where x (X), x X, is the degenerate distribution (Kro-
neckers delta), (
1, if y = x,
x (y) =
0, otherwise.

Definition 2.4.8. Two strategies for a player i are realization equiv-

alent if, fixing the strategies of the other players, the two strategies
induce the same distribution over outcomes (terminal nodes).
Thus, two strategies that are realization equivalent are strategi-
cally equivalent (Definition 1.3.6).
Moreover, if two extensive form strategies only differ in the
specification of behavior at an information set that one of those
strategies had precluded, then the two strategies are realization
equivalent. For example the strategies Stop,Stop1 and Stop,Go1 in
Example 2.2.3 are realization equivalent.
Given a behavior strategy bi , the realization-equivalent mixed
strategy i (Si ) is
Y
i (si ) = bi (h)(si (h)).
hHi

Theorem 2.4.1 (Kuhn, 1953). Every mixed strategy has a realization

equivalent behavior strategy.
The behavior strategy realization equivalent to the mixture i
can be calculated as follows: Fix an information set h for player i
(i.e., h Hi ). Suppose h is reached with strictly positive probability
under i , for some specification si . Then, bi (h) is the distribution
over A(h) implied by i conditional on h being reached. While this
January 18, 2016 41

calculation appears to depend on the particular choice of si , it

turns out it does not. (If for all specifications si , h is reached with
zero probability under i , then bi (h) can be determined arbitrarily.)
The proof is in Appendix 10.1.
Using behavior strategies, mixing can be easily accommodated in
subgame perfect equilibria (for an example, see Problem 2.6.19). In
particular, every finite extensive form game has a subgame perfect
equilibrium, once we allow for mixing (via behavior strategies, see
Problem 4.3.3(a)).5

2.5 Dealing with Multiplicity

2.5.1 I: Refinements
Example 2.5.1. Following game has two Nash equilibria (UL and
DR), but only DR is plausible. The other eq is in weakly dominated
strategies.

L R
U 2, 1 0, 0
D 2, 0 1, 1

Natural to require eq be robust to small mistakes. The follow-

ing notion captures the minimal requirement that behavior should
be robust at least to some mistakes. The stronger requirement that
behavior be robust to all mistakes is too strong, in the sense that
typically, no behavior satisfies that requirement, and so is rarely
imposed (see Problem 2.6.22) for an example).

Definition 2.5.1. An equilibrium of a finite normal from game G

is (normal
 form) trembling hand perfect if there exists a sequence
k k of completely mixed strategy profiles converging to such
k
that i is a best reply to every i in the sequence.
5
However, once players have infinite action spaces, even if payoffs are contin-
uous and action spaces are compact, subgame perfect equilibria need not exist.
See Problem 4.3.3(b).
42 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

This is NOT the standard definition in the literature, but is equiv-

alent to it (see Subsection 10.2.1).
Every finite normal form game has a trembling hand perfect
equilibrium (see Subsection 10.2.1).
Weakly dominated strategies cannot be played in a trembling
hand perfect equilibrium:
Theorem 2.5.1. If a strategy profile in a finite normal form game is
trembling hand perfect then it is a Nash equilibrium in weakly un-
dominated strategies. If there are only two players, every Nash equi-
librium in weakly undominated strategies is trembling hand perfect.
Proof. The proof of the first statement is straightforward and left
as an exercise (Problem 2.6.20). A proof of the second statement
can be found in van Damme (1991, Theorem 3.2.2).
Problem 2.6.21 illustrates the two statements of Theorem 2.5.1.
We explore the role of trembles in extensive form games in Sec-
tion 5.3 (see also Problem 2.6.8).
Some additional material on trembling hand perfect equilibria
are collected in the appendix to this chapter (Section 10.2).

2.5.2 II: Selection

Example 2.5.2 (Focal Points).

A a b
A 2, 2 0, 0 0, 0
a 0, 0 0, 0 2, 2
b 0, 0 2, 2 0, 0

Example 2.5.3 (payoff dominance).

` r
T 2, 2 0, 0
B 0, 0 1, 1
January 18, 2016 43

Example 2.5.4 (Renegotiation). Compare with example 2.3.3. The

stage game is
L C R
T 4, 4 0, 0 9, 0
.
M 0, 0 6, 6 0, 0
B 0, 0 0, 0 8, 8
(T , L) and (M, C) are both Nash eq of the stage game. The profile
(s1 , s2 ) of the once-repeated game with payoffs added is subgame
perfect:

s11 = B,
(
M, if x = B,
s12 (x, y) =
T , if x B,

s21 = R,
(
C, if x = B, and
s22 (x, y) =
L, if x B.

The outcome path induced by (s1 , s2 ) is (BR, MC). But T L is Pareto

dominated by MC, and so players may renegotiate from T L to MC
after 1s deviation. But then 1 has no incentive not to play T in the
first period.

Example 2.5.5 (stag hunt game; illustration of risk dominance).

A B
A 9, 9 0, 5
B 5, 0 7, 7

(A is hunting the stag, while B is catching the hareRousseau.) The

profile AA is the efficient profile (or is payoff dominant). But B is
 44 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

less risky than A: technically, it is risk dominant since B is the

unique best reply to the uniform lottery over {A, B}, i.e., to the
mixture
1 1
A + B.
2 2

2.6 Problems
2.6.1. Suppose {(Si , Ui )n
i=1 } is a normal form game, and s1 S1 is a weakly
dominated strategy for player 1. Let S10 = S1 \ {s1 }, and Si0 = Si for
i 1. Suppose s is a Nash equilibrium of {(Si0 , Ui )n
i=1 }. Prove that s
is a Nash equilibrium of {(Si , Ui )n
i=1 }.

2.6.2. Suppose {(Si , Ui )n

i=1 } is a normal form game, and s is a Nash equi-
librium of {(Si , Ui )n 0 n
i=1 }. Let {(Si , Ui )i=1 } be the normal form game
obtained by the iterated deletion of some or all strictly dominated
strategies. Prove that s is a Nash equilibrium of {(Si0 , Ui )n i=1 }. (Of
0
course, you must first show that si Si for all i.) Give an example
showing that this is false if strictly is replaced by weakly.

2.6.3. Consider (again) the Cournot example (Example 1.1.4). What is the
Nash Equilibrium of the n-firm Cournot oligopoly? [Hint: To cal-
culate the equilibrium, first solve for the total output in the Nash
equilibrium.] What happens to both individual firm output and total
output as n approaches infinity?

2.6.4. Consider now the Cournot duopoly where inverse demand is P (Q) =
a Q but firms have asymmetric marginal costs: ci for firm i, i =
1, 2.

(a) What is the Nash equilibrium when 0 < ci < a/2 for i = 1, 2?
What happens to firm 2s equilibrium output when firm 1s
costs, c1 , increase? Can you give an intuitive explanation?
(b) What is the Nash equilibrium when c1 < c2 < a but 2c2 > a+c1 ?

2.6.5. Consider the following Cournot duopoly game: The two firms are
identical. The cost function facing each firm is denoted by C(q),
is continuously differentiable with C(0) = 0, C 0 (0) = 0, C 0 (q) >
0 q > 0. Firm i chooses qi , i = 1, 2. Inverse demand is given
 January 18, 2016 45

by p = P (Q), where Q = q1 + q2 is total supply. Suppose P is contin-

uous and there exists Q > 0 such that P (Q) > 0 for Q [0, Q) and
P (Q) = 0 for Q Q. Assume firm is profits are strictly concave in
qi for all qj , j i.

(a) Prove that for each value of qj , firm i (i j) has a unique

profit maximizing choice. Denote this choice Ri (qj ). Prove that
Ri (q) = Rj (q), i.e., the two firms have the same reaction func-
tion. Thus, we can drop the subscript of the firm on R.
(b) Prove that R(0) > 0 and that R(Q) = 0 < Q.
(c) We know (from the maximum theorem) that R is a continu-
ous function. Use the Intermediate Value Theorem to argue
that this Cournot game has at least one symmetric Nash equi-
librium, i.e., a quantity q , such that (q , q ) is a Nash equi-
librium. [Hint: Apply the Intermediate Value Theorem to the
function f (q) = R(q) q. What does f (q) = 0 imply?]
(d) Give some conditions on C and P that are sufficient to imply
that firm is profits are strictly concave in qi for all qj , j i.

2.6.6. (easy) Prove that the information set containing the initial node of a
subgame is necessarily a singleton.

2.6.7. In the canonical Stackelberg model, there are two firms, I and II, pro-
ducing the same good. Their inverse demand function is P = 6 Q,
where Q is market supply. Each firm has a constant marginal cost
of $4 per unit and a capacity constraint of 3 units (the latter restric-
tion will not affect optimal behavior, but assuming it eliminates the
possibility of negative prices). Firm I chooses its quantity first. Firm
II, knowing firm Is quantity choice, then chooses its quantity. Thus,
firm Is strategy space is S1 = [0, 3] and firm IIs strategy space is
S2 = {2 | 2 : S1 [0, 3]}. A strategy profile is (q1 , 2 ) S1 S2 ,
i.e., an action (quantity choice) for I and a specification for every
quantity choice of I of an action (quantity choice) for II.

(a) What are the outcome and payoffs of the two firms implied by
the strategy profile (q1 , 2 )?
(b) Show that the following strategy profile does not constitute a
Nash equilibrium: ( 12 , 2 ), where 2 (q1 ) = (2 q1 )/2. Which
firm(s) is (are) not playing a best response?
 46 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

(c) Prove that the following strategy profile constitutes a Nash equi-
1 3 1
librium: ( 2 , 2 ), where 2 (q1 ) = 4 if q1 = 2 and 2 (q1 ) = 3 if
q1 12 , i.e., II threatens to flood the market unless I produces
exactly 12 . Is there any other Nash equilibrium which gives the
outcome path ( 21 , 34 )? What are the firms payoffs in this equi-
librium?
(d) Prove that the following strategy profile constitutes a Nash equi-
librium: (0, 2 ), where 2 (q1 ) = 1 if q1 = 0 and 2 (q1 ) = 3 if
q1 0, i.e., II threatens to flood the market unless I produces
exactly 0. What are the firms payoffs in this equilibrium?
(e) Given q1 [0, 2], specify a Nash equilibrium strategy profile
in which I chooses q1 . Why is it not possible to do this for
q1 (2, 3]?
(f) What is the unique backward induction solution of this game?

2.6.8. Player 1 and 2 must agree on the division of a pie of size 1. They
are playing a take-it-or-leave-it-offer game: Player 1 makes an offer
x from a set S1 [0, 1], which player 2 accepts or rejects. If player 2
accepts, the payoffs are 1 x to player 1 and x to player 2; if player
2 rejects, both players receive a zero payoff.

(a) Describe the strategy spaces for both players.

n o
1 n1
(b) Suppose S1 = 0, n , . . . , n , 1 , for some positive integer n
2. Describe all the backward induction solutions.
(c) Suppose S1 = [0, 1]. Describe all the backward induction so-
lutions. (While the game is not a finite game, it is a game of
perfect information and has a finite horizon, and so the notion
of backward induction applies in the obvious way.)

2.6.9. Consider the extensive form in Figure 2.6.1.

(a) What is the normal form of this game?

(b) Describe the pure strategy Nash equilibrium strategies and out-
comes of the game.
(c) Describe the pure strategy subgame perfect equilibria (there
may only be one).
 January 18, 2016 47

L R

I II

L0 R0 `0 r0

II
2 0
` r ` r 1 0

3 4 1 5
1 0 0 1

Figure 2.6.1: The game for Problem 2.6.9

2.6.10. Consider the following game G between two players. Player 1 first
chooses between A or B, with A giving payoff of 1 to each player,
and B giving a payoff of 0 to player 1 and 3 to player 2. After player
1 has publicly chosen between A and B, the two players play the
following bimatrix game (with 1 being the row player):

L R
U 1, 1 0, 0
D 0, 0 3, 3

Payoffs in the overall game are given by the sum of payoffs from 1s
initial choice and the bimatrix game.
(a) What is the extensive form of G?
(b) Describe a subgame perfect equilibrium strategy profile in pure
strategies in which 1 chooses B.
(c) What is the reduced normal form of G?
(d) What is the result of the iterated deletion of weakly dominated
strategies?
 48 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

2.6.11. Suppose s is a pure strategy Nash equilibrium of a finite extensive

form game, . Suppose 0 is a subgame of that is on the path of
play of s. Prove that s prescribes a Nash equilibrium on 0 . (It is
probably easier to first consider the case where there are no moves
of nature.) (The result is also true for mixed strategy Nash equilibria,
though the proof is more notationally intimidating.)

2.6.12. Prove that player i s security level (Definition 2.4.2) is also given by

vi = sup inf
Q
ui (i , si ).
i (Si ) si ji Sj

Prove that
v i sup inf
Q
ui (si , si ),
si Si si ji Sj

and give an example illustrating that the inequality can be strict.

2.6.13. Suppose the 2 2 normal form game G has a unique Nash equi-
librium, and each players Nash equilibrium strategy and security
strategy are both completely mixed.

(a) Describe the implied restrictions on the payoffs in G.

(b) Prove that each players security level is given by his/her Nash
equilibrium payoff.
(c) Give an example showing that (in spite of part 2.6.13(b)), the
Nash equilibrium profile need not agree with the strategy pro-
file in which each player is playing his or her security strategy.
(This is not possible for zero-sum games, see Problem 4.3.2.)
(d) For games like you found in part 2.6.13(c), which is the better
prediction of play, security strategy or Nash equilibrium?

2.6.14. Suppose {(S1 , u1 ), . . . , (Sn , un )} is a finite normal form game. Prove

that if s10 S1 is strictly dominated in the sense of Definition 2.4.3,
then it is not a best reply to any belief over Si . [While you can prove
this by contradiction, try to obtain the direct proof, which is more
informative.] (This is the contrapositive of the straightforward
direction of Lemma 2.4.1.)

2.6.15. (a) Prove that Lemma 2.4.1 also holds for mixed strategies, i.e.,
prove that 1 (S1 ) is strictly dominated by some other strat-
egy 10 (i.e., u1 (10 , s2 ) > u1 (1 , s2 ), s2 S2 ) if and only if 1
is not a best reply to any mixture 2 (S2 ).
 January 18, 2016 49

L R
T 5, 0 0, 1
C 2, 6 4, 0
B 0, 0 5, 1

Figure 2.6.2: The game for Problem 2.6.15(b).

1 1
(b) For the game illustrated in Figure 2.6.2, prove that 2 T + 2 B
is not a best reply to any mixture over L and R. Describe a
strategy that strictly dominates it.

2.6.16. Suppose {(S1 , u1 ), (S2 , u2 )} is a two player finite normal form game
and let Sb2 be a strict subset of S2 . Suppose s10 S1 is not a best
reply to any beliefs with support Sb2 . Prove that there exists > 0
such that s10 is not a best reply to any beliefs (S2 ) satisfying
(Sb2 ) > 1 . Is the restriction to two players important?

2.6.17. Consider the three player game in Figure 2.6.3 (only player 3s pay-
offs are presented).

(a) Prove that player 3s strategy of M is not strictly dominated.

(b) Prove that player 3s strategy of M is not a best reply to any
mixed strategy profile (1 , 2 ) (S1 ) (S2 ) for players 1
and 2. (The algebra is a little messy. Given the symmetry, it
suffices to show that L yields a higher expected payoff than M
1
for all mixed strategy profiles satisfying 1 (t) 2 .)

2.6.18. Prove Lemma 2.4.2.

2.6.19. Suppose player 1 must publicly choose between the game on the left
and the game on the right in Figure 2.6.4 (where player 1 is choosing
rows and player 2 is choosing columns. Prove that this game has no
Nash equilibrium in pure strategies. What is the unique subgame
perfect equilibrium (the equilibrium is in behavior strategies).

2.6.20. Is it necessary to assume that is a Nash equilibrium in the defi-

nition of normal form trembling hand perfection (Definition 2.5.1)?
50 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

` r ` r ` r
t 3 0 t 2 4 t 0 0
b 0 0 b 4 2 b 0 3

L M R

Figure 2.6.3: The game for Problem 2.6.17. Player 1 chooses rows (i.e.,
s1 {t, b}), player 2 chooses columns (i.e., s2 {`, r }), and
player 3 chooses matrices (i.e., s3 {L, M, R}). Only player
3s payoffs are given.

t h T H
t 1, 1 1, 1 T 3, 1 1, 1
h 1, 1 1, 1 H 1, 1 1, 1

Figure 2.6.4: The game for Problem 2.6.19.

 January 18, 2016 51

L C R
T 2, 2 1, 1 0, 0
B 2, 0 0, 0 4, 2

Figure 2.6.5: The game for Problem 2.6.21(a).

` r ` r
T 1, 1, 1 1, 0, 1 T 1, 1, 0 0, 0, 0
B 1, 1, 1 0, 0, 1 B 0, 1, 0 1, 0, 0

L R

Figure 2.6.6: The game for Problem 2.6.21(b). Player 1 chooses a row (T or
B), player 2 chooses a column (` or r ), and player 3 chooses
a matrix (L or R). In each cell, the first payoff is player 1s,
the second is player 2s, and the third is player 3s.

Prove that every trembling hand perfect equilibrium of a finite nor-

mal form game is a Nash equilibrium in weakly undominated strate-
gies.

2.6.21. Two examples to illustrate Theorem 2.5.1.

(a) For the game in Figure 2.6.5, prove that T L is trembling hand
perfect by explicitly describing the sequence of trembles.
(b) The game in Figure 2.6.6 has an undominated Nash equilibrium
that is not trembling hand perfect. What is it?
n o
2.6.22. Say a profile is robust to all trembles if for all sequences k of
k
completely mixed strategy profiles converging to , i is eventually
k
a best reply to every i in the sequence.6
6 k
By eventually, I mean there exists K such that i is a best reply to i for all
k > K. Note that K depends on the sequence.
 52 CHAPTER 2. A FIRST LOOK AT EQUILIBRIUM

L M R
T 1, 5 2, 3 0, 0
B 1, 5 0, 0 3, 2

Figure 2.6.7: The game for Problem 2.6.22

(a) Prove that no profile in the game in Figure 2.6.7 is robust to all
trembles.
(b) There is an extensive form with a nontrivial subgame that has
as its normal the game in Figure 2.6.7. This extensive form
game has two subgame perfect equilibria. What are they? Com-
pare with your analysis of part 2.6.22(a).

2.6.23. It is not true that every trembling hand perfect equilibrium of a nor-
mal form game induces subgame perfect behavior in an extensive
game with that normal form. Illustrate using the game in Figure
2.6.8.

L M R
T 1, 5 2, 6 0, 0
B 1, 5 0, 3 3, 2

Figure 2.6.8: The game for Problem 2.6.23

Chapter 3

Games with Nature1

3.1 An Introductory Example

Example 3.1.1 (Incomplete information version of example 1.1.4).
Firm 1s costs are private information, while firm 2s are public.
Nature determines the costs of firm 1 at the beginning of the game,
with Pr(c1 = cH ) = (0, 1). As in example 1.1.4, firm is profit is
i (q1 , q2 ; ci ) = [(a q1 q2 ) ci ]qi ,
where ci is firm is cost. Assume cL , cH , c2 < a/2. A strategy for
player 2 is a quantity q2 . A strategy for player 1 is a function q1 :
{cL , cH } R+ . For simplicity, write qL for q1 (cL ) and qH for q1 (cH ).
Note that for any strategy profile ((qH , qL ), q2 ), the associated
outcome is
(qH , q2 ) + (1 ) (qL , q2 ),
that is, with probability , the terminal node (qH , q2 ) is realized,
and with probability 1 , the terminal node (qL , q2 ) is realized.
To find a Nash equilibrium, we must solve for three numbers
qL , qH

, and q2 .
Assume interior solution. We must have:

(qH , qL ) = arg max [(a qH q2 ) cH ]qH
qH ,qL

+ (1 )[(a qL q2 ) cL ]qL .
1
Copyright January 18, 2016 by George J. Mailath

53
54 CHAPTER 3. GAMES WITH NATURE

This implies pointwise maximization, i.e.,

qH = arg max [(a q1 q2 ) cH ]q1
q1
1
= (a q2 cH ).
2
and
1
qL = (a q2 cL ).
2
We must also have
q2 = arg max [(a qH

q2 ) c2 ]q2
q2

+ (1 )[(a qL q2 ) c2 ]q2

= arg max [(a qH (1 )qL q2 ) c2 ]q2
q2
1


= a c2 qH (1 )qL .
2
Solving,

a 2cH + c2 1
qH = + (cH cL )
3 6
a 2cL + c2
qL = (cH cL )
3 6
a 2c2 + cH + (1 ) cL
q2 =
3

3.2 Purification
Player is mixed strategy i in a complete information game G is
said to be purified if in an incomplete information version of G
(with player is type space given by Ti ), that players behavior can
be written as a pure strategy si : Ti Ai such that
i (ai ) = Pr{si (ti ) = ai },
January 18, 2016 55

A B
A 9, 9 0, 5
B 5, 0 7, 7

Figure 3.2.1: The benchmark game for Example 3.2.1.

where Pr is given by the prior distribution over Ti (and so is player

j i beliefs over Ti ).

Example 3.2.1. The game in Figure 3.2.1 has two strict pure strat-
egy Nash equilibria and one symmetric mixed strategy Nash equi-
librium. Let p = Pr {A}, then (using Lemma 2.4.2)


9p = 5p + 7 1 p
a 9p = 7 2p
7
a 11p = 7 a p = .
11
Trivial purification: give player i a payoff irrelevant type ti where
ti U([0, 1]), and t1 and t2 are independent. Then, the mixed
strategy eq is purified by many pure strategy eq in the incomplete
information game, such as
(
B, if ti 4/11,
si (ti ) =
A, if ti 4/11.

Harsanyi (1973) purification: Consider the game G() displayed

in Figure 3.2.2. This is a game of incomplete information, where
ti U([0, 1]) and t1 and t2 are independent.
A pure strategy for player i is si : [0, 1] {A, B}. Suppose 2 is
following a cutoff strategy,
(
A, t2 t2 ,
s2 (t2 ) =
B, t2 < t2 ,

with t2 (0, 1).

56 CHAPTER 3. GAMES WITH NATURE

A B
A 9 + t1 , 9 + t2 0, 5
B 5, 0 7, 7

Figure 3.2.2: The game G() for Example 3.2.1. Player i has type ti , with
ti U([0, 1]), and t1 and t2 are independent.

Type t1 expected payoff from A is

U1 (A, t1 , s2 ) = (9 + t1 ) Pr {s2 (t2 ) = A}


= (9 + t1 ) Pr t2 t2
= (9 + t1 )(1 t2 ),

while from B is
 
U1 (B, t1 , s2 ) = 5 Pr t2 t2 + 7 Pr t2 < t2
= 5(1 t2 ) + 7t2
= 5 + 2t2 .

Thus, A is optimal iff

(9 + t1 )(1 t2 ) 5 + 2t2

i.e.,
11t2 4
t1 .
(1 t2 )
Thus the best reply to the cutoff strategy s2 is a cutoff strategy with
t1 = (11t2 4)/(1 t2 ).2 Since the game is symmetric, try for a
symmetric eq: t1 = t2 = t. So

11t 4
t = ,
(1 t)
or
t 2 + (11 )t 4 = 0. (3.2.1)
2
Indeed, even if player 2 were not following a cutoff strategy, player 1s best
reply is a cutoff strategy.
January 18, 2016 57

Let t() denote the value of t satisfying (3.2.1). Note first that
t(0) = 4/11, and that writing (3.2.1) as g(t, ) = 0, we can apply the
implicit function theorem to conclude that for > 0 but close to 0,
the cutoff type t() is close to 4/11, the probability of the mixed
strategy eq in the unperturbed game. In other words, for small,
there is a symmetric equilibrium in cutoff strategies, with t (0, 1).
This equilibrium is not only pure, but almost everywhere strict!
The interior cutoff equilibrium of G() approximates the mixed
strategy equilibrium of G(0) in the following sense: Let p() be
the probability assigned to A by the symmetric cutoff equilibrium
strategy of G(). Then p(0) = 7/11 and p() = 1 t(). Since we
argued in the previous paragraph that t() 4/11 as 0, we
have that for all > 0, there exists > 0 such that

|p() p(0)| < .

Harsanyis (1973) purification theorem is the most compelling

justification for mixed equilibria in finite normal form games. In-
tuitively, it states that for almost all complete information normal
form games, and any sequence of incomplete-information games,
where each players payoffs are subject to independent non-atomic
private shocks, converging to the complete-information normal form
game, the following is true: every equilibrium (pure or mixed) of the
original game is the limit of equilibria of these close-by games with
incomplete information. Moreover, in the incomplete-information
games, players have essentially strict best replies, and so will not
randomize. Consequently, a mixed strategy equilibrium can be
viewed as a pure strategy equilibrium of any close-by game of in-
complete information.
See Govindan, Reny, and Robson (2003) for a modern exposition
and generalization of Harsanyi (1973). A brief introduction can also
be found in Morris (2008).

3.3 Auctions and Related Games

Example 3.3.1 (First-price sealed-bid auctionprivate values).
Bidder is value for the object, vi is known only to i. Nature
58 CHAPTER 3. GAMES WITH NATURE

chooses vi , i = 1, 2 at the beginning of the game, with vi being

independently drawn from the interval [vi , vi ], with CDF Fi , density
fi . Bidders know Fi (and so fi ).
This is an example of independent private values.

Remark 3.3.1. An auction (or similar environment) is said to have

private values if each buyers (private) information is sufficient to
determine his value (i.e., it is a sufficient statistic for the other buy-
ers information). The values are independent if each buyers pri-
vate information is stochastically independent of every other bid-
ders private information.
An auction (or similar environment) is said to have interdepen-
dent values if the value of the object to the buyers is unknown at
the start of the auction, and if a bidders (expectation of the) value
can be affected by the private information of other bidders. If all
bidders have the same value, then we have the case of pure common
value.

Set of possible bids, R+ .

Bidder is ex post payoff as a function of bids b1 and b2 , and
values v1 and v2 :


0, if bi < bj ,



1
ui (b1 , b2 , v1 , v2 ) = (vi bi ) , if bi = bj ,

2


vi bi , if bi > bj .

Suppose 2 uses strategy 2 : [v2 , v2 ] R+ . Then, bidder 1s

expected (or interim) payoff from bidding b1 at v1 is
Z
U1 (b1 , v1 ; 2 ) = u1 (b1 , 2 (v2 ) , v1 , v2 ) dF2 (v2 )
1
= (v1 b1 ) Pr {2 (v2 ) = b1 }
2 Z
+ (v1 b1 ) f2 (v2 ) dv2 .
{v2 :2 (v2 )<b1 }
January 18, 2016 59

Player 1s ex ante payoff from the strategy 1 is given by

Z
U1 (1 (v1 ), v1 ; 2 ) dF1 (v1 ),

and so for an optimal strategy 1 , the bid b1 = 1 (v1 ) must maxi-

mize U1 (b1 , v1 ; 2 ) for almost all v1 .
Suppose 2 is strictly increasing. Then, Pr {2 (v2 ) = b1 } = 0
and
Z
U1 (b1 , v1 ; 2 ) = (v1 b1 ) f2 (v2 ) dv2
{v2 :2 (v2 )<b1 }
= E[v1 b1 | winning] Pr{winning}
= (v1 b1 ) Pr {2 (v2 ) < b1 }
= (v1 b1 ) Pr{v2 < 21 (b1 )}
= (v1 b1 )F2 (21 (b1 )).
Assuming 2 is, moreover, differentiable, and that the bid b1 =
1 (v1 ) is an interior maximum, the first order condition is


d21 (b1 )
0 = F2 21 (b1 ) + (v1 b1 ) f2 21 (b1 ) .
db1
But
d21 (b1 ) 1
= 0 1 ,
db1 2 (2 (b1 ))
so



F2 21 (b1 ) 20 21 (b1 ) = (v1 b1 ) f2 21 (b1 ) ,
i.e.,

(v1 b1 ) f2 21 (b1 )


20 21
(b1 ) =
.
F2 21 (b1 )
Suppose F1 = F2 and suppose the eq is symmetric, so that 1 = 2 =
, and b1 = 1 (v) = v = 21 (b1 ). Then, dropping subscripts,
(v (v))f (v)
0 (v) = . (3.3.1)
F (v)
If v = v + 1 and values are uniformly distributed on [v, v + 1], then

0 v (v)
(v) = ,
v v

60 CHAPTER 3. GAMES WITH NATURE

i.e.,
(v v) 0 (v) + (v) = v.

But,
d


v v (v) = v v 0 (v) + (v) ,
dv
so

v2
v v (v) = + k,
2
where k is a constant of integration. Moreover, evaluating both
sides at v = v shows that k = v 2 /2, and so


v2 v2
v v (v) =
2
1

= (v) = v +v . (3.3.2)
2
Summarizing the calculations till this point, I have shown that
if ( , ) is a Nash equilibrium in which is a strictly increasing
and differentiable function, and (v) is interior (which here means
strictly positive), then it is given by (3.3.2).
It remains to verify the hypotheses. It is immediate that the
function in (3.3.2) is strictly increasing and differentiable. More-
over, for v > 0, (v) is strictly positive. It remains to verify the
optimality of bids (i.e., we still need to verify that bidding b1 =
(v1 ) is optimal when bidder 2 is bidding according to ; given
the symmetry, this is enough). Bidder 1s interim payoff is (since
1
(v2 ) [v, v + 2 ] for all v2 [v, v + 1])

U1 (b1 , v1 ; 2 ) = (v1 b1 )F2 (21 (b1 ))

1
v1 b1 ,
if b1 > v + 2 ,
1
= (v1 b1 )(2b1 2v), if b1 [v, v + 2 ],


0, if b1 < v.

1
This function is strictly concave for b1 [v, v + 2 ], and it is clearly
not optimal to bid outside the interval, and so
bidding b = (v )
1 1
is optimal.
As an illustration of the kind of arguments that are useful, I
now argue that every Nash equilibrium must be in nondecreasing
strategies.
January 18, 2016 61

Lemma 3.3.1. Suppose (1 , 2 ) is a Nash equilibrium of the first

price sealed bid auction with independent private values, with CDF
Fi on [vi , vi ]. Suppose type vi0 wins the auction with positive proba-

bility. Then, i (vi00 ) i (vi0 ) for all vi00 > vi0 .

Proof. Without loss of generality, consider i = 1. Suppose the

Lemma is false. Then there exists v100 > v10 with 1 (v10 ) =: b10 >
b100 := 1 (v100 ).
Incentive compatibility implies

U1 (b10 , v10 ; 2 ) U1 (b100 , v10 ; 2 ),

and U1 (b10 , v100 ; 2 ) U1 (b100 , v100 ; 2 ).

Note that

U1 (b1 , v10 ; 2 ) U1 (b1 , v100 ; 2 ) =

1 0
(v v100 ) Pr{2 (v2 ) = b1 } + (v10 v100 ) Pr{2 (v2 ) < b1 }.
2 1
Subtracting the second from the first inequality gives

U1 (b10 , v10 ; 2 ) U1 (b10 , v100 ; 2 ) U1 (b100 , v10 ; 2 ) U1 (b100 , v100 ; 2 ),

and so substituting,

1 0
(v v100 ) Pr{2 (v2 ) = b10 } + (v10 v100 ) Pr{2 (v2 ) < b10 }
2 1
1 0
(v1 v100 ) Pr{2 (v2 ) = b100 } + (v10 v100 ) Pr{2 (v2 ) < b100 },
2
and simplifying (and dividing by (v10 v100 ) < 0) we get

0 Pr{b100 2 (v2 ) < b10 }

1
+ Pr{2 (v2 ) = b10 } Pr{2 (v2 ) = b100 }
2
= Pr{b100 < 2 (v2 ) < b10 }
1
+ Pr{2 (v2 ) = b10 } + Pr{2 (v2 ) = b100 } .
2
62 CHAPTER 3. GAMES WITH NATURE

This implies

0 = Pr{2 (v2 ) = b10 },

0 = Pr{2 (v2 ) = b100 },
and 0 = Pr{b100 < 2 (v2 ) < b10 }.

That is, bidder 2 does not make a bid between b100 and b10 , and there
are no ties at b10 or b100 . A bid of b10 and b100 therefore wins with the
same probability. But this implies a contradiction: Since b10 wins
with positive probability, v10 strictly prefers to win with the same
probability at the strictly lower bid of b100 .

Example 3.3.2 (independent private values, symmetric n bidders).

Suppose now there n identical bidders, with valuations vi indepen-
dently distributed on [v, v] according to F with density f .

Interested in characterizing the symmetric Nash equilibrium (if
it exists). Let be the symmetric strategy, and suppose it is strictly
increasing. Consequently, the probability of a tie is zero, and so
bidder is interim payoff from the bid bi is

Ui (bi , vi ; ) = E[vi bi | winning] Pr{winning}

= (vi bi ) Pr{vj < 1 (bi ), j i}
Y
= (vi bi ) Pr{vj < 1 (bi )}
ji

= (vi bi )F n1 ( 1 (bi )).

As before, assuming is differentiable, and an interior solution,

the first order condition is

0 = F n1 ( 1 (bi ))
d 1 (bi )
+ (vi bi )(n 1)F n2 ( 1 (bi ))f ( 1 (bi )) ,
dbi
and simplifying (similarly to (3.3.1)), we get

0 (v)F n1 (v) + (v)(n 1)F n2 (v)f (v)

= v(n 1)F n2 (v)f (v),
January 18, 2016 63

that is,
d
(v)F n1 (v) = v(n 1)F n2 (v)f (v),
dv
or (where the constant of integration is zero, since F (v) = 0),

Zv
1
(v) = n1 x dF n1 (x).
F (v) v

Remark 3.3.2 (Order statistics). Given n independent draws from

n n n
a common distribution F , denoted v1 , . . . , vn , let y(1) , y(2) , . . . , y(n)
n n n
denote the rearrangement satisfying y(1) y(2) . . . y(n) . The
n
statistic y(k) is the k -order statistic from the sample of n draws.
th
n n
The distribution of y(n) is Pr{y(n) y} = F n (y).

If is a symmetric Nash equilibrium, then

n1 n1
(v) = E[y(n1) | y(n1) v].

That is, each bidder bids the expectation of the maximum of all the
other bidders valuation, conditional on that valuation being less
than his (i.e., conditional on his value being the highest). Equiv-
alently, the bidder bids the expected value of the (n 1)th order
statistic of values, conditional on his value being the nth order
statistic.

Example 3.3.3 (First-price sealed-bid auctionpure common val-

ues). Each bidder receives a private signal about the value of the ob-
ject, ti , with ti Ti = [0, 1], uniformly independently distributed.
The common (to both players) value of the object is v = t1 + t2 .
Ex post payoffs are given by


t1 + t2 bi ,
if bi > bj ,
1
ui (b1 , b2 , t1 , t2 ) = 2 (t1 + t2 bi ), if bi = bj ,


0, if bi < bj .
64 CHAPTER 3. GAMES WITH NATURE

Suppose 2 uses strategy 2 : T2 R+ . Suppose 2 is strictly

increasing. Then, t1 s expected payoff from bidding b1 is
U1 (b1 , t1 ; 2 ) = E[t1 + t2 b1 | winning] Pr{winning}
= E[t1 + t2 b1 | t2 < 21 (b1 )] Pr{t2 < 21 (b1 )}
Z 21 (b1 )
1
= (t1 b1 )2 (b1 ) + t2 dt2
0
= (t1 b1 )21 (b1 ) + (21 (b1 ))2 /2.
If 2 is differentiable, the first order condition is
d21 (b1 ) d21 (b1 )
0 = 21 (b1 ) + (t1 b1 ) + 21 (b1 ) ,
db1 db1
and so
21 (b1 )20 (21 (b1 )) = (t1 + 21 (b1 ) b1 ).
Suppose F1 = F2 and suppose the eq is symmetric, so that 1 =
2 = , and b1 = 1 (t) = t = 21 (b1 ). Then,
t 0 (t) = 2t (t).
Integrating,
t (t) = t 2 + k,
where k is a constant of integration. Evaluating both sides at t = 0
shows that k = 0, and so
(t) = t.
Note that this is NOT the profile that results from the analysis
of the private value auction when v = 1/2 (the value of the object

in the common value auction, conditional on only t1 , is E[t1 + t2 |
t1 ] = t1 + 1/2). In particular, letting v = t + 12 , we have
0

v 0 + 1/2 t+1
private value (t) = (v 0 ) = = > t = common value (t).
2 2
This illustrates the winners curse: E[v | t1 ] > E[v|t1 , winning]. In
particular, in the equilibrium just calculated,
E[v | t1 , winning] = E[t1 + t2 | t1 , t2 < t1 ]
Z t1
1
= t1 + t2 dt2
Pr {t2 < t1 } 0
1 h 2
it1 3t1
= t1 + (t2 ) /2 = ,
t1 0 2
January 18, 2016 65

1
while E[v | t1 ] = t1 + 2
> 3t1 /2 (recall t1 [0, 1]).

Example 3.3.4 (War of attrition).

Action spaces Si = R+ .
Private information (type) ti Ti R+ , CDF Fi , density fi , with
fi strictly positive on Ti .
Ex post payoffs
(
ti sj , if sj < si ,
ui (s1 , s2 , t1 , t2 ) =
si , if sj si .

Suppose 2 uses strategy 2 : T2 S2 . Then, t1 s expected (or

interim) payoff from stopping at s1 is
Z
U1 (s1 , t1 ; 2 ) = u1 (s1 , 2 (t2 ) , t) dF2 (t2 )
Z
= s1 Pr {2 (t2 ) s1 } + (t1 2 (t2 )) dF2 (t2 ) .
{t2 :2 (t2 )<s1 }

Any Nash equilibrium is sequentially rational on the equilibrium

path: Suppose < 1 (t1 ) is reached (i.e., 2 has not yet dropped
out) and Pr{2 (t2 ) > } > 0 (so that such an event has positive
probability). Is stopping at 1 (t1 ) still optimal? Suppose that, con-
ditional on play reaching , the stopping time s1 > yields a higher
payoff than the original stopping time s1 = 1 (t1 ), i.e.,

Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) > ]

< Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) > ]
Then,
U1 (s1 , t1 ; 2 ) = Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) ] Pr{2 (t2 ) }
+ Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) > ] Pr{2 (t2 ) > }
< Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) ] Pr{2 (t2 ) }
+ Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) > ] Pr{2 (t2 ) > }
= Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) ] Pr{2 (t2 ) }
+ Et2 [u1 (s1 , 2 (t2 ), t) | 2 (t2 ) > ] Pr{2 (t2 ) > }
= U1 (s1 , t1 ; 2 ),
66 CHAPTER 3. GAMES WITH NATURE

and so s1 cannot have been the unconditionally optimal stopping

time. This is an application of the principle of Problem 2.6.11 to an
infinite game.
Define si inf{si : Pr{i (ti ) si } = 1} = inf{si : Pr{i (ti ) >
si } = 0}, where inf{} = . It can be shown that in any Nash
eq with s1 , s2 > 0, s1 = s2 . (If s2 < s1 , then for sufficiently large
types for player 2, there are late stopping times that are profitable
deviationssee Problem 3.6.5).
Lemma 3.3.2. Suppose (1 , 2 ) is a Nash eq. profile. Then, i is
nondecreasing for i = 1, 2.
Proof. We use a standard revealed preference argument. Let s10 =
1 (t10 ) and s100 = 1 (t100 ), with s10 , s100 s1 . If 1 is a best reply to 2 ,
U1 (s10 , t10 ; 2 ) U1 (s100 , t10 ; 2 )
and
U1 (s100 , t100 ; 2 ) U1 (s10 , t100 ; 2 ).
Thus,
U1 (s10 , t10 ; 2 ) U1 (s10 , t100 ; 2 ) U1 (s100 , t10 ; 2 ) U1 (s100 , t100 ; 2 ).
Since,
U1 (s1 , t10 ; 2 ) U1 (s1 , t100 ; 2 ) = (t10 t100 ) Pr{t2 : 2 (t2 ) < s1 }
we have
(t10 t100 ) Pr{t2 : 2 (t2 ) < s10 } (t10 t100 ) Pr{t2 : 2 (t2 ) < s100 },
i.e.,
 
(t10 t100 ) Pr{t2 : 2 (t2 ) < s10 } Pr{t2 : 2 (t2 ) < s100 } 0.
Suppose t10 > t100 . Then, Pr{t2 : 2 (t2 ) < s10 } Pr{t2 : 2 (t2 ) <
s100 }.If s10 < s100 , then Pr{t2 : s10 2 (t2 ) < s100 } = 0. That is, 2 does
not stop between s10 and s100 .
The argument to this point has only used the property that 1 is
a best reply to 2 . To complete the argument, we appeal to the fact
that s1 s2 (an implication of 2 being a best reply to 1 ), which
implies Pr{2 (t2 ) s100 } > 0, and so stopping earlier (at a time
s1 (s10 , s100 )) is a profitable deviation for t100 . Thus, s10 = 1 (t10 )
s100 = 1 (t100 ).
January 18, 2016 67

It can also be shown that in any Nash eq, i is a strictly increas-

ing and continuous function. Thus,

U1 (s1 , t1 ; 2 )
Z
 1

= s1 Pr t2 2 (s1 ) + (t1 2 (t2 )) f2 (t2 ) dt2
{t2 <21 (s1 )}
Z 21 (s1 )
1



= s1 1 F2 2 (s1 ) + (t1 2 (t2 )) f2 (t2 ) dt2 .
0

Assuming 2 is, moreover, differentiable, the first order condition

is



0 = 1 F2 21 (s1 )
1

d21 (s1 ) 1

d21 (s1 )
+ s1 f2 2 (s1 ) + (t1 s1 ) f2 2 (s1 ) .
ds1 ds1
But
d21 (s1 ) 

= 1 20 21 (s1 ) ,
ds1
so 



1 F2 21 (s1 ) 20 21 (s1 ) = t1 f2 21 (s1 ) ,
i.e.,

1

t 1 f2 2 (s1 )
20 21 (s1 ) = 
.
1 F2 21 (s1 )
Suppose F1 = F2 and suppose the eq is symmetric, so that 1 =
2 = , and s1 = 1 (t) = t = 21 (s1 ). Then,

tf (t)
0 (t) = .
1 F (t)
Since (0) = 0, Zt
f ()
(t) = d.
0 1 F ()
If f (t) = e , then F (t) = 1 et , and
t

Zt
e
(t) = d = t 2 /2.
0 e

68 CHAPTER 3. GAMES WITH NATURE

Note that (t) > t for t > 2!

If we extend the strategy space to allow for never stopping, i.e.,
Si = R+ {} and allow payoffs to take on the value , then there
are also two asymmetric equilibria, in which one player drops out
immediately, and the other never drops out.

Example 3.3.5 (Double Auction). Let vs U [0, 1], vb U [0, 1], vs

and vb independent. If sale occurs at price p, buyer receives vb p,
seller receives p vs . Seller and buyer simultaneously propose

1
prices ps [0, 1] and pb [0, 1] respectively. Trade at 2 ps + pb
if ps pb ; otherwise no trade.
Buyers strategy is a function p eb : [0, 1] [0, 1], sellers strategy
is a function p es : [0, 1] [0, 1]. We check for interim optimality
(i.e., optimality of a strategy conditional on a type). (With a con-
tinuum of strategies, ex ante optimality formally requires only that
the strategy is optimal for almost all types.)
Fix sellers strategy, p es : [0, 1] [0, 1], buyers valuation vb and
his bid pb . Buyers (conditional) expected payoff is:
Z  

1

Ub pb , vb ; pes , = vb p
es (vs ) + pb dvs
2
{vs :pb pes (vs )}


= Pr vs : pb p es (vs )
 
1 1 

vb pb E p 
es (vs ) vs : pb pes (vs ) .
2 2

Suppose that sellers strategy is linear in his valuation, i.e. p

es (vs ) =
as + cs vs , with as 0, cs > 0 and as + cs 1. Then,
 

pb a s
Pr vs : pb p es (vs ) = Pr vs : vs .
cs

So,

if pb as ,

0,

pb as
Pr vs : pb p
es (vs ) = , if as pb as + cs ,
cs

1, if pb as + cs ,
January 18, 2016 69

and so
  




 pb as
E p
es (vs ) vs : pb p
es (vs ) = as + cs E vs  vs : vs
cs

(and if as pb as + cs )

1 pb as pb + as
= as + cs = .
2 cs 2
So,


not defined, if pb < as ,

pb +as
es (vs )  vs : pb p
E p es (vs ) = , if as pb as + cs ,

2
a + 1 c , if pb as + cs .
s 2 s

and



0,   if pb as ,

pb as 3 1
Ub pb , vb ; p
es , = vb 4 pb 4 as , if as pb as + cs ,

cs
v 1 p 1 a 1 c , if pb as + cs .
b 2 b 2 s 4 s

The interior expression is maximized by solving the FOC

 
1 3 1 3 3
0= vb pb as pb + as ,
cs 4 4 4 4
and so
6 as 3
pb =vb + as
4 4 4
1
=vb + as
2
1 2
pb = a s + v b .
3 3
Thus any bid less than as is optimal if vb as , 13 as + 23 vb is the
unique optimal bid if as < vb as + 32 cs and as + cs is the unique
eb (vb ) = 31 as + 23 vb is a
optimal bid if vb as + 23 cs . Thus strategy p
best response to pes (vs ) = as + cs vs as long as 1 as + 23 cs .
70 CHAPTER 3. GAMES WITH NATURE

L R L R
T 1, 1 0, 0 T 1, x 0, 1
B 0, 1 1, 0 B 0, 0 1, 1

Figure 3.4.1: The payoff matrices for Example 3.4.1.

Symmetric argument shows that if p eb (vs ) = ab + cb vb , then

es (vs ) = 32 vs + 13 (ab + cb ). Thus a
sellers optimal bid (if interior) is p
linear equilibrium must have ab = 13 as , as = 13 (ab + cb ), cb = 23 and
cs = 23 , so as = 14 and ab = 12 1
. There is then a linear equilibrium
with:

1 2
p
es (vs ) = + vs
4 3
1 2
p
eb (vb ) = + vb
12 3
Efficient trade requires trade if vs < vb , and no trade if vs > vb .
Under linear equilibrium, trade occurs iff p es (vs ) p
eb (vb ), which
requires
1
vs + vb .
4
n o
1
Thus, for valuations in the set (vs , vb ) | vs < vb < vs + 4 , trade is
efficient but does not occur in equilibrium.
Note: there are other equilibria (see Problem 3.6.7).

3.4 Games of Incomplete Information

Example 3.4.1. Suppose payoffs of a two player two action game
are given by one of the two bimatrices in Figure 3.4.1.
Suppose first that x = 0. Either player II has dominant strategy
to play L or a dominant strategy to play R. Suppose that II knows
January 18, 2016 71

his own payoffs but player I thinks there is probability that pay-
offs are given by the left matrix, probability 1 that they are
given by the right matrix. Say that player II is of type 1 if payoffs
are given by the left matrix, type 2 if payoffs are given by the right
matrix. Clearly equilibrium must have: II plays L if II is of type 1,
R if II is of type 2; I plays T if > 21 , B if < 21 .
But now suppose x = 2. Should player II still feel comfortable
playing B if he is of type 2? The optimality of IIs action choice
of B depends on his believing that I will play T , which only occurs
if I assigns probability at least 12 to the right matrix. But suppose
II does not know Is beliefs . Then, II has beliefs over Is beliefs
1
and so II finds R optimal if he assigns probability at least 2 to I
1
assigning probability at least 2 to the right matrix.
But, we are not done: how confident is I that II will play R in the
right matrix? Now we need to deal with Is beliefs over IIs beliefs.
Player I will only play B if he assigns probability at least 21 to II
assigning probability at least 21 to I assigning probability at least 21
to the right matrix. And so on.
This leads us into an infinite regress, since now Is beliefs about
IIs beliefs about Is beliefs become relevant! So, how to analyze
such problems in general?

Definition 3.4.1 (Harsanyi). A 

game of incomplete information or
n
Bayesian game is the collection (Ai , Ti , pi , ui )i=1 , where
Ai is is action space,
Ti is is type space,
Q 
pi : Ti T
ji j is is subjective beliefs about the other
players types, given is type and
Q Q
ui : j Aj j Tj R is is payoff function.

A strategy for i is
si : Ti Ai .
Let s(t) (s1 (t1 ), . . . , sn (tn )), etc.
72 CHAPTER 3. GAMES WITH NATURE

Definition 3.4.2. The profile (s1 , . . . , sn ) is a Bayesian (or Bayes-

Nash) equilibrium if, for all i and all ti Ti ,

Eti [ui (s(t), t)] Eti [ui (ai , si (ti ), t)], ai Ai , (3.4.1)

where the expectation over ti is taken with respect to the probability

distribution pi (ti ).

If the type spaces Q are finite, then the probability i assigns to

the vector ti ji Tj Ti when his type is ti can be denoted
pi (ti ; ti ), and (3.4.1) can be written as
X X
ui (s(t), t)pi (ti ; ti ) ui (ai , si (ti ))pi (ti ; ti ), ai Ai .
ti ti

Example 3.4.2 (Revisiting Example 3.1.1). The Cournot game is rep-

resented as a game of incomplete information, as follows: The ac-
tion spaces are Ai = R+ . Firm 1s type space is T1 = {t10 , t100 } while
firm 2s type space is a singleton T2 = {t2 }. The belief mapping p1
for firm 1 is trivial: both types assign probability one to the type
t2 (since T2 is a singleton, there is no alternative), while the belief
mapping for firm 2 is

p2 (t2 ) = t10 + (1 ) t100 (T1 ).

Finally, payoffs are

(
[(a q1 q2 ) cH ]q1 , if t1 = t10 ,
u1 (q1 , q2 , t1 , t2 ) =
[(a q1 q2 ) cL ]q1 , if t1 = t100 ,

and

u2 (q1 , q2 , t1 , t2 ) = [(a q1 q2 ) c2 ]q2 .

In this example, it is of course more natural to denote the type t10

by cH and t100 by cL . See Problem 3.6.6 for a version of this game
capturing higher order beliefs of the kind hinted at in the end of
Example 3.4.1.
January 18, 2016 73

Remark 3.4.1. The idea that games of incomplete information (as

defined in Definition 3.4.1) formally capture hierarchies of beliefs
can be made precise, and leads to the notion of the universal types
space (which describes all possible hierarchies). This is clearly well
beyond the scope of this course. For a (relatively) gentle introduc-
tion to this, see Myerson (1991, Section 2.9).

Definition 3.4.3. The subjective beliefs are consistent or are said to

satisfy the Common Prior Assumption
Q
( CPA) if there exists a prob-
ability distribution p i Ti such that pi (ti ) is the probability
distribution on Ti conditional on ti implied by p.
If the type spaces are finite, this is equivalent to
p (t)
pi (ti ; ti ) = p(ti |ti ) = P 0

.
0 p t
ti i , ti

If beliefs are consistent, Bayesian game can be interpreted as

having an initial move by nature, which selects t T according
to p. The Common Prior Assumption is controversial, sometimes
viewed as a mild assumption (Aumann, 1987) and sometimes not
(Gul, 1998). Nonetheless, in applications it is standard to assume
it.
For simplicity of notation only, suppose type spaces are finite.
Viewed as a game of complete information, a profile s is a Nash
equilibrium if, for all i,
X X
ui (s(t), t)p(t) ui (si (ti ), si (ti ), t)p(t), si : Ti Ai .
t t
P
This inequality can be rewritten as (where pi (ti ) ti p (ti , ti ))

X X
ui (s (t) , t) pi (ti ; ti ) pi (ti )

ti ti

X X
ui (si (ti ) , si (ti ) , t) pi (ti ; ti ) pi (ti ) ,

ti ti

si : Ti Ai .
74 CHAPTER 3. GAMES WITH NATURE

A B
A , 9, 5
B 5, 9 7, 7

Figure 3.5.1: The game for Example 3.5.1. For = 9, this is the game
studied in examples 3.2.1 and 2.5.5.

If pi (ti ) 0, this is then equivalent to the definition of a Bayesian

eq.

3.5 Higher Order Beliefs and Global Games

Example 3.5.1. Consider the game in Figure 3.5.1. For = 9, this is
the game studied in examples 3.2.1 and 2.5.5.
Suppose, as in example 3.2.1, that there is incomplete infor-
mation about payoffs. However, now the information will be cor-
related. In particular, suppose {4, 9}, with prior probability
Pr { = 9} > 7/9.
If players have no information about , then there are two pure
strategy Nash eq, with (A, A) Pareto dominating (B, B).
Now suppose players have some private information represented
as follows: Underlying state space is {1 , 2 , . . . , K }, K odd.
In all states k , k K 1, = 9, while in state K , = 4. Player
1 has a partition on given by {{1 } , {2 , 3 } , . . . , {K1 , K }};
we denote {1 } by t10 , and {2` , 2`+1 } by t1` . Player 2 has a parti-
tion {{1 , 2 } , {3 , 4 } , . . . , {K2 , K1 } , {K }}, and we denote
(K+1)/2
{2`1 , 2` } by t2` and {K } by t2 . Finally, the probability dis-
tribution on is uniform. Figure 3.5.2 illustrates for K = 9.
Say that a player knows if the player assigns probability 1 to a
particular value of . Suppose for illustrative purposes that K = 9.
So, player 2 knows at every , while player 1 only knows for
{8 , 9 }. Write Ki E if player i knows the event E. Then, at
9 , we only have K2 { = 4}, and at 8 , we only have K2 { = 9}. At
7 , we have K1 { = 9} and K2 { = 9} (both players know = 9)
January 18, 2016 75

!0
!1 !2
!3 !4
!5 !6
!7 !8
!9

Figure 3.5.2: The solid lines describe player 1s information sets, while the
dashed lines describe player 2s. Player 1 knows the row and
player 2 knows the column containing the realized state.
In state 7 , both players know = 9, player 1 knows that
player 2 knows that = 9, but player 2 assigns probability
1/2 to 8 , and so to the event that player 1 does not know
that = 9.
In state 6 , both players know = 9, both players know
that both players know that = 9, player 2 knows that player
1 knows that player 2 knows that = 9, but player 1 does
not know that player 2 knows that player 1 knows that = 9.
76 CHAPTER 3. GAMES WITH NATURE

and K1 K2 { = 9} (player 1 knows that player 2 knows = 9), but

no more. Note that moving from 8 to 7 does not change what
player 2 knows, since both states are in the same element of player
2s partition. As a final illustration, consider 6 . Since 6 and 7
are in the same element of 1s partition, player 1s knowledge does
not change, and so for 1 we only have K1 { = 9} and K1 K2 { = 9}.
But player 2 knows what player 1 knows at 6 , so in addition to
K2 { = 9}, we also have K2 K1 { = 9} and K2 K1 K2 { = 9}.
A pure strategy for player 1 is
(K1)/2
s1 : {t10 , t11 , . . . , t1 } {A, B},
while a pure strategy for player 2 is
(K+1)/2
s2 : {t21 , t22 , . . . , t2 } {A, B}.
Before we begin examining eq, note that Pr { = 9} = (K 1) /K,
and this converges to 1 as K .
This game of asymmetric information has a unique equilibrium
(the idea is similar to Rubinsteins (1989) email game).
Claim 3.5.1. This game has a unique Nash equilibrium (s1 , s2 ), and
in this equilibrium, both players necessarily choose B, i.e., s1 (t1 ) =
s2 (t2 ) = B for all t1 and t2 .
Proof. (by induction) Let s be a Nash equilibrium. Note first that
(K+1)/2 (K+1)/2
s2 (t2 ) = B(since
 B is the unique best response at t2 ).
`+1
Suppose s2 t2 = B for ` > 1. Then, since
n o
`+1 `
Pr t2 |t1 = Pr {{2`+1 , 2`+2 } | {2` , 2`+1 }}
Pr {2`+1 } 1/K 1
= = = ,
Pr {2` , 2`+1 } 2/K 2
 
s1 t1` = B (the probability that 2 plays B is at least 1/2). Moreover,
 
if s1 t1` = B, then since
n o
` `
Pr t1 |t2 = Pr {{2` , 2`+1 } | {2`1 , 2` }}
Pr {2` } 1
= = ,
Pr {2`1 , 2` } 2
January 18, 2016 77

 
we also have s2 t2` = B.

The proof actually proves something a little stronger, that only

profile that survives the iterated deletion of strictly dominated strate-
gies involves both players always choosing B. Since B is the unique
(K+1)/2 (K+1)/2
best response at t2 , any strategy s20 satisfying s20 (t2 )=A
0 0 (K+1)/2
is strictly dominated by the strategy s2 given by s2 (t2 ) = B
and s20 (t2 ) = s 0 (t2 ) for all other t2 . We now proceed by iteratively
deleting strictly dominated strategies.

Remark 3.5.1. As another illustration of the explicit representation

of games with the incomplete information, we can represent the
game of Example 3.5.1 as follows. The action spaces for both play-
(K1)/2
ers are {A, B}. The types spaces are T1 = {t10 , t11 , . . . , t1 } and
1 2 (K+1)/2
T2 = {t2 , t2 , . . . , t2 }. From the proof of Claim 3.5.1, the belief
mappings are
(1 1
` t2` + 2 t2`+1 , if ` 1,
p1 (t1 ) = 2
1 t21 , if ` = 0,

and
(1 1
t1`1 + 2 t1` , if ` < (K + 1)/2,
p2 (t2` ) = 2
(K1)/2
1 t2 , if ` = (K + 1)/2.

Finally, payoffs are given by, for i = 1, 2,

(K+1)/2

9, if a = AA, and t2 t2 ,



(K+1)/2

4, if a = AA, and t2 = t2 ,


0, if ai = A, aj = B, and t2 t2
(K+1)/2
,
ui (a1 , a2 , t1 , t2 ) = (K+1)/2

5, if ai = A, aj = B, and t2 = t2 ,






5, if ai = B, aj = A,

7, if a = BB.

78 CHAPTER 3. GAMES WITH NATURE

The complete information version of the game with = 9 has

two strict equilibria. None the less, by making a small perturbation
to the game by introducing a particular form of incomplete infor-
mation, the result is stark, with only BB surviving, even in a state
like 1 , where both players know the state to an some large finite
order.
Carlsson and van Damme (1993) introduced the term global games
to emphasize the importance of viewing the benchmark complete
information game in a broader (global), i.e., perturbed, context. The
term global game is now commonly understood to refer to a mod-
elling that incorporates both a richness assumption on the uncer-
tainty (so that each action is dominant for at least one value of the
uncertainty) and small noise (as illustrated next).
Example 3.5.2 (Global Games). The stage game is as in example
3.5.1. We change the information structure: We now assume is
uniformly distributed on the interval [0, 20]. For < 5, B is strictly
dominant, while if > 16, A is strictly dominant.
Each player i receives a signal xi , with x1 and x2 independently
and uniformly drawn from the interval [ , + ] for > 0. A
pure strategy for player i is a function

si : [, 20 + ] {A, B}.

First observe that, for xi [, 20 ], player is posterior on

is uniform on [xi , xi + ]. This is most easily seen as follows:
Letting g be the density of and h be the density of x given , we
1
immediately have g() = 20 for all [0, 20] and
(1
, if x [ , + ],
h(x | ) = 2
0, otherwise.

Since
f (x, )
h(x | ) = ,
g()
where f is the joint density, we have
( 1
, if x [ , + ] and [0, 20],
f (x, ) = 40
0, otherwise.
January 18, 2016 79

The marginal density for x [, 20] is thus simply the constant

1
function 20 , and so the density of conditional on an x [, 20]
1
is the constant function 2 on the interval [x , x + ].
Similar considerations show that for xi [, 20 ], player is
posterior on xj is symmetric around xi with support [xi 2, xi +
2]. Hence, Pr{xj > xi | xi } = Pr{xj < xi | xi } = 12 .
5
Claim 3.5.2. For < 2 , the game has an essentially unique Nash
equilibrium (s1 , s2 ), given by
(
A, if xi 10 12 ,
si (xi ) =
B, if xi < 10 12 .

Proof. We again apply iterated deletion of dominated strategies.

Suppose xi < 5 . Then, player is conditional expected payoff
from A is less than that from B irrespective of player js action,
and so i plays B for xi < 5 (as does j for xj < 5 ). But then at
xi = 5 , since < 5 , player i assigns at least probability 21 to
j playing B, and so i strictly prefers B. Let xi be the largest signal
for which B is implied by iterated dominance (.e., xi = sup{xi0 | B
is implied by iterated strict dominance for all xi < xi0 }). By sym-
metry, x1 = x2 = x . At xi = x , player i cannot strictly prefer
B to A (otherwise, we can expand the set of signals for which iter-
ated dominance implies B), and he assigns at least probability 12 to
1
j playing B. Hence, x 10 2 .
Similarly, for xi > 16, player is conditional expected payoff
from A is greater than that from B irrespective of player js ac-
tion, and so i plays A for xi > 16 (as does j for xj > 16). Let xi
be the smallest signal for which A is implied by iterated dominance
(.e., xi = inf{xi0 | A is implied by iterated strict dominance for all
xi > xi0 }). By symmetry, x1 = x2 = x . At xi = x , player i
cannot strictly prefer A to B, and he assigns at least probability 12
to j playing A. Hence, x 10 12 .
But then
1 1
10 x x 10 .
2 2
80 CHAPTER 3. GAMES WITH NATURE

The iterated deletion argument connecting xi in the dominance

regions to values not in the dominance regions is often called an
infection argument.
This idea is not dependent on the particular distributional as-
sumptions made here. See Morris and Shin (2003) for details.

Remark 3.5.2 (CAUTION). Some people have interpreted the global

games literature as solving the multiplicity problem, at least in
some settings. There is in fact a stronger result: Weinstein and
Yildiz (2007) show that almost all games have a unique rationaliz-
able outcome (which of course implies a unique Nash equilibrium)!
Does this mean that we dont need to worry about multiplicity?
Of course not: This is a result about robustness. The uniqueness of
the rationalizable outcome is driven by similar ideas to that in ex-
ample 3.5.1almost all simply means that all information struc-
tures can be approximated by information structures allowing an
infection argument. In order for a modeler to be confident that he
knows the unique rationalizable outcome, he needs to be confident
of the information structure.
Indeed, the stronger result in Weinstein and Yildiz (2007) proves
even more: for almost all games, the following is true: Fix any
rationalizable action a0i . There are games close by (terms of players
beliefs) for which that action is the unique rationalizable action
for player i. This means that, in this setting at least, there are no
robust refinements of rationalizability: If there were, then some
rationalizable action a0i would have failed the refinement, and yet
it must pass the refinement for some games close by (since it is
the only rationalizable action on those games). In particular, Nash
equilibrium is not a robust concept, at least in this belief setting!

Remark 3.5.3. Superficially, purification and global games appear

very similar. In both cases, we take a benchmark game, and con-
sider perturbations in which players have small amounts of private
information. Yet the conclusions are very different: With Harsanyi
purification, given any equilibrium (pure or mixed), we obtain es-
sentially strict equilibria of the perturbed game that approximate
 January 18, 2016 81

A B
A 9, 9 0, 5
B 4, 0 7, 7

Figure 3.6.1: The game for Problem 3.6.2.

that equilibrium. With global games, given a benchmark game with

multiple equilibria, we obtain a unique essentially strict equilibrium
of the perturbed game. This difference arises from the private
information in purification being independently distributed (one
players type is uninformative about the realized type of the other
player), while this is not true for global games (one players type can
be very informative about the realized type of the other player).

3.6 Problems
3.6.1. There are two firms, 1 and 2, producing the same good. The inverse
demand curve is given by P = q1 q2 , where qi R+ is firm
is output. (Note that we are allowing negative prices.) There is de-
mand uncertainty with nature determining the value of , assigning
probability (0, 1) to = 3, and complementary probability 1
to = 4. Firm 2 knows (is informed of) the value of , while firm 1
is not. Finally, each firm has zero costs of production. As usual, as-
sume this description is common knowledge. Suppose the two firms
choose quantities simultaneously. Define a strategy profile for this
game. Describe the Nash equilibrium behavior and interim payoffs
(which may be unique).

3.6.2. Redo Example 3.2.1 for the game in Figure 3.6.1.

3.6.3. Consider the following variant of a sealed bid auction in a setting

of independent private values. The highest bidder wins, and pays
a price determined as the weighted average of the highest bid and
second highest bid, with weight (0, 1) on the highest bid (ties are
82 CHAPTER 3. GAMES WITH NATURE

resolved by a fair coin). Suppose there are two bidders, with bidder
is value vi randomly drawn from the interval [vi , vi ] according to
the distribution function Fi , with density fi .

(a) What are the interim payoffs of player i?

(b) Suppose (1 , 2 ) is a Nash equilibrium of the auction, and as-
sume i is a strictly increasing and differentiable function, for
i = 1, 2. Describe the pair of differential equations the strate-
gies must satisfy.
(c) Suppose v1 and v2 are uniformly and independently distributed
on [0, 1]. Describe the differential equation a symmetric in-
creasing and differentiable equilibrium bidding strategy must
satisfy.
(d) Solve the differential equation found in part 3.6.3(c). [Hint:
Conjecture a functional form.]
(e) For the assumptions under part 3.6.3(c), prove the strategy
found in part 3.6.3(d) is a symmetric equilibrium strategy.

3.6.4. Consider the two bidder sealed bid auction of Example 3.3.1, when
the two bidders valuations are independent uniform draws from the
interval [0, 1].

(a) What is the expected revenue of the symmetric equilibrium?

In a first price sealed bid auction with reserve price r , the highest
bidder only wins the auction (and pays his bid) if the bid is at least
r.

(b) What is the symmetric Nash equilibrium of the first price sealed
bid auction with reserve price r [0, 1]? [Hint: Since any win-
ning bid must be no less than r , bidders with a value less than
r can be assumed to bid 0 (there are multiple symmetric equi-
libria that only differ in the losing bids of bidders with values
less than r ). It remains to determine the bidding behavior of
those bidders with values r . Prove that any bidder with value
r must bid r . While the equilibrium is not linear in valuation,
the recipe from Example 3.3.1 still works.]
(c) What reserve price maximizes expected revenue?

3.6.5. This question asks you to prove a claim made in Example 3.3.4 as
follows:
 January 18, 2016 83

(a) Suppose s2 < s1 , and set Pr{s2 < 1 (t1 ) s1 } > 0. Prove
that there exists s1 satisfying Pr{1 (t1 ) > s1 } < /2. [Hint: This
is trivial if s1 < (why?). The case where s1 = uses a basic
continuity property of probability.]
(b) Show that a deviation by type t2 > 2s1 to a stopping time s2 > s1
(which implies that t2 wins the war of attrition with probability
of at least /2) satisfying s2 < t2 /2 is strictly profitable.

3.6.6. This question concerns Example 3.1.1, the Cournot game with in-
complete information. The idea is to capture the possibility that
firm 2 may know that firm 1 has low costs, cL . This can be done as
follows: Firm 1s space of uncertainty (types) is, as before, {cL , cH },
while firm 2s is {tL , tU }. Nature determines the types according to
the distribution


,
if (t1 , t2 ) = (cL , tL ),
Pr(t1 , t2 ) = , if (t1 , t2 ) = (cL , tU ),


1 , if (t , t ) = (c , t ),
1 2 H U

where , (0, 1) and 1 > 0. Firm 2s type, tL or tU , does not

affect his payoffs (in particular, his cost is c2 , as in Example 3.1.1).
Firm 1s type is just his cost, c1 .

(a) What is the probability firm 2 assigns to c1 = cL when his type

is tL ? When his type is tU ?
(b) What is the probability firm 1 assigns to firm 2 knowing firm
1s cost? [This may depend on 1s type.]
(c) What is the Nash equilibrium of this game. Compare your anal-
ysis to that of Example 3.1.1.

3.6.7. The linear equilibrium of Example 3.3.5 is not the only equilibrium
of the double auction.

(a) Fix a price p (0, 1). Show that there is an equilibrium at

which, if trade occurs, then it occurs at the price p.
(b) What is the probability of trade?

(c) At what p is the probability of trade maximized?

 84 CHAPTER 3. GAMES WITH NATURE

(d) Compare the expected gains from trade under these fixed price
equilibria with the linear equilibrium of Example 3.3.5.

3.6.8. In Example 3.5.1, suppose that in the state 1 , = 20, while in

states k , 2 k K 1, = 9 and in state K , = 4. Suppose
the information partitions are as in the example. In other words,
apart from the probability distribution over (which we have not
yet specified), the only change is that in state 1 , = 20 rather than
9.

(a) Suppose the probability distribution over is uniform (as in

the lecture notes). What is the unique Nash equilibrium, and
why? What is the unconditional probability of both players
choosing A in the equilibrium?
(b) Suppose now the probability of k is 410k , for k = 1, . . . , 9,
P9
where is chosen so that k=1 410k = 1. What is the unique
Nash equilibrium, and why? What is the unconditional proba-
bility of both players choosing A in the equilibrium

3.6.9. This question asks you to fill in some of the details of the calcula-
tions in Example 3.5.2.

(a) What is the marginal density of x for x [, 20 + ]?

(b) What is expected value of conditional on x, for x [, 20 +
]?
(c) Derive player is posterior beliefs about player js signal xj ,
conditional on xi [, 20 + ].
Chapter 4

Existence and Foundations for

Nash Equilibrium1

4.1 Existence of Nash Equilibria

Recall Nash eq are fixed points of best reply correspondence:

s (s ).

When does have a fixed point?

Theorem 4.1.1 (Kakutanis fixed point theorem). Suppose X Rm

for some m and F : X X. Suppose

1. X is nonempty, compact, and convex;

2. F has nonempty convex-values (i.e., F (x) is a convex set and

F (x) x X); and

3. F has closed graph: (x k , x k ) (x, x), x k F (x k ) x F (x).

Then F has a fixed point.

Remark 4.1.1. A correspondence F : X X is upperhemicontinous

at x X if for all open sets O Rm satisfying F (x) O, there
exists such that for all x 0 X satisfying |x x 0 | < , F (x 0 ) O.
1
Copyright January 18, 2016 by George J. Mailath

85
86 CHAPTER 4. NASH EQUILIBRIUM

A correspondence F : X X is upperhemicontinous if it is upper-

hemicontinous at all x X. A singleton-valued correspondence is
a function, and the notion of upperhemicontinuity for singleton-
valued correspondences is precisely that of continuity of the func-
tion.
An upperhemicontinuous correspondence with closed values has
a closed graph.2 Unlike upperhemicontinuity, the closed graph
property by itself does not imply continuity of functions: the func-
tion f : R R given by f (x) = 1/x for x > 0 and f (0) = 0 has a
closed graph but is not continuous. But if F has a closed graph and
X is compact, then F is upperhemicontinuous; for more on this, see
Ok (2007, E.2).
So, the hypotheses of Theorem 4.1.1 could have been written as
F is closed valued and upperhemicontinuous. See Problem 4.3.1 for
an illuminating example.

Theorem 4.1.2. Given a normal form game G = {(Si , ui ) : i =

1, . . . , n}, if for all i,

1. Si is a nonempty, convex, and compact subset of Rk for some k,

and

2. ui : S1 Sn R is continuous in s S1 Sn and
quasiconcave in si ,

then G has a Nash equilibrium strategy profile.

Proof. Since ui is continuous, the Maximum Theorem (MWG Theo-

rem M.K.6) implies that i has a closed graph.
The quasiconcavity of ui implies that i is convex-valued: For
fixed si Si , suppose si0 , si00 arg max ui (si , si ). Then, from the
quasiconcavity of ui , for all [0, 1],

ui (si0 + (1 )si00 , si ) min{ui (si0 , si ), ui (si00 , si )},

2
The correspondence F (x) := (0, 1) is trivially upperhemicontinous, does not
have closed-values, and does not have a closed graph.
January 18, 2016 87

and so
ui (si0 + (1 )si00 , si ) max ui (si , si )
so that si0 + (1 )si00 arg max ui (si , si ).
The theorem then follows from Kakutanis fixed point theorem
by taking X = S1 Sn and F = (1 , . . . , n ).
Theorem 4.1.3 (Nash). Every finite normal form game, (Si , ui )i , has
a mixed strategy Nash equilibrium.
Q
Proof. Define Xi = (Si ) and X i Xi . Then X R|Si | and X is
nonempty, convex and compact.
In this case, rather than appealing to the maximum theorem, it
is an easy (and worthwhile!) exercise to prove that i is convex-
valued and has a closed graph.
In some applications, we need an infinite dimensional version of
Kakutanis fixed point theorem.
Theorem 4.1.4 (Fan-Glicksberg fixed point theorem). Suppose X is
a nonempty compact convex subset of a locally convex Hausdorff
space, and suppose F : X X. Suppose

1. F has nonempty convex-values (i.e., F (x) is a convex set and

F (x) x X); and
2. F has closed graph: (x k , x k ) (x, x), x k F (x k ) x F (x).

Then F has a fixed point.

In particular, every normed vector space is locally convex Haus-
dorff. Locally convex Hausdorff spaces generalize many of the nice
properties of normed vector spaces. This generalization is needed
in the following theorem, since the spaces are typically not normed.
Theorem 4.1.5. Suppose X is a compact metric space. Then the
space of probability measures on X is a nonempty compact convex
subset of a locally convex Hausdorff
R space. Moreover, if f : X R is
a continuous function, then f d is a continuous function of .
Corollary 4.1.1. Suppose Si is a compact subset of a finite dimen-
sional Euclidean space Rmi , and suppose ui : S R is continuous.
Then {(Si , ui )i } has a Nash equilibrium in mixed strategies.
88 CHAPTER 4. NASH EQUILIBRIUM

Proof. The proof mimics that of Theorem 4.1.3.

Example 4.1.1 (An example of mixed strategies). Return to first
price sealed bid auction with independent private values (Example
3.3.1), but assume that the value for each bidder is drawn indepen-
dently from a uniform distribution on the two point set {v, v}, with
v < v. This game has no equilibrium in pure strategies(see prob-

lem 4.3.6). Even though the game has discontinuous payoffs (why?),
it does have an equilibrium in mixed strategies. It is also worth not-
ing that rather than working directly with mixed strategies (which
are randomizations over pairs of bids (b0 , b00 ), where b0 is the bid
of v and b00 is the bid of v), it is convenient to work with behavior

strategies. 3
In a behavior strategy, the randomizations over bids
are specified separately for the bidder of value v and v.
Suppose bidder 2 follows the strategy 2 of bidding v if v2 = v
(why is that a reasonable guess?), and of bidding according
to the
distribution function F2 (b) if v2 = v. Then, assuming there are no
atoms in F2 , player 1 has interim payoffs from b > v given by

1 1
U1 (b, v, 2 ) = (v b) + (v b)F2 (b)
2 2
1
= (v b)(1 + F2 (b)).
2
Note that the minimum of the support of F2 is given by v (why?).
Denote the maximum of the support by b.
Suppose 1 is also randomizing over the set of bids, (v, b]. The
indifference condition requires that 1 is indifferent over all b
(v, b]. The bid b = v is excluded because there is a positive prob-

ability
of a tie at v (from the low value bidder) and so it cannot be

optimal for v to bid v. That is, for all > 0,

1
(v b)(1 + F2 (b)) = U1 (v + , v, 2 ),
2
and
1
U1 (v + , v, 2 ) = (v v )(1 + F2 (v + )).
2
3
Moreover, behavior strategies are also more natural when viewing the game
as one of incomplete information.
January 18, 2016 89

Since lim0 F2 (v + ) = F2 (v) = 0 (where the first equality follows

from the continuity
of probabilities and the second equality follows
from the assumption of no atoms), and so
(v b)(1 + F2 (b)) = v v,

yielding

bv
F2 (b) = , b (v, b].
v b
Note that F2 (v) = 0.
Moreover, F2 (b) = 1 implies b = (v+v)/2. An alternative deriva-

tion of the value of b is to note that bidder 1 should receive the
1
payoff 2 (v v) from bidding b (at which 1 wins for sure, since

there are no atoms), or
1
(v v) = v b,
2
and then solving for b.
It is straightforward to verify that the symmetric profile in which
each bidder bids v if v = v, and bids according to the distribution
function F (b) = (b v)/( v b) if v = v is a Nash equilibrium.
Note that the high value bidder is not indifferent over every bid
in the support [v, b]:


v b, if b > b,

1
(v v), if b (v, b],
U1 (b, v, 2 ) = 12

(v v), if b = v,

4


0, if b < v.

See Figure 4.1.1 for a graph of this payoff function.

Remark 4.1.2 (More formal treatment of mixed strategies). For con-

tinuum action spaces (such as auctions), a mixed strategy for a
player i is a probability distribution on R (which we can denote
Fi ). Player 1s expected payoff from an action b1 is
Z
u1 (s1 , s2 )dF2 (s2 ).
90 CHAPTER 4. NASH EQUILIBRIUM

U1
1
v v)
2 (

1
v v)
4 (

v 1
v + v) v b
 2 (

Figure 4.1.1: The payoff function U1 at equilibrium for Example 4.1.1.

As an aside, note that this notation (which may not be familiar to

all of you) covers all relevant possibilities: If the mixed strategy of
player 2 has a countable support {s k } with action s k having proba-
bility 2 (s k ) > 0 (the distribution is said to be discrete in this case),
we have Z X
u1 (s1 , s2 )dF2 (s2 ) = u1 (s1 , s k )2 (s k ).
sk
P k
Note that s k 2 (s ) = 1. Any single action receiving strictly pos-
itive probability is called an atom. If the distribution function de-
scribing player 2s behavior has a density f2 , then
Z Z
u1 (s1 , s2 )dF2 (s2 ) = u1 (s1 , s2 )f2 (s2 )ds2 .

Finally, combinations of distributions with densities on part of the

support and atoms elsewhere, as well as more esoteric possibilities
(that are almost never relevant) are also covered.
Suppose F1 is a best reply for player 1 to player 2s strategy F2 .
Then
ZZ Z
u1 (s1 , s2 ) dF2 (s2 ) dF1 (s1 ) = max u1 (s1 , s2 ) dF2 (s2 ).
s1

Observe first that if F1 is discrete with support {s k } and action s k

having probability 1 (s k ) > 0 , then
ZZ XZ
u1 (s1 , s2 ) dF2 (s2 ) dF1 (s1 ) = u1 (s k , s2 ) dF2 (s2 )1 (s k ),
sk
January 18, 2016 91

and so we immediately have

XZ Z
k k
u1 (s , s2 ) dF2 (s2 )1 (s ) = max u1 (s1 , s2 ) dF2 (s2 )
s1
sk

and so, for all s k ,

Z Z
k
u1 (s , s2 ) dF2 (s2 ) = max u1 (s1 , s2 ) dF2 (s2 ).
s1

This is just the familiar statement that player 1 is indifferent over

all actions in his support, and each such action maximizes his pay-
off against F2 .
What is the appropriate version of this statement for general
F1 ? The key observation is that zero probability sets dont mat-
ter. Thus, the statement is: Let S1 be the set of actions that are
suboptimal against F2 , i.e.,
 Z Z 
S1 = s1 : u1 (s1 , s2 ) dF2 (s2 ) < max u1 (s1 , s2 ) dF2 (s2 ) .
s1

Then, the set S1 is assigned zero probability by any best response

F1 . [If such a set received positive probability under some F1 , then
F1 could not be a best reply, since expected payoffs are clearly in-
creased by moving probability off the set S1 .]
In most applications, the set S1 is disjoint from the support of
F1 , in which case player 1 is indeed indifferent over all actions in
his support, and each such action maximizes his payoff against F2 .
However, Example 4.1.1 is an example where S1 includes one point
of the support.

Example 4.1.2 (Examples of nonexistence). See Problem 4.3.5 for a

simple example of nonexistence in a game of complete information.
Nonexistence in games of incomplete information arises more
naturally. Consider the following example of a private value first
price auction with ties broken using a fair coin. Bidder 1 has value
1
3 and bidder 2 has value 3 with probability 2 and value 4 with
92 CHAPTER 4. NASH EQUILIBRIUM

v2 = 3 v2 = 4
1 1
v1 = 3 2 4
1
v1 = 4 4
0

Figure 4.1.2: The joint distribution for nonexistence in Example 4.1.2.

1
probability 2 . An intuition for nonexistence can be obtained by
observing that since there is positive probability that 2 has value
3, neither bidder will bid over 3 when their value is 3 (since he
may win). Moreover, neither bidder can randomize on bids below
3 when their value is 3 for standard reasons. But if bidder 1 bids
3, then bidder of value 4 does not have a best reply. Existence is
restored if ties are broken in favor of bidder 2.
Suppose now the joint distribution over bidder valuations is
given by the table in Figure 4.1.2 (a symmetric version of the ex-
ample in the previous paragraph). The critical feature of this joint
distribution is that the bidder with value 4 knows that the other
bidder has value 3 (and so the valuations are not independent). For
any tie breaking rule, the value 4 bidder has no best reply, and so
there is no equilibrium (in pure or mixed strategies). Existence can
only be restored by using a tie breaking rule that awards the item
to the highest value bidder. For (much) more on this, see Jackson,
Simon, Swinkels, and Zame (2002).

Existence results that replace continuity assumptions on pay-

offs with complementarity or supermodularity assumptions are of
increasing importance. For an introduction, see chapter 7 in Vohra
(2005).

4.2 Foundations for Nash Equilibrium

A good book length treatment of the general topic covered here
is Fudenberg and Levine (1998). For much more on evolution, see
Samuelson (1997) and Weibull (1995) (or Mailath (1998) for a longer
nontechical introduction).
January 18, 2016 93

Consider two players repeatedly playing a 2 2 matrix game.

Neither player knows how opponent will play.
Suppose each player is Bayesian in the following way: each be-
lieves the opponent plays the first strategy in each period with the
same fixed probability, and the prior on this probability is uniform
on [0, 1].
After each round of play, players update to posterior.
This behavior, while Bayesian, is not rational: players make as-
sumptions on opponents that they should know, by introspection,
are false.
Rational Learning: Bayesian learning with a prior and likelihood
that encompasses the truth.
Must good Bayesians be rational learners?
If so, not all Savage-Bayesians are good Bayesians.

4.2.1 Boundedly Rational Learning

Motivated by concerns over

informational (knowledge) requirements of traditional analy-

ses.

Coordination: beliefs about endogenous variables are correct.

where does information about endogenous variables (that leads
to coordinated behavior) come from?

rationality (computational) requirements and

(?) Bayesian paradigm.

Can agents formulate a sufficiently general model to en-

compass the truth? Updating on small (prior) probabil-
ity events. At which point should agents question their
view of the world?

Rationality and coordination are distinct and independent ideas.

Common thread:
94 CHAPTER 4. NASH EQUILIBRIUM

0 1 1
2

Figure 4.2.1: The dynamics for Example 4.2.1.

behavior is myopic,
dynamic analysis, focusing on asymptotic properties, and
focus on interaction of learning with evolution of system.

4.2.2 Social Learning (Evolution)

Example 4.2.1. Large population of players randomly and repeat-
edly matched (paired) to play the same game:

A B
A 1 0
B 0 1
No role identification, so the payoff represents the payoff to a
player who chooses the row action, when facing the column action.
If is fraction of population playing A, then
u(A; ) =,
and u(B; ) =1 .
Then
d 1
= >0 iff u(A; ) > u(B; ) iff > ,
dt 2
and
1
< 0 iff u(A; ) < u(B; ) iff < .
2
The dynamics are illustrated in Figure 4.2.1.
1 1
The symmetric mixed strategy equilibrium 2 A+ 2 B is unstable.
January 18, 2016 95

0 1 1
2

Figure 4.2.2: The dynamics for Example 4.2.2.

Example 4.2.2. Large population paired to play:

A B
A 1 2
B 2 1

No role identification (so that AB and BA are infeasible). If is

fraction of population playing A, then

u(A; ) = + 2(1 ) = 2 ,
and u(B; ) =2 + 1 = 1 + .

Then
1
> 0 iff 2>1+ iff > ,
2
and
1
< 0 iff 2<1+ iff < .
2
The dynamics are illustrated in Figure 4.2.2.

Let S denote a finite set of strategies in a symmetric game. In

the above examples, S = {A, B}. Payoff to playing the strategy s S
against an opponent who plays r S is u(s, r ).
State of society is (S). Expected payoff to s when state of
society is is X
u(s, ) = u(s, r ) (r ).
r
96 CHAPTER 4. NASH EQUILIBRIUM

Dynamics:
F : (S) R+ (S),
with
F ( , t 0 + t) = F (F ( , t 0 ), t).

Definition 4.2.1. A state is a rest (or stationary) point of F if

= F ( , t) t.

A rest point is asymptotically stable under F if there exists > 0

such that if | 0 |< , then F ( 0 , t) .

Assume F is continuously differentiable in all its arguments (on

boundaries, the appropriate one-sided derivatives exist and are con-
tinuous).
Interpretation: if population strategy profile is 0 at t 0 , then at
time t 0 + t it will be F ( 0 , t). Write for F ( , t)/t|t=0 .
Note that X X
(s) = 1 (s) = 0.
s s

Definition 4.2.2. F is a myopic adjustment dynamic if , s, r S

satisfying (s), (r ) > 0,

u(s, ) > u(r , ) (s) > (r ).

Theorem 4.2.1. Suppose F is a myopic adjustment dynamic.

1. If is asymptotically stable under F , then it is a symmetric

Nash equilibrium.

2. If is a strict Nash equilibrium, then is asymptotically

stable under F .

Proof. 1. Left as an exercise.

2. Suppose is a strict Nash equilibrium. Then is a pure

strategy s and u(s, s) > u(r , s) for all r s. This implies that
there exists > 0 such that for all satisfying (s) > 1 ,

u(s, ) > u(r , ), r s.

January 18, 2016 97

Suppose 1 > (s) > 1 and (r ) > 0 for all r . Myopic

P implies (s) > max{ (r ) : r s}, and so (s) >
adjustment
0 (since r S (r ) = 0).
Consider now satisfying 1 > (s) > 1 with (r ) = 0
for some r . Since is continuous in (since F is continu-
ously differentiable, including on the boundaries), (s) 0
and (s) (r ). Suppose (s) = 0 (so that (r ) 0). Then,
(r 0 ) < 0 for all r 0 satisfying (r 0 ) > 0 and so (r ) > 0, a
contradiction.
Hence, if 1 > (s) > 1 , then (s) > 0. Defining t
F ( , t) (S), this implies t (s) > 1 for all t, and so
t (s) 1 .

There are examples of myopic adjustment dynamics that do not

eliminate strategies that are iteratively strictly dominated. Stronger
conditions (such as aggregate monotonicity) are neededsee
Fudenberg and Levine (1998). These conditions are satisfied by the
replicator dynamic:
Biological model, payoffs are reproductive fitness (normalize pay-
offs so u(s, r ) > 0 for all s, r S). At the end of each period, each
agent is replaced by a group of agents who play the same strategy,
with the size of the group given by the payoff (fitness) of the agent.
Let xs (t) be the size of the population playing s in period t. Then,

xs (t + 1) = xs (t)u(s, t ),

where
xs (t) xs (t)
t (s) = P .
r xr (t) x(t)
Then, since x(t + 1) = x(t)u( t , t ),

t+1 tu(s, t )
(s) = (s) ,
u( t , t )
so the difference equation is

t+1 t u(s, t ) u( t , t )
t
(s) (s) = (s) .
u( t , t )
98 CHAPTER 4. NASH EQUILIBRIUM

Thus, t+1 (s) > (<) t (s) iff u(s, t ) > (<)u( t , t ). In continu-
ous time, this is
u(s, ) u( , )
(s) = (s) .
u( , )
This has the same trajectories as

(s) = (s)[u(s, ) u( , )].

Note that under the replicator dynamic, every pure strategy pro-
file is a rest point: if (s) = 0 then (s) = 0 even when u(s, ) >
u( , ).
Idea extends in straightforward fashion to games with role iden-
tification. In that case, we have

i (si ) = i (si )[ui (si , i ) ui (i , i )].

Example 4.2.3 (Domination). The game is:

L R
T 1, 1 1, 0
B 1, 1 0, 0

Let p t be the fraction of row players choosing T , while qt is the

fraction of column players choosing L. The replicator dynamics are

p = p(1 p)(1 q)
and q = q(1 q).

The phase diagram is illustrated in Figure 4.2.3.4 No rest point

is asymptotically stable.

Example 4.2.4 (Simplified ultimatum game). In the simplified ulti-

matum game, the proposer offer either an equal split, or a small
payment. The responder only responds to the small payment (he
must accept the equal split). The extensive form is given in Figure
4.2.4. The normal form is
4
Note that the phase diagram in Mailath (1998, Figure 11) is incorrect.
January 18, 2016 99

0 p 1

Figure 4.2.3: The phase diagram for the domination example.

row

equal split small offer

column
50
50 N Y

0 80
0 20

Figure 4.2.4: The extensive form of the simplified ultimatum game.

100 CHAPTER 4. NASH EQUILIBRIUM

N Y
equal split 50, 50 50, 50
small offer 0, 0 80, 20

Let p be the fraction of row players choosing equal split, while

q is the fraction of column players choosing N.
The subgame perfect profile is (0, 0). There is another Nash
outcome, given by the row player choosing equal division. The set
of Nash equilibrium yielding this outcome is N = {(1, q) : 3/8 q}.
The replicator dynamics are
p = p(1 p)(80q 30)
and q = 20q(1 q)(1 p).
Note that p > 0 if q > 3/8 and p < 0 if q < 3/8, while q < 0 for all
(p, q).
The phase diagram is illustrated in Figure 4.2.5. The subgame
perfect equilibrium B is the only asymptotically stable rest point.
In the presence of drift, the dynamics are now given by
1
p = p(1 p)(80q 30) + 1 ( p)
2
1
and q = 20q(1 q)(1 p) + 2 ( q).
2
For small i , with 1 << 2 , the dynamics have two asymptoti-
cally stable rest points, one near B and one near A (see Samuelson
(1997, chapter 5)).

4.2.3 Individual learning

Fix n-player finite strategic form game, G = (S, u), S S1 Sn ,
u : S Rn .
Players play G .
History ht (s 0 , ..., s t1 ) H t S t .
Assessments it : H t (Si ).
Behavior rule ti : H t (Si ).
January 18, 2016 101

0 B p 1
Figure 4.2.5: The phase diagram for the simplified ultimatum example. A
is the nonsubgame perfect equilibrium (1, 3/8), and B is the
subgame perfect equilibrium.
102 CHAPTER 4. NASH EQUILIBRIUM

Definition 4.2.3. i is myopic with respect to i if, t and ht ,

ti (ht ) maximizes ui (i , it (ht )).

Definition 4.2.4. i is adaptive if, > 0 and t, T (, t) s.t. t 0 >

0
T (, t) and ht , it (ht ) puts no more than probability on pure
0 0

0
strategies not played by i between t and t 0 in ht .

Examples:

fictitious play (play best reply to empirical dsn of history)

Cournot dynamics

exponential weighting of past plays

Rules out rationalizability-type sophisticated analysis. Notion of

adaptive does not impose any restrictions on the relative weight
on strategies that are not excluded.

Definition 4.2.5. h (s 0 , s 1 , . . .) is compatible with if sit is in the

support of ti (ht ), i and t.

Theorem 4.2.2. Suppose (s 0 , s 1 , . . .) is compatible with behavior that

is myopic with respect to an adaptive assessment.

1. There exists T s.t. s t S t T , where S is the result of the

iterative deletion of all strictly dominated strategies (= rational-
izable if n = 2).

2. If T s.t. s t = s t > T , then s is a (pure-strategy) Nash

equilibrium of G.

Proof. 1. Let Sik denote the set of player is strategies after k

rounds of deletions of strictly dominated strategies. Since S
is finite, there exists K < such that S = S K . Proof proceeds
by induction.
There exists T s.t. s t S 1 t T : Any si Si1 is not a best
reply to any beliefs, myopia implies never chosen.
Suppose T s.t. s t S k t T . If si Sik+1 , then si is not a
k
best reply to any beliefs with support in Si . But then > 0
January 18, 2016 103

s.t. si is not a best reply to any belief i (Si ) satisfying

k
i (Si ) > 1 . (Exercise: calculate the bound on .) Since
k
assessments are adaptive, T 0 > T s.t. it (ht )(Si ) > 1 for
all t > T . Since behavior is myopic, s t S k+1 t T 0 .
2. Suppose T s.t. s t = s t > T and s is not a Nash equilib-
rium of G. Then i and si0 Si s.t. ui (si0 , si
) > ui (s ). There
0
exists > 0 s.t. ui (si , i ) > ui (si , i ) if i (s ) > 1 .
But then adaptive assessments with myopic behavior implies
sit si for t large, a contradiction.

Stronger results on convergence (such as to mixed strategy equi-

libria) require more restrictions on assessments. For more, see Fu-
denberg and Levine (1998).
Convergence of beliefs need not imply imply convergence in be-
havior. For example, in matching pennies, the empirical distribu-
tion converges to ( 12 , 12 ), but players always play a pure strategy.

4.3 Problems
4.3.1. Consider the correspondence F : [1, 1] [1, 1] defined as


(0, 1) if x 0,
F (x) =

(0, x ] if x > 0.
2

(a) Prove that F is upperhemicontinuous (i.e., prove that for all

x [1, 1], for all open sets O R satisfying F (x) O, there
exists such that for all x 0 (x, x+)[1, 1], F (x 0 ) O).
(b) Does F have a fixed point?
(c) Which hypothesis of Theorem 4.1.1 fails for F ?
(d) (A simple exercise to clarify the definition of upperhemiconti-
nuity.) Consider the correspondences G : [1, 1] [1, 1] and
H : [1, 1] [1, 1] defined as

(0, 1), if x 0,
G(x) =  x 
0, , if x > 0,
2
 104 CHAPTER 4. NASH EQUILIBRIUM

and
(0, 1), if x 0,
H(x) = h x x i
, , if x > 0.
2 2

For which values of x is G not upperhemicontinuous? For

which values of x is H not upperhemicontinuous?

4.3.2. (a) Suppose f : X Y R is a continuous function and X and Y

are compact subsets of R. Prove that

max min f (x, y) min max f (x, y).

xX yY yY xX

Give an example showing that the inequality can hold strictly

(it suffices to do this for X and Y each only containing two
pointsrecall matching pennies from Section 2.4.1).
(b) von Neumanns celebrated Minmax Theorem states the follow-
ing equality: Suppose X and Y are finite sets,
P f : X Y R, and
f : (X)(Y ) R is given by f (, ) = x,y f (x, y)(x)(y).
Then,

max min f (, ) = min max f (, ).

(X) (Y ) (Y ) (X)

Prove this theorem by applying Theorem 4.1.3 to the game G

given by S1 = X, S2 = Y , u1 (s1 , s2 ) = f (s1 , s2 ), and u2 (s1 , s2 ) =
u1 (s1 , s2 ).5 (Any two-player normal form game satisfying u1 (s)
= u2 (s) for all s S is said to be zero sum.)
(c) Prove that (1 , 2 ) is a Nash equilibrium of a zero sum game
if and only if i is a security strategy for player i, and that
player is security level v i is given by is payoff in any Nash
equilibrium. (Compare with problem 2.6.13.)
(d) Prove the following generalization of Problem 2.6.13(b): Sup-
pose a two-player normal form game (not necessarily zero sum)
has a unique Nash equilibrium, and each players Nash equilib-
rium strategy and security strategy are both completely mixed.
5
von Neumanns original argument (1928), significantly predates Nashs ex-
istence theorem, and the result is true more generally. There are elementary
proofs of the minmax theorem (based on the basic separating hyperplane theo-
rem) that do not rely on a fixed point theorem. See, for example, Owen (1982,
II.4) for the finite dimensional case, and Ben-El-Mechaiekh and Dimand (2011)
for the general case.
 January 18, 2016 105

Prove that each players security level is given by his/her Nash

equilibrium payoff.

4.3.3. (a) Prove that every finite extensive form game has a subgame per-
fect equilibrium.
(b) (Harris, Reny, and Robson, 1995, Section 2.1) Verify that the
following two-stage extensive-form game does not have a sub-
game perfect equilibrium (it is worth noting that this game has
continuous payoffs and compact actions spaces). There are
four players. In stage 1, players 1 and 2 simultaneously choose
a1 [1, 1] and a2 {L, R} respectively. In stage 2, players
3 and 4 are informed of the choices of players 1 and 2, and
then simultaneously choose a3 {L, R} and a4 {L, R} re-
spectively. Payoffs are

u1 (a1 , a2 , a3 , a4 ) =



|a1 | 12 |a1 |2 , if a2 = a3 and a3 = a4 ,


|a | 1 |a |2 , if a2 a3 and a3 = a4 ,
1 2 1
|a1 | 10 1 |a1 |2 ,
if a2 = a3 and a3 a4 ,

2

|a1 | 10 1 |a1 |2 ,
2 if a2 a3 and a3 a4 ,


1, if a2 = a3 = L,



1, if a2 = L, a3 = R,
u2 (a1 , a2 , a3 , a4 ) =

2, if a2 = a3 = R,



2, if a2 = R, a3 = L,
(
a1 , if a3 = L,
u3 (a1 , a2 , a3 , a4 ) =
a1 , if a3 = R,
and (
a1 , if a4 = L,
u4 (a1 , a2 , a3 , a4 ) =
a1 , if a4 = R.

4.3.4. Consider a first price sealed bid auction with private values. There
are two bidders with values v1 < v2 . These values are common
knowledge. Prove that this auction has no pure strategy equilibrium.
Characterize the set of mixed strategy equilibria. [Hint: In these
equilibria, bidder 2 plays a pure strategy and wins with probability
1.]
 106 CHAPTER 4. NASH EQUILIBRIUM

4.3.5. Two psychologists have to choose locations on a portion of Inter-

state 5 running through California and Oregon. The relevant portion
of Interstate 5 is represented by the interval [0, 4]; the California
portion is represented by [0, 3] and the Oregon portion by [3, 4].
There is a continuum of potential clients, uniformly distributed on
the Interstate; each client patronizes the psychologist located clos-
est to him (irrespective of state). If the two psychologists are at
the same location, each receives half the market. Finally, assume
each psychologist chooses location to maximize his or her number
of clients.

(a) (The classic Hotelling location model.) Suppose both psycholo-

gists are licensed to practice in both states, and so can locate
anywhere in the interval [0, 4]. What is the pure strategy equi-
librium? Is it unique?
(b) Suppose now that one psychologist is only licensed to practice
in California (so his/her location is restricted to [0, 3]), while
the other is licensed to practice in both states. How does this
change your answer to part 4.3.5(a)?
(c) (The next two parts are based on Simon and Zame (1990).) Sup-
pose that one psychologist is only licensed to practice in Cal-
ifornia, while the other is only licensed to practice in Oregon.
Prove that this game now has no equilibrium (in either pure or
mixed strategies).
(d) Finally, maintain the licensing assumptions of part 4.3.5(c), but
suppose that when the two psychologists both locate at 3, the
3
Californian psychologist receives 4 of the market. What is the
pure strategy equilibrium?

4.3.6. This question asks you to fill in the details of Example 4.1.1.

(a) Prove that in any equilibrium, any bidder with value v must bid
v.

(b) Prove that there is no equilibrium in pure strategies.
(c) Prove that in any mixed strategy equilibrium, the minimum of
the support of F2 is given by v.

(d) Prove that it is not optimal for v to bid v.

 January 18, 2016 107

(e) Prove that the symmetric profile in which each bidder bids v
if v = v, and according to the distribution function F (b) =
v b) if v = v is a Nash equilibrium.
(b v)/(

4.3.7. A variant of Example 4.1.1. The two bidders have independent pri-
vate values drawn independently and identically from the common
three point set {v1 , v2 , v3 }, with v1 < v2 < v3 . The common dis-
tribution assigns probability pk to value vk , k = 1, 2, 3. There is a
symmetric equilibrium in which the low value bidders bid v1 , the
mid value bidders randomize over a support [v1 , b2 ], and the high
value bidders randomize over a support [b2 , b3 ]. Fully describe the
equilibrium, and verify that it is indeed an equilibrium.

4.3.8. Consider again the two bidder sealed bid auction of Example 3.3.1,
but with a different valuation structure. In particular, the two bid-
ders valuations are independent draws from the interval [1, 2], with
each bidders valuation being a uniform draw from that interval with
probability p (0, 1) and with complementary probability 1 p
equaling some v [1, 2].

(a) Suppose v = 1. What is the symmetric Nash equilibrium?

[Hint: What must v bid?]
(b) Suppose v = 2. What is the symmetric Nash equilibrium?
[Hint: Type v must randomize. Why? Can v have an atom in
his bid distribution?]
(c) Suppose v (1, 2). What is the symmetric Nash equilibrium?
(d) Compare and contrast the comparative statics with respect to
p of the equilibria for different values of v .

4.3.9. Consider the following variant of a sealed bid auction: There are two
bidders who each value the object at v, and simultaneously submit
bids. As usual, the highest bid wins and in the event of a tie, the
object is awarded on the basis of a fair coin toss. But now all bidders
pay their bid. (This is an all-pay auction.)

(a) Formulate this auction as a normal form game.

(b) Show that there is no equilibrium in pure strategies.
(c) This game has an equilibrium in mixed strategies. What is it?
(You should verify that the strategies do indeed constitute an
equilibrium).
 108 CHAPTER 4. NASH EQUILIBRIUM

4.3.10. Fill in the details of Example 4.1.2.

4.3.11. An asymmetric variant of Example 4.1.1. The two bidders have inde-
pendent private values drawn independently from the common two
point set {v, v}, with v < v. The probability that bidder i has value
1), and p < p . Ties are broken in favor of bidder 2.
v is pi (0, 1 2

(a) As in Problem 4.3.6, in any equilibrium any bidder with value v

must bid v. Prove that in any equilibrium, the support of the

bids made by bidder 1 when high value is the same as that of
bidder 2 when high value, and the minimum of the common
support is v. Denote the common support by [v, b].

(b) Prove that one of the bidders behavior strategies has an atom
at v (the other does not). What is the size of the atom?

(c) Describe the equilibrium.
(d) Suppose now that the high value bidder 1 has value v1 , while
the high value bidder 2 has value v2 , with the two values not
necessarily equal. This game only has an equilibrium if ties are
broken in the right way. What is that way?

4.3.12. This question is a variation of the Cournot duopoly of Example 1.1.4.

The market clearing price is given by P (Q) = max{a Q, 0}, where
Q = q1 + q2 is total quantity and qi is firm is quantity. There is
a constant marginal cost of production c, with 0 < c < a. Finally,
there is a fixed cost of production . Suppose (a c)2 /9 < <
(a c)2 /4.

(a) Suppose firm i only incurs the fixed cost of production when
qi > 0. Then, firm is profits are given by
(
(P (q1 + q2 ) c)qi , if qi > 0,
Ui (q1 , q2 ) =
0, if qi = 0.

This game has two pure strategy Nash equilibria. What are
they? Why doesnt this game have a symmetric pure strategy
Nash equilibrium? Since firm payoffs are not continuous func-
tions (why not?), the existence of a symmetric Nash equilibrium
in mixed strategies is not implied by any of the theorems in Sec-
tion 4.1.
January 18, 2016 109

(b) Suppose the fixed cost is an entry cost (and so is sunk). The
game is still a simultaneous move game, but now firm is strat-
egy space is {1}R+ , with the strategy 1 meaning dont en-
ter, and a number qi R+ meaning enter and choose qi 0
(note the weak inequality). Then, firm is profits are given by


(P (q1 + q2 ) c)qi , if (s1 , s2 ) = (q1 , q2 ),

Ui (s1 , s2 ) = (P (qi ) c)qi , if si = qi and sj = 1,


0, if si = 1.

The existence of a symmetric Nash equilibrium in mixed strate-

gies is implied by Theorems 4.1.4 and 4.1.5 as follows:
i. Argue that we can restrict the strategy space of firm i to
{1}[0, a], and so the strategy space is closed and bounded
(i.e., compact). Note that the strategy space is not convex.
(In contrast, the strategy spaces in part 4.3.12(a) are con-
vex. But see part iii below!)
ii. Prove that Ui is continuous function of the strategy pro-
files.
iii. We can make the strategy space convex, while maintaining
continuity as follows. Define Si := [1, a], and extend Ui
to Si be setting


(P (s1 + s2 ) c)si , if sj 0,

Ui (s1 , s2 ) = (P (si ) c)si , if si 0 and sj < 0,


(1 + s ), if si < 0.
i

Prove Ui is continuous on Si . Prove that every strategy in

(1, 0) is strictly dominated (and so the addition of these
strategies has no strategic impact). In particular, the set of
Nash equilibria (in pure or mixed strategies) is unaffected
by this change. Prove that Ui is not quasiconcave.
iv. As for finite games, since the payoff defined on pure strate-
gies is continuous, by considering mixed strategies, we ob-
tain convex strategy spaces and payoffs that are continu-
ous and quasiconcave. Nothing to prove here. This yields
for firm each i, a best reply correspondence (where j i)
i : (Sj ) (Si )
that satisfies all the conditions of Theorem 4.1.4.
 110 CHAPTER 4. NASH EQUILIBRIUM

v. Explain why Theorem 4.1.4 implies the existence of a sym-

metric Nash equilibrium in mixed strategies.
(c) The symmetric mixed strategy equilibrium of the game in part
4.3.12(b) can be easily calculated for this parameterization (and
coincides with the symmetric mixed strategy equilibrium of the
game in part 4.3.12(a)).
i. In the mixed strategy equilibrium the support must be {1, q },
for some q > 0, since firm payoffs are a strictly concave
function of q > 0. Explain the role of strict concavity.
ii. The symmetric mixed strategy equilibrium is thus deter-
mined by two numbers, q and , the probability of q .
Express q as a function of using an appropriate first
order condition.
iii. Finally, solve for from the appropriate indifference con-
dition.

4.3.13. Prove that the phase diagram for example 4.2.3 is as portrayed in
Figure 4.2.3. [This essentially asks you to give an expression for
dq/dp.]

4.3.14. Prove that if is asymptotically stable under a myopic adjustment

dynamic defined on a game with no role identification, then it is a
symmetric Nash equilibrium.

4.3.15. Suppose F : (S) R+ (S) is a dynamic on the strategy simplex

with F is continuously differentiable (including on the boundaries).
Suppose that if
< (s) < 1,
for some (0, 1), then
(s) > 0,
where 
F ( , t) 

 .
t t=0

Fix 0 satisfying 0 (s) > . Prove that

t (s) 1,

where t F ( 0 , t).
 January 18, 2016 111

4.3.16. Suppose a large population of players are randomly paired to play

the game (where the payoffs are to the row player)

A B C
A 1 1 0
B 0 1 1
C 0 0 1

(such a game is said to have no role identification). Let denote the

fraction of the population playing A, and denote the fraction of
the population playing C (so that 1 is the fraction of the
population playing B). Suppose the state of the population adjusts
according to the continuous time replicator dynamic.

(a) Give an expression for and for .

(b) Describe all the rest points of the dynamic.
(c) Describe the phase diagram in the space {(, ) R2+ : +
1}. Which of the rest points are asymptotically stable?
112 CHAPTER 4. NASH EQUILIBRIUM
Chapter 5

Dynamic Games and

Sequential Equilibria1

5.1 Sequential Rationality

Example 5.1.1 (Seltens horse).

1
I A II a
1
1
D d

III
L R L R

0 3 0 4
0 3 0 4
0 2 1 0

Figure 5.1.1: Seltens horse.

1
Copyright January 18, 2016 by George J. Mailath

113
114 CHAPTER 5. DYNAMIC GAMES

Let (p1 , p2 , p3 ) denote the mixed strategy profile where

Pr(I plays A) = p1 ,
Pr(II plays a) = p2 , and
Pr(III plays L) = p3.

Consider the Nash equilibrium profile (0, 1, 0) (i.e., DaR). This pro-
file is subgame perfect, and yet player II is not playing sequentially
rationally. It is also not trembling hand perfect (Definition 2.5.1):
playing a is not optimal against any mixture close to DR.
The only trembling hand perfect equilibrium outcome is Aa. The
3
set of Nash equilibria with this outcome is {(1, 1, p3 ) : 4 p3 1}.
In these equilibria, player IIIs information set is not reached, and
so the profile cannot be used to obtain beliefs for III. However,
each Nash equilibrium in the set is trembling hand perfect: Fix an
equilibrium (1, 1, p3 ). Suppose first that p3 [ 34 , 1) (so that p3 1!)
and consider the completely mixed profile

1
p1n =1 ,
n
2
p2n =1 ,
(n 1)
and p3n =p3 .

Note that p1n , p2n 1 as n . Suppose n 4. Easy to verify

that both I and II are playing optimally against the mixed profile in
(1, 1, p3 ). What about III? The probability that III is reached is

1 (n 1) 2
+ = 3/n,
n n (n 1)

and so the induced beliefs for III at his information set assign prob-
1 2
ability 3 to the left node and 3 to the right. Player III is therefore
indifferent and so willing to randomize.
The same argument shows that (1, 1, 1) is trembling hand per-
January 18, 2016 115

fect, using the trembles

1
p1n =1 ,
n
2
p2n =1 ,
(n 1)
1
and p3n =1 .
n
Indeed, any sequence of trembles satisfying p1n 1, p2n 1, and
p3n 1 will work, providing

(1 p1n ) 1
lim sup n n .
n (1 p1 p2 ) 3

(It is not even necessary for (1 p1n )/(1 p1n p2n ) to have a well-
defined limit.)

Definition 5.1.1. A system of beliefs in a finite extensive form is a

specification of a probability distribution over the decision nodes in
every information set, i.e., : X [0, 1] such that
X
(x) = 1, h.
xh
Q
Note that hi Hi (h), a compact set. We can write as a
vector (h )h specifying h (h) for each h i Hi .
We interpret as describing player beliefs. In particular, if h is
player is information set, then h (h) describes is beliefs over
the nodes in h.
Let Pb denote the probability distribution on the set of termi-
nal nodes Z implied by the behavior profile b (with b0 describing
natures moves ).

Example 5.1.2. Consider the profile (LR, UD) in the game displayed
in Figure 5.1.2. The label [p] indicates that nature chooses the node
t1 with probability p (so that (t1 ) = p and (t2 ) = 1 p). The
induced distribution Pb on Z is p z1 + (1 p) z8 .
116 CHAPTER 5. DYNAMIC GAMES

z1 U U z5
L t1 R
[p]
z2 D D z6
II II
z3 U U z7
L t2 R
[1 p]
z4 D D z8

Figure 5.1.2: Game for Example 5.1.2.

The expected payoff to player i is (recalling Definition 1.3.1)

X
E[ui |b] ui (z)Pb (z).
zZ

Let Z(h) = {z Z : x h, x z}. Let P,b (|h) denote the

probability distribution on Z(h) implied by h (h) (the beliefs
specified by over the nodes in h), the behavior profile b (inter-
preted as describing behavior at information set h and any that
could be reached from h), and nature (if there are any moves
of nature following h). By setting P,b (z|h) = 0 for all z 6 Z(h),
P,b (|h) can be interpreted as the distribution on Z, conditional
on h being reached. Note that P,b (|h) only depends on through
h ; it does not depend on h0 for any h0 6= h.2
Player is expected payoff conditional on h is
X
E ,b [ui |h] ui (z)P,b (z|h).
zZ

2
More formally, for z Z(h), there is a unique path from some x 0 h to z.
Denote the unique sequence of actions on the path from x 0 to z by a1 , . . . , aL
(where a` may be an action of nature), with player (`) choosing the action a` .
Then, for all z h,
YL
P,b (z | h) = h (x 0 ) `=0 b(`) (a` ).
January 18, 2016 117

I A1 II A2 I A1
0
0
h h0 1

D1 D2 D1
0

5 1 10
0 0 1

Figure 5.1.3: The game for Example 5.1.3.

Definition 5.1.2. A behavior strategy profile b in a finite extensive

form is sequentially rational at h Hi , given a system of beliefs ,
if
E ,b [ui | h] E ,(bi ,bi ) [ui | h],
for all bi .
A behavior strategy profile b in an extensive form is sequentially
rational, given a system of beliefs , if for all players i and all infor-
mation sets h Hi , b is sequentially rational at h.
A behavior strategy profile b in an extensive form is sequentially
rational if it is sequentially rational given some system of beliefs.

Definition 5.1.3. A one-shot deviation by player i from b is a strat-

egy bi0 with the property that there exists a (necessarily unique) in-
formation set h0 Hi such that bi (h) = bi0 (h) for all h h0 , h Hi ,
and bi (h0 ) bi0 (h0 ).
A one-shot deviation bi0 (from b, given a system of beliefs ) is
profitable if
0
E ,(bi ,bi ) [ui | h0 ] > E ,b [ui | h0 ],
where h0 Hi is the information set for which bi0 (h0 ) bi (h0 ).

Example 5.1.3. Consider the profile ((D1 , A01 ), A2 ) in the game in

Figure 5.1.3. Player I is not playing sequentially rationally at his
first information set h, but does not have a profitable one-shot de-
viation there. Player I does have a profitable one-shot deviation at
his second information set h0 . Player II also has a profitable one-
shot deviation.
118 CHAPTER 5. DYNAMIC GAMES

1
A01 2

[1] 0
U2 D10 2
I
I A1 II A2 A01 0
h [0] 1

D1 D2 D10
1 5 10
0 0 1

Figure 5.1.4: The game for Example 5.1.4. Player I is not playing sequen-
tially rationally at h. Nonetheless, player I does not have a
profitable one-shot deviation at any information set (given
the beliefs specified).

Consider, now the profile ((D1 , A01 ), D2 ). Now, player I is playing

sequentially rationally at h, even though he still has a profitable
one-shot deviation from the specified play at h0 .
The following result is obvious.
Lemma 5.1.1. If b is sequentially rational given , then there are no
profitable one-shot deviations.
Without further restrictions on (see Theorems 5.1.1 and 5.3.2),
a profile may fail to be sequentially rational and yet have no prof-
itable one-shot deviations.
Example 5.1.4. Consider the profile((D1 , A01 ), A2 ) in the game in Fig-
ure 5.1.4. Player I is not playing sequentially rationally at his first
information set h, but does not have a profitable one-shot deviation
at any information set, given the system of beliefs indicated.
Recall from Definition 1.3.5 that a game of perfect information
has singleton information sets. In such a case, the system of be-
liefs is trivial, and sequential rationality is equivalent to subgame
perfection.
January 18, 2016 119

The following is the first instance of what is often called the

one-shot deviation principle (it appears again in Theorems 5.3.2 and
7.1.3)

Theorem 5.1.1. The following three statements about a strategy pro-

file b in a finite game of perfect information are equivalent:

1. The strategy profile b is subgame perfect.

2. The strategy profile b is is sequentially rational.

3. The strategy profile b has no profitable one-shot deviations.

Proof. The equivalence of subgame perfection and sequential ratio-

nality in finite games of perfect information is immediate.
It is also immediate that a sequentially rational strategy profile
has no profitable one-shot deviations.
The proof of the remaining direction is left as an exercise (Prob-
lem 5.4.2).

5.2 Perfect Bayesian Equilibrium

Without some restrictions connecting beliefs to behavior, even Nash
equilibria need not be sequentially rational. For any behavior pro-
file b, for any x X, define
X
Pb (x) := Pb (z).
{zZ:xz}

This is equivalent to the following: For any node x, there is a unique

sequence of actions, a0 , a1 , . . . , aL , describing the path from the ini-
tial node t0 to x, with player (`) choosing the action a` . Then,
L
Y
Pb (x) = b(`) (a` ). (5.2.1)
`=0

(Recall that nature is player 0 with b0 describing natures random-

ization.)
120 CHAPTER 5. DYNAMIC GAMES

I
L
R
2
0 I

T B
[0] [1]
II
` r ` r

1 4 0 5
1 0 0 1

Figure 5.2.1: Game for Example 5.2.1

Definition 5.2.1. The information set h in a finite extensive form

game is reached with positive probability under b, or is on the path-
of-play, if X
Pb (h) = Pb (x) > 0.
xh

Theorem 5.2.1. The behavior strategy profile b of a finite extensive

form game is Nash if and only if it is sequentially rational at every
information set on the path of play, given a system of beliefs ob-
tained using Bayes rule at those information sets, i.e., for all h on
the path of play,
Pb (x)
(x) = b x h. (5.2.2)
P (h)

The proof of Theorem 5.2.1 is left as an exercise (Problem 5.4.3).

Example 5.2.1. Recall the extensive form from Example 2.3.4, re-
produced in Figure 5.2.1. The label [p] indicates that the player
owning that information set assigns probability p to the labeled
January 18, 2016 121

node. The profile RBr (illustrated) is Nash and satisfies the condi-
tions of the theorem.

Note that Theorem 5.2.1 implies Problem 2.6.11.

In Theorem 5.2.1, sequential rationality is only imposed at infor-
mation sets on the path of play. Strengthening this to all informa-
tion sets yields:
Definition 5.2.2. A strategy profile b of a finite extensive form game
is a weak perfect Bayesian equilibrium if there exists a system of
beliefs such that
1. b is sequentially rational given , and

2. for all h on the path of play,

Pb (x)
(x) = x h.
Pb (h)

Remark 5.2.1. Note that a strategy profile b is a weak perfect Bayesian

equilibrium, if and only if, it is a Nash equilibrium that is sequen-
tially rational.

Using Bayes rule where possible yields something even stronger.

The phrase where possible is meant to suggest that we apply
Bayes rule in a conditional manner. We first need the follows re-
lation from from Problem 1.4.6 (recall also from Problem 1.4.6
that information sets are not partially ordered by ):
Definition 5.2.3. The information set h follows h0 (h h0 ) if there
exists x h and x 0 h0 such that x 0 x. If h h0 , define hh0 as
the set of nodes in h that can be reached from h0 , i.e., hh0 := {x
h : x 0 h0 , x 0 x}.
An information set h (following h0 ) is reached with positive prob-
ability from h0 under (, b) if
X
P,b (hh0 | h0 ) = P,b (x | h0 ) > 0.
xhh0
122 CHAPTER 5. DYNAMIC GAMES

Since is not antisymmetric,3 care must be taken in the fol-

lowing definition
P (see Problem 5.4.4). We must also allow for the
possibility that x 0 hh0 (x 0 ) = 0.
Definition 5.2.4. A strategy profile b of a finite extensive form game
is an almost perfect Bayesian equilibrium if there exists a system of
beliefs such that
1. b is sequentially rational given , and

2. for any information set h0 and following information set h

reached with positive probability from h0 under (, b),

P,b (x | h0 ) X
(x) = (x) x hh0 .
P,b (hh0 | h0 ) xhh0

Theorem 5.2.2. Every almost perfect Bayesian equilibrium is sub-

game perfect.
The proof is left as an exercise (Problem 5.4.5).
Example 5.2.2. Continuing with the extensive form from Example
2.3.4 displayed in Figure 5.2.2: The profile LB` (illustrated) is weak
perfect Bayesian, but not almost perfect Bayesian. Note that LT ` is
not weak perfect Bayesian. The only subgame perfect eq is RBr .

We are not yet at perfect Bayesian equilibrium, because we still

need to address other phenomena, such as the one illustrated by
in Figure 5.2.3. While it is straightforward to directly deal with the
example, the conditions that deal with the general phenomenon are
complicated and hard to interpret. It is rare for the complicated
conditions to be used in practice.
The term perfect Bayesian equilibrium (or PBE) is often used in
applications to describe the collections of restrictions on the sys-
tem of beliefs that do the right/obvious thing, and as such is one
of the more abused notions in the literature. I will similarly abuse
the term.
3
That is, there can be two information sets h and h0 , owned by different play-
ers, with h following h0 and h0 following h; this cannot occur if the information
sets are owned by the same player.
January 18, 2016 123

I
L
R
2
0 I

T B
[1] [0]
II
` r ` r

1 4 0 5
1 0 0 1

Figure 5.2.2: The profile LB` (illustrated) is weak perfect Bayesian, but not
almost perfect Bayesian.
124 CHAPTER 5. DYNAMIC GAMES

I x1
L
C R

x2 x3
II
` `
c r c r

x4 x5 x6 x7
III

Figure 5.2.3: A game illustrating a weakness of almost perfect Bayesian

equilibrium. Note that under the profile where II plays `, IIIs
information set is not reached. Since II cannot distinguish
between x2 and x3 , it seems reasonable that a deviation by
II not be interpreted as a signal about the relative likelihood
of these two nodes, which would require (x2 )({x6 , x7 }) =
(x3 )({x4 , x5 }). (The equality is easier to interpret when
written in ratio form; this form allows for some probabilities
to equal zero.) The equality is not required by almost perfect
Bayesian equilibrium.
January 18, 2016 125

5.3 Sequential Equilibrium

A natural way of restricting the system of beliefs without simply
adding one seemingly ad hoc restriction after another is to use
Bayes rule on completely mixed profiles as follows:

Definition 5.3.1. In a finite extensive form game, a system of beliefs

is consistent with the strategy profile b if there exists a sequence of
completely mixed sequence of behavior strategy profiles {bk }k con-
verging to b such that the associated sequence of system of beliefs
{ k }k obtained via Bayes rule converges to .
A strategy profile b is a sequential equilibrium if, for some con-
sistent system of beliefs , b is sequentially rational at every infor-
mation set.

It might be helpful to be a little more explicit here. Since ev-

ery information set is reached with positive probability, the beliefs
obtained by Bayes rule are, for all information sets h,
k k
k bk Pb (x) Pb (x)
(x) = P (x | h) = bk = P x h. (5.3.1)
P (h) x 0 h P (x )
bk 0

k
Since bk b, from (5.2.1) we have Pb (x) Pb (x), and so if h is on
the path of play of b (i.e., Pb (h) > 0), we immediately have
k
k Pb (x) Pb (x)
(x) = bk - b ,
P (h) P (h)

the expression for (x) in (5.2.2).

Example 5.3.1. We first apply consistency to the game in Figure

5.2.3. The tremble probabilities for I are bIk (C) = k , bIk (R) = k ,
and complementary probability on L, and for II are bIIk (c) = k ,
bIk (r ) = k , and complementary probability on `. Note that player
IIs trembles must be equal at two nodes x2 and x3 , since these
three nodes are in the same information set. We then have

k k k k
(x2 ) = k and (x3 ) = k .
+ k + k
126 CHAPTER 5. DYNAMIC GAMES

If k > 0 and k 0, then k (x2 ) 1. If we also have k 0

the n any limit probabilities can be obtained in the limit (depending
on the limiting behavior of k / k ).
If II plays `, player IIIs information set is unreached conditional
on IIs information set, and so any beliefs at this information set are
consistent with almost perfect Bayesian equilibrium.
Along the sequence bk , it is reached with probability (k + k )(k +
k
), and so we have

k k
k (x4 ) = ,
(k + k )(k + k )
k k
k (x5 ) = ,
(k + k )(k + k )
k k
k (x6 ) = , and
(k + k )(k + k )
k k
k (x7 ) = .
(k + k )(k + k )

In particular, if k > 0 and k 0, then k + k (x5 ) 1.

But suppose we also have k 0. It is still true that

k k k + k k
k (x2 ) k ({x6 , x7 }) =
(k + k ) (k + k )(k + k )
k k
= ,
(k + k ) (k + k )

which equals k (x3 ) k ({x4 , x5 }). That is, (since all probabilities are
positive along the sequence, and so we can divide), we have that for
all k,
k (x2 ) k ({x4 , x5 })
= k ,
k (x3 ) ({x6 , x7 })
which must be preserved in the limit (and is the desirable property
suggested in the caption of Figure 5.2.3).

Problems 5.4.9 illustrates further the restrictions that consis-

tency places on beliefs.
January 18, 2016 127

Theorem 5.3.1. A sequential equilibrium is almost perfect Bayesian.

Proof. While this is obvious, it is useful to spell out the details.

Suppose b is a sequential equilibrium, with associated system
of beliefs and sequence (bk , k ). We need to prove the second
condition in Definition 5.2.4. Suppose the information set h fol-
lows the information set h0 and is reached with positive probability
under (b, ).
For any x hh0 , there is a unique path from some x 0 h0 .
Denote the unique sequence of actions on the path from x 0 to x
by a1 , . . . , aL , with player (`) choosing the action a` . Then, for all
x hh0 , from (5.2.1) we have

k k
YL
k
Pb (x) = Pb (x 0 ) `=0
b(`) (a` ),

and so

k QL k
bk 0
Pb (x 0 ) `=0 b(`) (a` )
P (x | h ) =
Pbk (h0 )
k
YL
k
= Pb (x 0 | h0 ) `=0 b(`) (a` )
YL
k
= k (x 0 ) `=0 b(`) (a` )
YL
(x 0 ) `=0 b(`) (a` ) = P,b (x | h0 ).

P
If xhh0 (x) = 0, then the second condition in the definition
trivially holds.
128 CHAPTER 5. DYNAMIC GAMES

P
Suppose xhh0 (x) > 0. Then, for all x hh0 ,
k k
k (x) Pb (x)/Pb (h)
P = P P
xhh0 (x) xhh0 P (x)/ xh P (h)
k bk bk

k
Pb (x)
=P
xhh0 P (x)
bk

k
Pb (x)
= bk
P (hh0 )
k k
Pb (x)/Pb (h0 )
= bk
P (hh0 )/Pbk (h0 )
k
Pb (x | h0 )
= bk .
P (hh0 | h0 )
The equality is preserved when taking limits (since the limits of the
first and last denominator are both strictly positive), and so the
second condition in the definition again holds.
Another instance of the one-shot deviation principle, which we
first saw in Theorem 5.1.1:
Theorem 5.3.2. In a finite extensive form game, suppose is consis-
tent with a profile b. The profile b is sequentially rational given
(and so a sequential equilibrium) if and only if there are no profitable
one-shot deviations from b (given ).
Proof. Lemma 5.1.1 is the easy direction.
Suppose b is not sequentially rational given . Then there is a
player, denoted i, with a profitable deviation. Denote the profitable
deviation (by player i) by bi0 and the information set h0 . Player i
information sets Hi are strictly partially ordered by precedence in
the obvious way (see Problem 1.4.6). Let Hi (h0 ) denote the finite
(since the game is finite) collection of information sets that follow
h0 . Let K be the length of the longest chain in Hi (h0 ), and say an
information set h Hi (h0 ) is of level k if the successor chain from
h0 to h has k links (h0 is 0-level and its immediate successors are
all 1-level). If i has a profitable deviation from bi (given ) at any
K-level information set, then that deviation is a profitable one-shot
deviation (given ), and we are done.
January 18, 2016 129

Suppose i does not have a profitable deviation from bi at any

(K)
K-level information set. Define a strategy bi by
(
(K) bi (h), if h is a K-level information set or h 6 Hi (h0 ),
bi (h) =
bi0 (h), if h is a k-level information set, k = 0, . . . , K 1.
(K) 0
Then E ,(bi ,bi ) [ui |h0 ] E ,(bi ,bi ) [ui |h0 ]. (This requires proof, which
is left as an exercise, see Problem 5.4.11. This is where consistency
is important.)
(K)
But this implies that, like bi0 , the strategy bi is a profitable
deviation at h0 . We now induct on k. Either there is profitable one-
shot deviation from bi at a (K 1)-level information set (in which
(K1)
case we are again done), or we can define a new strategy bi
that is a profitable deviation at h0 and which agrees with bi on the
(K 1)-level as well as the K-level information sets.
Proceeding in this way, we either find a profitable one-shot devi-
ation at some k-level information set, or the action specified at h0
by bi0 is a profitable one-shot deviation.

Remark 5.3.1. Sequential equilibrium is only defined for finite games.

For some informative examples of what can go wrong when consid-
ering finite horizon games with infinite action spaces, see Myerson
and Reny (2015).

Remark 5.3.2. Sequential equilibrium is very close to (and is im-

plied by) trembling hand perfect in the extensive form. Roughly
speaking (see Section 10.2.2 for a more precise description), se-
quential equilibrium requires behavior to be sequentially rational
against a limit assessment, while trembling hand perfection in the
extensive form requires the behavior to be sequentially rational
along the sequence as well. Problem 2.6.23 shows why trembling
hand perfect in the normal form does not imply sequential equilib-
rium.

 130 CHAPTER 5. DYNAMIC GAMES

Go II Go I Go1 II Go1 I Go2 3

I 2
Stop1 Stop1 Stop2
Stop Stop
5 6 0 1 4
1 2 0 0 3

Figure 5.4.1: The game for Problem 5.4.1.

5.4 Problems
5.4.1. This problem concerns the game given in Figure 5.4.1.

(a) Show that (GoStop1 Stop2 , StopGo1 ) is a Nash equilibrium.

(b) Identify all of the profitable one-shot deviations.

(c) Does player I choose Go in any subgame perfect equilibrium?

5.4.2. Complete the proof of Theorem 5.1.1.

5.4.3. Prove Theorem 5.2.1 (recall Problem 2.6.11).

5.4.4. This problem illustrates how badly behaved the follows relation
on information sets can be, and the reason for the introduction
of hh0 into the definition of almost perfect Bayesian equilibria. Con-
sider the game in Figure 5.4.2.

(a) Verify that the profile ( 12 U + 12 D, B, A) is a Nash equilibrium.

(b) Denote player IIs information set by h and player IIIs infor-
mation set by h0 . Verify that h follows h0 and that h0 follows h.
Evaluate
P,b (x | h0 ) and P,b (h | h0 )
for all x h. Conclude that (x) should not equal their ratio.

5.4.5. Prove Theorem 5.2.2.

 January 18, 2016 131

0 0
1 1
0
A A 1
B

U B 10
1
10 II I III 1
1
1
A D

0 A 0
1 B B 0
1 1

Figure 5.4.2: The game for Problem 5.4.4.

5.4.6. Show that (A, a, L) is a sequential equilibrium of Seltens horse (Fig-

ure 5.1.1) by exhibiting the sequence of converging completely mixed
strategies and showing that the profile is sequentially rational with
respect to the limit beliefs.

5.4.7. Prove by direct verification that the only sequential equilibrium of

the first extensive form in Example 2.3.4 is (RB, r ), but that (L, `) is
a sequential equilibrium of the second extensive form.

5.4.8. We return to the environment of Problem 3.6.1, but with one change.
Rather than the two firms choosing quantities simultaneously, firm
1 is a Stackelberg leader: Firm 1 chooses its quantity, q1 , first. Firm
2, knowing firm 1s quantity choice then chooses its quantity. De-
scribe a strategy profile for this dynamic game. What is the ap-
propriate equilibrium notion for this game and why? Describe an
equilibrium of this game. Is it unique?

5.4.9. Consider the game in Figure 5.4.3.

(a) Suppose x = 0. Verify that (L`r 0 ) is an almost perfect Bayesian

equilibrium that is not a sequential equilibrium.
 132 CHAPTER 5. DYNAMIC GAMES

I
x1

L R
C

II

` r ` r ` r

III
1 1
1 0 `0 r0 `0 r0 `0 r0 `0 r0
0 1

2 0 1 1 2 0 0 0
0 0 0 1 0 0 0 1
1 0 0 1 1 x 0 1

Figure 5.4.3: The game for Problem 5.4.9.

(b) Describe a pure strategy sequential equilibrium for this game

for x = 0.
(c) Suppose x = 3. Verify that (L`r 0 ) is a sequential equilibrium,
and describe the supporting beliefs, showing they are consis-
tent.

5.4.10. Fix a finite extensive form game. Suppose is consistent with b.

Suppose for some player i there are two information sets h, h0 Hi
with h h0 and P(,b) (h|h0 ) = 0. Prove that if there exists another
strategy bi for player i with the property that P(,(bi ,bi )) (h|h0 ) > 0,
then
P(,(bi ,bi )) (x|h0 )
(x) = , x h.
P(,(bi ,bi )) (h|h0 )
 January 18, 2016 133

5.4.11. Complete the proof of Theorem 5.3.2 by showing that

(K) 0
E ,(bi ,bi )
[ui |h0 ] E ,(bi ,bi ) [ui |h0 ].

Be sure to explain the role of consistency. (Hint: use Problem 5.4.10).

134 CHAPTER 5. DYNAMIC GAMES
Chapter 6

Signaling1

6.1 General Theory

Sender (informed player) types t T R. T may be finite. Proba-
bility distribution (T ) with (t) > 0 for all t T .
Sender chooses m M R. M may be finite.
Responder chooses r R R. R may be finite.
Payoffs: u(m, r , t) for sender and v(m, r , t) for responder.
Strategy for sender, : T M.
Strategy for responder, : M R.

Definition 6.1.1. The pure strategy profile (, ) is a perfect Bayesian

equilibrium of the signaling game if

1. for all t T ,

(t) arg max u(m, (m), t),

mM

2. for all m, there exists some (T ) such that

(m) arg max E [v(m, r , t)],

r R

where E denotes expectation with respect to , and

1
Copyright January 18, 2016 by George J. Mailath

135
136 CHAPTER 6. SIGNALING

1, 1 f f 2, 1
b t1 q
[0.5]
1, 0 r r 0, 0

II II

2, 1 f f 3, 1
b t2 q
[0.5]
0, 0 r r 1, 0

Figure 6.1.1: A signaling game

3. for m (T ), in part 2 is given by

(t) = {t | m = (t)}.

Since the different information sets for player II are not ordered
by (recall Problem 1.4.6), consistency places no restrictions on
beliefs at different information sets of player II. This implies the
following result (which Problem 6.3.3 asks you to prove).
Theorem 6.1.1. Suppose T , M, and R are finite. A profile is a perfect
Bayesian equilibrium if, and only if, it is a sequential equilibrium.
Example 6.1.1 (Separating equilibria). In the game given in Figure
6.1.1, (bq, f r ) is a separating eq.

Example 6.1.2 (Beer-quiche). In the game in Figure 6.1.2, (bb, r f )

and (qq, f r ) are both pooling eq.
The eq in which the types pool on q is often argued to be un-
intuitive: Would the w type ever rationally deviate to b. In this
pooling eq, w receives 0, and this is strictly larger than his pay-
off from b no matter how II responds. On the other hand, if by
deviating to B, s can signal that he is indeed s, he is strictly bet-
ter off, since IIs best response is r , yielding payoff of 0. This is
an example of the intuitive criterion, or of the test of equilibrium
domination.
January 18, 2016 137

3, 1 f f 2, 1
b w q
[0.1]
1, 0 r r 0, 0

II II

2, 1 f f 3, 1
b s q
[0.9]
0, 0 r r 1, 0

Figure 6.1.2: The Beer-Quiche game.

Let BR(T 0 , m) be the set of best replies to m of the responder,

when the beliefs have support in T 0 , i.e.,

BR(T 0 , m) = {r R : (T 0 ), r arg max E [v(m, r 0 , t)]}

r 0 R
[
= arg max E [v(m, r 0 , t)].
(T 0 ) r 0 R

Definition 6.1.2. Fix a perfect Bayesian equilibrium (, ), and let

u(t) = u((t), ((t)), t). Define D(m) T as the set of types
satisfying
u(t) > max u(m, r , t).
r BR(T ,m)

The equilibrium (, ) fails the intuitive criterion if there exists m0

(necessarily not in (T ), i.e., an unsent message) and a type t 0 (nec-
essarily not in D(m0 )) such that

u(t 0 ) < min u(m0 , r , t 0 ).

r BR(T \D(m0 ),m0 )

Remark 6.1.1. The test concerns equilibrium outcomes, and not

the specification of behavior after out-of-equilibrium messages. In
particular, if an equilibrium (, ) fails the intuitive criterion, then
138 CHAPTER 6. SIGNALING

every equilibrium with the same outcome as that implied by (, )

fails the intuitive criterion. (Such equilibria differ from (, ) in the
specification of the responses to out-of-equilibrium messages.)

Remark 6.1.2. For messages m0 that satisfy D(m0 ) T , it is

in the spirit of the test to require responses r to out-of-equilibrium
messages m0 to satisfy r BR(T \ D(m0 ), m0 ).

6.2 Job Market Signaling

Worker with private ability R.
Worker can signal ability through choice of level of education,
e R+ .
Worker utility
w c(e, ),
w is wage, and c is disutility of education. Assume c is C 2 and
satisfies single-crossing:

2 c(e, )
< 0.
e
Also assume c(e, ) 0, ce (e, ) c(e, )/e 0, ce (0, ) = 0,
cee (e, ) > 0, and lime ce (e, ) = .
Two identical firms competing for worker. Each firm values
worker of type with education e at f (e, ). In any discretization
of the game, in any almost perfect Bayesian equilibrium, after any
e, firms have identical beliefs () about worker ability (see
Problem 6.3.5). Consequently, the two firms are effectively playing
a sealed bid common value first price auction, and so both firms
bid their value E f (e, ). To model as a game, replace the two
firms with a single uninformed receiver (the market) with payoff

(f (e, ) w)2 .
January 18, 2016 139

Strategy for worker, e : R+ .

Strategy for market, w : R+ R+ .
Assume f is C 2 . Assume f (e, ) 0, fe (e, ) f (e, )/e 0,
f (e, ) > 0, fee (e, ) 0, and fe (e, ) 0.

1. Unproductive education. When f is independent of e, we can

interpret as the productivity of the worker, and so assume
f (e, ) = .

2. Productive education. fe (e, ) > 0.

If market believes worker has ability , firm pays wage f (e, ).

The result is a signaling game as described in Section 6.1, and so we
can apply equilibrium notion of perfect Bayesian as defined there.

6.2.1 Full Information

If firm knows worker has ability , worker chooses e to maximize

f (e, ) c(e, ). (6.2.1)

For each there is a unique e maximizing (6.2.1). That is,

e () = arg max f (e, ) c(e, ).

e0

Assuming fe (0, ) > 0 (together with the assumption on c above)

is sufficient to imply that e () is interior for all and so

de fe (e, ) ce (e, )
= > 0.
d fee (e, ) cee (e, )

6.2.2 Incomplete Information

Define
U(, , e) f (e, ) c(e, ).
Note that
e () = arg max U(, , e). (6.2.2)
e0
140 CHAPTER 6. SIGNALING

We are first interested in separating perfect Bayesian equilibria.

The outcome associated with a profile (e, w) is

(e(), w(e())) .

If e0 = e( 0 ) for some 0 , then w(e0 ) = f (e0 , (e)1 (e0 )) =

f (e0 , 0 ), and so the payoff to the worker of type is

w(e0 ) c(e0 , ) = f (e0 , 0 ) c(e0 , ) = U(, 0 , e0 ).

A separating (one-to-one) strategy e is incentive compatible if no

type strictly benefits from mimicking another type, i.e.,

U( 0 , 0 , e( 0 )) U( 0 , 00 , e( 00 )), 0 , 00 . (6.2.3)

Figure 6.2.1 illustrates the case = { 0 , 00 }.

Definition 6.2.1. The separating strategy profile (e, w) is a perfect
Bayesian equilibrium if
1. e satisfies (6.2.3), i.e., is incentive compatible,

2. w(e) = f (e, e1 (e)) for all e e(),

3. for all e R+ and all ,

w(e) c(e, ) U(, , e()),

and

4. w is sequentially rational, i.e., for all e R+ , there is some

() such that

w(e) = E f (e, ).

For e e(), is of course given by the belief that assigns

probability one to the type for which e = e(), i.e. = e1 (e)).
Sequential rationality restricts wages for w 6 e().
Note that the Intermediate Value Theorem implies that for all
e R+ and all (), there exists a unique conv() such
that
f (e, ) = E f (e, ).
January 18, 2016 141

e

U ( 0 , , e) = k0

U ( 00 , , e) = k0

U ( 00 , , e) = k00
00

E[|e]

e
e ( 0 ) e ( 00 ) e00

Figure 6.2.1: Indifference curves in e space. Space of types is =

{ 0 , 00 }. Note k0 = U ( 0 , 0 , e ( 0 )), k00 = U ( 00 , 00 , e00 ),
and k0 = U ( 00 , 00 , e ( 00 )), and that incentive compati-
bility is satisfied at the indicated points: U ( 00 , 00 , e00 )
U ( 00 , 0 , e ( 0 )) and U ( 0 , 0 , e ( 0 )) U ( 0 , 00 , e00 ). For
any e < e00 , firms believe = 0 , and for any e e00 , firms
believe = 00 .
The figures in this subsection are drawn using the pro-
duction function f (e, ) = e and cost function c(e, ) =
e5 /(5), so that U (, , e) = e (e5 )/(5). The set of pos-
sible s is {1, 2}, with full information optimal educations
of 1 and 2.
142 CHAPTER 6. SIGNALING

(This extends to canonical signaling games, see Problem 6.3.4(a).)

Thus, condition .4 in Definition 6.2.1 can be replaced by: for all
e R+ , there is some conv() such that

w(e) = f (e, ).

In particular, if = { 0 , 00 }, then [ 0 , 00 ]. (This is useful in

interpreting the figures in this section.)
Let = min . Sequential rationality implies

e() = e (). (6.2.4)

(Why?)
The set of separating perfect Bayesian equilibrium outcomes is
illustrated in Figure 6.2.2. Note that in this figure, the full informa-
tion choices are not consistent with (6.2.3).
The Riley outcome is the separating outcome that minimizes
the distortion. If the full information choices are consistent with
(6.2.3), then no distortion is necessary.
Suppose there are two types (as in Figure 6.2.2). Suppose more-
over, as in the figures, that the full information choices are not
consistent with (6.2.3). The Riley outcome is

((e ( 0 ), f (e ( 0 ), 0 )), (e100 , f (e100 , 00 ))).

It is the separating outcome that minimizes the distortion, since 0

is indifferent between ((e ( 0 ), f (e ( 0 ), 0 )) and (e100 , f (e100 , 00 ))).
Any lower education level for 00 violates (6.2.3).
Finally, worth noting that inequality (6.2.3) can be rewritten as

U( 0 , 0 , e( 0 )) U( 0 , (e)1 (e), e) e e().

That is, the function e : R+ satisfies the functional equation

e( 0 ) arg max U( 0 , (e)1 (e), e), 0 . (6.2.5)

ee()

Note that (6.2.2) and (6.2.5) differ in two ways: the set of possible
maximizers and how e enters into the objective function.
January 18, 2016 143

e

U ( 0 , , e) = k0

U ( 00 , , e)

00

e100 e200 e
e ( 0 )

Figure 6.2.2: Separating equilibria when space of types is = { 0 , 00 }.

The set of separating perfect Bayesian equilibrium outcomes
is given by {((e ( 0 ), f (e ( 0 ), 0 )), (e00 , f (e00 , 00 ))) : e00
[e100 , e200 ]}. Note that 00 cannot receive a lower payoff than
maxe U ( 00 , 0 , e).
144 CHAPTER 6. SIGNALING

e
U ( 0 , , e) = k0p

U ( 0 , , e) = k0

U ( 00 , , e) = k00
p

00

E[]

E[|e]

e ( 0 ) ep e e

Figure 6.2.3: A pooling outcome at e = ep . k0p = U ( 0 , E, ep ), k00 p =

00 p
U ( , E, e ). Note that E[|e], firms beliefs after poten-
tial deviating es must lie below the 0 and 00 indifference
curves indexed by k0p and k00p , respectively.

6.2.3 Refining to Separation

Suppose two types, 0 , 00 .
Suppose f is affine in , so that Ef (e, ) = f (e, E) (but see
Problem 6.3.4(a)).
The pooling outcome in Figure 6.2.3 is a perfect Bayesian out-
come, but is ruled out by the intuitive criterion. To see this, con-
sider the out-of-equilibrium message e in the figure. Note first that
(using the notation from Definition 6.1.2) that D(e) = { 0 }. Then,
the market wage after e must be 00 (since the market puts zero
probability on 0 ). Since
p
U( 00 , 00 , e) > k00 00
p = U( , E[], e ),

the equilibrium fails the intuitive criterion.

January 18, 2016 145

Indeed, for two types, the intuitive criterion selects the Riley
separating outcome, i.e., the separating outcome that minimizes
the signaling distortion. Consider a separating outcome this is not
the Riley outcome. Then, e( 00 ) > e100 (see Figure 6.2.2). Consider an
out-of-equilibrium message e (e100 , e( 00 )). Since e > e100 , D(e) =
{ 0 }. Then, as above, the market wage after e must be 00 , and so
since e < e( 00 ),

U( 00 , 00 , e) > U( 00 , 00 , e( 00 )),

and non-Riley separating equilibria also fail the intuitive criterion.

With three or more types, many equilibria in addition to the Ri-
ley equilibrium outcome survive the intuitive criterion. There are
stronger refinements (D1) that do select the Riley outcome (Cho and
Kreps, 1987).

6.2.4 Continuum of Types

Suppose = [, ] (so that there is a continuum of types), and

suppose e is differentiable (this can be justified, see Mailath (1987);
Mailath and von Thadden (2013)).
Then the first derivative of the objective function in (6.2.5) w.r.t.
e is

0 1 d(e)1 (e)
U ( , (e) (e), e) + Ue ( 0 , (e)1 (e), e)
de
 !1
0 1 de() 

= U ( , (e) (e), e)  + Ue ( 0 , (e)1 (e), e).
d =(e)
1 (e)

The first order condition is obtained by evaluating this derivative

at e0 = e( 0 ) (so that (e)1 (e0 ) = 0 ) and setting the result equal to
0:  1
0 0 0 de( 0 )
U ( , , e ) + Ue ( 0 , 0 , e0 ) = 0.
d
The result is a differential equation characterizing e,

de( 0 ) U ( 0 , 0 , e0 ) f (e0 , 0 )
= = .
d Ue ( 0 , 0 , e0 ) fe (e0 , 0 ) ce (e0 , 0 )
 146 CHAPTER 6. SIGNALING

Together with (6.2.4), we have an initial value problem that char-

acterizes the unique separating perfect Bayesian equilibrium strat-
egy for the worker.
Note that because of (6.2.4), as , de()/d +, and that
is necessarily a signalling
for > , e() > e (), that is, there

distortion.
Remark 6.2.1. The above characterization of separating strategies
works for any signaling game, given U(, , e), the payoff to the
informed player of type , when the uninformed player best re-
sponds to a belief that the type is , and e is chosen by the in-
formed player. See Problem 6.3.4 for a description of the canonical
signaling model.

6.3 Problems
6.3.1. Show that for the game in Figure 6.3.1, for all values of x, the out-
come in which both types of player I play L is sequential by explic-
itly describing the converging sequence of completely mixed behav-
ior strategy profiles and the associated system of beliefs. For what
values of x does this equilibrium pass the intuitive criterion?
6.3.2. (a) Prove that the signaling game illustrated in Figure 6.3.2 has no
Nash equilibrium in pure strategies.
(b) Definition 6.1.1 gives the definition of a pure strategy perfect
Bayesian equilibrium for signaling games. How should this def-
inition be changed to cover mixed strategies? More precisely,
give the definition of a perfect Bayesian equilibrium for the case
of finite signaling games.
1
(c) For the game in Figure 6.3.2, suppose p = 2 . Describe a perfect
Bayesian equilibrium.

(d) How does your answer to part 6.3.2(c) change if p = 0.1?

6.3.3. Prove Theorem 6.1.1.

January 18, 2016 147

U 6, 3
L t1 R
2, 0
1
[2]
D 0, 0
II
U x, 0
L t2 R
2, 0
1
[2]
D 0, 2

Figure 6.3.1: Game for problem 6.3.1. The probability that player I is type
t1 is 1/2 and the probability that he is type t2 is 1/2. The
first payoff is player Is payoff, and the second is player IIs.

2, 0 3, 0
TL TR
L t1 R
2, 2 ML
[p]
BL BR
1, 3 3, 1
II II
1, 3 3, 1
TL TR
L t2 R
2, 2 ML
[1 p]
BL BR
2, 0 3, 0

Figure 6.3.2: Figure for question 6.3.2.

 148 CHAPTER 6. SIGNALING

6.3.4. The canonical signaling game has a sender with private information,
denoted R choosing a message m R, where is compact.
A receiver, observing m, but not knowing then chooses a response
r R. The payoff to the sender is u(m, r , ) while the payoff to the
receiver is v(m, r , ). Assume both u and v are C 2 . Assume v is
strictly concave in r , so that v(m, r , ) has a unique maximizer in
r for all (m, ), denoted (m, ). Define

U (, , m) = u(m, (m, ), ).

Assume u is strictly increasing in r , and is strictly increasing in

, so that U is also strictly increasing in . Finally, assume that for
all (, ), U (, , m) is bounded above (and so has a well-defined
maximum).

(a) Given a message m and a belief F over , suppose r maxi-

mizes the receivers expected payoff. Prove there exists such
that r = (m , ). Moreover, if the support of F is a contin-
uum, [, ], prove that is in the support of F .

Assume u satisfies the single-crossing condition:

If < 0 and m < m0 , then u(m, r , ) u(m0 , r 0 , )

implies u(m, r , 0 ) < u(m0 , r 0 , 0 ).

(Draw the indifference curves for different types in m r space to

see that they can only cross once.)

(b) Provide restrictions on the productive education case covered

in Section 6.2 so that the senders payoff satisfies the single-
crossing condition as defined here.

(c) Prove that U satisfies an analogous version of the single-crossing

condition: If < 0 and m < m0 , then U (, , m) U (, 0 , m0 )
implies U ( 0 , , m) < U ( 0 , 0 , m0 ).

(d) Prove that the messages sent by the sender in any separating
Nash equilibrium are strictly increasing in type.
January 18, 2016 149

(e) Prove that in any separating perfect Bayesian equilibrium, type

min chooses the action m maximizing u(m, (m, ), )
(recall (6.2.4)). How is this implication of separating perfect
Bayesian equilibrium changed if u is strictly decreasing in r ? If
is strictly decreasing in ?

6.3.5. Prove that in any discretization of the job market signaling game,
in any almost perfect Bayesian equilibrium, after any e, firms have
identical beliefs about worker ability.

6.3.6. Suppose that, in the incomplete information model of Section 6.2,

the payoff to a firm from hiring a worker of type with education e
at wage w is
f (e, ) w = 3e w.

The utility of a worker of type with education e receiving a wage

w is
e3
w c(e, ) = w .

Suppose the support of the firms prior beliefs on is = {1, 3}.

(a) Describe a perfect Bayesian equilibrium in which both types of

worker choose their full information eduction level. Be sure to
verify that all the incentive constraints are satisfied.

(b) Are there other separating perfect Bayesian equilibria? What

are they? Do they depend on the prior distribution ?

Now suppose the support of the firms prior beliefs on is =

{1, 2, 3}.

(c) Why is it no longer consistent with a separating perfect Bayesian

equilibrium to have = 3 choose his full information eduction
level e (3)? Describe the Riley outcome (the separating equi-
librium outcome that minimizes the distortion), and verify that
it is indeed the outcome of a perfect Bayesian equilibrium.
 150 CHAPTER 6. SIGNALING

(d) What is the largest education level for = 2 consistent with

separating perfect Bayesian equilibrium? Prove that any sep-
arating equilibrium in which = 2 chooses that level of edu-
cation fails the intuitive criterion. [Hint: consider the out-of-
equilibrium education level e = 3.]

(e) Describe the separating perfect Bayesian equilibria in which

= 2 chooses e = 2.5. Some of these equilibria fail the in-
tuitive criterion and some do not. Give an example of one of
each (i.e., an equilibrium that fails the intuitive criterion, and
an equilibrium that does not fail).

6.3.7. The owner of a small firm is contemplating selling all or part of his
firm to outside investors. The profits from the firm are risky and the
owner is risk averse. The owners preferences over x, the fraction of
the firm the owner retains, and p, the price per share paid by the
outside investors, are given by
u(x, , p) = x x 2 + p(1 x),
where is the value of the firm (i.e., expected profits). The quadratic
term reflects the owners risk aversion. The outside investors are
risk neutral, and so the payoff to an outside investor of paying p
per share for 1 x of the firm is then
(1 x) p(1 x).

There are at least two outside investors, and the price is determined
by a first price sealed bid auction: The owner first chooses the frac-
tion of the firm to sell, 1 x; the outside investors then bid, with
the 1 x fraction going to the highest bidder (ties are broken with a
coin flip).

(a) Suppose is public information. What fraction of the firm will

the owner sell, and how much will he receive for it?

(b) Suppose now is privately known by the owner. The outside

investors have common beliefs, assigning probability (0, 1)
to = 1 > 0 and probability 1 to = 2 > 1 . Characterize
the separating perfect Bayesian equilibria. Are there any other
perfect Bayesian equilibria?
 January 18, 2016 151

(c) Maintaining the assumption that is privately known by the

owner, suppose now that the outside investors beliefs over
have support [1 , 2 ], so that there a continuum of possi-
ble values for . What is the initial value problem (differential
equation plus initial condition) characterizing separating per-
fect Bayesian equilibria?

6.3.8. Firm 1 is an incumbent firm selling widgets in a market in two pe-

riods. In the first period, firm 1 is a monopolist, facing a demand
curve P 1 = A q11 , where q11 R+ is firm 1s output in period 1 and
P 1 is the first period price. In the second period, a second firm, firm
2, will enter the market, observing the first period quantity choice
of firm 1. In the second period, the two firms choose quantities
simultaneously. The inverse demand curve in period 2 is given by
P 2 = A q12 q22 , where qi2 R+ is firm is output in period 2 and
P 2 is the second period price. Negative prices are possible (and will
arise if quantities exceed A). Firm i has a constant marginal cost of
production ci > 0. Firm 1s overall payoff is given by

(P 1 c1 )q11 + (P 2 c1 )q12 ,

while firm 2s payoff is given by

(P 2 c2 )q22 .

Firm 2s marginal cost, c2 , is common knowledge (i.e., each firm

knows the marginal cost of firm 2), and satisfies c2 < A/2.

(a) Suppose c1 is also common knowledge (i.e., each firm knows

the marginal cost of the other firm), and also satisfies c1 < A/2.
What are the subgame perfect equilibria and why?

(b) Suppose now that firm 1s costs are private to firm 1. Firm 2
does not know firm 1s costs, assigning prior probability p
(0, 1) to cost c1L and complementary probability 1 p to cost
c1H , where c1L < c1H < A/2.
i. Define a pure strategy almost perfect Bayesian equilibrium
for this game of incomplete information . What restric-
tions on second period quantities must be satisfied in any
pure strategy almost perfect Bayesian equilibrium? [Make
 152 CHAPTER 6. SIGNALING

the game finite by considering discretizations of the action

spaces. Strictly speaking, this is not a signaling game, since
firm 1 is choosing actions in both periods, so the notion
from Section 6.1 does not apply.]

ii. What do the equilibrium conditions specialize to for sepa-

rating pure strategy almost perfect Bayesian equilibria?
(c) Suppose now that firm 2s beliefs about firm 1s costs have sup-
port [c1L , c1H ]; i.e., the support is now an interval and not two
points. What is the direction of the signaling distortion in the
separating pure strategy almost perfect Bayesian equilibrium?
What differential equation does the function describing first pe-
riod quantities in that equilibrium satisfy?

6.3.9. Suppose that in the setting of Problem 3.6.1, firm 2 is a Stackelberg

leader, i.e., we are reversing the order of moves from Problem 5.4.8.

(a) Illustrate the preferences of firm 2 in q2 - space, where q2 is

firm 2s quantity choice, and is firm 1s belief about .

(b) There is a separating perfect Bayesian equilibrium in which firm

1
2 chooses q2 = 2 when = 3. Describe it, and prove it is a
separating perfect Bayesian equilibrium (the diagram from part
6.3.9(a) may help).

(c) Does the equilibrium from part 6.3.9(b) pass the intuitive cri-
terion? Why or why not? If not, describe a separating perfect
Bayesian equilibrium that does.

6.3.10. We continue with the setting of Problem 3.6.1, but now suppose that
firm 2 is a Stackelberg leader who has the option of not choosing be-
fore firm 1: Firm 2 either chooses its quantity, q2 , first, or the action
W (for wait). If firm 2 chooses W , then the two firms simultaneously
choose quantities, knowing that they are doing so. If firm 2 chooses
its quantity first (so that it did not choose W ), then firm 1, knowing
firm 2s quantity choice then chooses its quantity.

(a) Describe a strategy profile for this dynamic game. Following

the practice in signaling games, say a strategy profile is per-
fect Bayesian if it satisfies the conditions implied by sequential
January 18, 2016 153

equilibrium in discretized versions of the game. (In the current

context, a discretized version of the game restricts quantities
to some finite subset.) What conditions must a perfect Bayesian
equilibrium satisfy, and why?

(b) For which parameter values is there an equilibrium in which

firm 2 waits for all values of .

(c) Prove that the outcome in which firm 2 does not wait for any
, and firms behave as in the separating outcome of question
6.3.9(b) is not an equilibrium outcome of this game.
154 CHAPTER 6. SIGNALING
Chapter 7

Repeated Games1

7.1 Basic Structure

Stage game G {(Ai , ui )}:
Action space for i is Ai , with typical action ai Ai . An action
profile is denoted a = (a1 , . . . , an ).
Discount factor (0, 1).
Play G at each date t = 0, 1, . . . .
At the end of each period, all players observe the action profile a
chosen. Actions of every player are perfectly monitored by all other
players.
History up to date t: ht (a0 , . . . , at1 ) At H t ; H 0 {}.
t
Set of all possible histories: H t=0 H .
Strategy for player i is denoted si : H Ai . Often written si =
(si , si1 , si2 , . . .), where sit : H t Ai . Since H 0 = {}, we have s 0 A,
0

and so can write a0 for s 0 . The set of all strategies for player i is Si .
Note distinction between

actions ai Ai and

strategies si : H Ai .

Given strategy profile s (s1 , s2 , . . . , sn ), outcome path induced by

1
Copyright January 18, 2016 by George J. Mailath

155
156 CHAPTER 7. REPEATED GAMES

s is a(s) = (a0 (s), a1 (s), a2 (s), . . .), where

a0 (s) =(s1 (), s2 (), . . . , sn ()),
a1 (s) =(s1 (a0 (s)), s2 (a0 (s)), . . . , sn (a0 (s))),
a2 (s) =(s1 (a0 (s), a1 (s)), s2 (a0 (s), a1 (s)), . . . , sn (a0 (s), a1 (s))),
..
.
Payoffs of G() are

X
Ui (s) = (1 ) t ui (at (s)).
t=0

We have now described a normal form game: G() = {(Si , Ui )n

i=1 }.

Definition 7.1.1. The strategy profile s is a Nash equilibrium of

G() if, for all i and all si : H Ai ,
Ui (si , si ) Ui (si , si ).
Definition 7.1.2. Player is pure strategy minmax utility is
p
vi = min max ui (ai , ai ).
ai ai

The profile ai arg minai maxai ui (ai , ai ) minmaxes player

i. The set of (pure strategy) strictly individually rational payoffs
p
in {(Si , ui )} is {v Rn : vi > vi }. The set of feasible payoffs in
{(Si , ui )} is conv{v Rn : a S, v = u(a)}. Define F p {v
p
Rn : vi > vi } conv{v Rn : a S, v = u(a)}.

Theorem 7.1.1. Suppose s is a pure strategy Nash equilibrium.
Then,
p
Ui (s ) vi .

Proof. Let si be a strategy satisfying
si (ht ) arg max ui (ai , si

(ht )), ht H t
ai

(if the arg max is unique for some history ht , s(i(ht ) is uniquely
determined, otherwise make a selection from the argmax). Since

Ui (s ) Ui (si , si ),
January 18, 2016 157

p
and since in every period vi is a lower bound for the flow payoff
, s ), we have
received under the profile (si i

X p p

Ui (s )
Ui (si , si ) (1 ) t vi = vi .
t=0

Remark 7.1.1. In some settings it is necessary to allow players to

randomize. For example, in matching pennies, the set of pure strat-
egy feasible and individually rational payoffs is empty.

Definition 7.1.3. Player is mixed strategy minmax utility is

vi = min
Q max ui (i , i ).
i ji(Aj ) i (Ai )

The profile i arg mini maxi ui (i , i ) minmaxes player

i. The set of (mixed strategy) strictly individually rational payoffs
in {(Si , ui )} is {v Rn : vi > vi }. Define F {v Rn : vi >
vi } conv{v Rn : a S, v =u(a)}.

The Minmax Theorem (Problem 4.3.2) implies that vi is is secu-

rity level (Definition 2.4.2).
A proof essentially identical to that proving Theorem 7.1.1 (ap-
plied to the behavior strategy profile realization equivalent to )
proves the following:

Theorem 7.1.2. Suppose is a (possibly mixed) Nash equilibrium.

Then,
Ui ( ) vi .

P
Since vi vi (with a strict inequality in some games, such
pennies),
as matching lower payoffs often can be enforced using
mixed strategies. The possibility of enforcing lower payoffs allows
higher payoffs to be enforced in subgame perfect equilibria.

Given ht = (a0 , . . . , at1 ) H t and h = (a0 , . . . , a ) H , the

0
history (a0 , . . . , at1 , a0 , . . . , a ) H t +t is the concatenation of ht
158 CHAPTER 7. REPEATED GAMES

E S
E 2, 2 1, 3
S 3, 1 0, 0

Figure 7.1.1: A prisoners dilemma for Example 7.1.1.

followed by h , denoted by (ht , h ). Given si , define si |ht : H Ai

as follows:
si|ht (h ) = si (ht , h ).
Note that for all histories ht ,

si|ht Si .

Remark 7.1.2. In particular, the subgame reached by any history ht

is an infinitely repeated game that is strategically equivalent to the
original infinitely repeated game, G().

Definition 7.1.4. The strategy profile s is a subgame perfect equi-

librium of G() if, for all histories, ht H t , s|ht = (si|ht , . . . , sn|ht ) is
a Nash equilibrium of G().
Example 7.1.1 (Grim trigger in the repeated PD). Consider the pris-
oners dilemma in Figure 7.1.1.
A grim trigger strategy profile is a profile where a deviation trig-
gers Nash reversion (hence trigger) and the Nash equilibrium min-
maxes the players (hence grim). For the prisoners dilemma, grim
trigger can be described as follows: player is strategy is given by

si () =E,

and for t 1,
( 0
E, if at = EE for all t 0 = 0, 1, . . . , t 1,
si (a0 , . . . , at1 ) =
S, otherwise.
January 18, 2016 159

P
Payoff to 1 from (s1 , s2 ) is: (1 ) 2 t = 2.
Consider a deviation by player 1 to another strategy s1 . In re-
sponse to the first play of S, player 2 responds with S in every
subsequent period, so player 1 can do no better than always play
S after the first play of S. The maximum payoff from deviating in
period t = 0 (the most profitable deviation) is (1 )3. The profile
is Nash if
1
2 3(1 ) .
3
Strategy profile is subgame perfect: Note first that the profile
only induces two different strategy profiles after any history. De-

note by s = (s1 , s2 ) the profile in which each player plays S for all

histories, si (ht ) = S for all ht H. Then,2
( 0
si , if at = EE for all t 0 = 0, 1, . . . , t 1,
si|(a0 ,...,at1 ) =
si , otherwise.

We have already verified that s is an Nash equilibrium of G(), and

it is immediate that s is Nash.

Grim trigger is an example of a strongly symmetric strategy pro-

file (the deviator is not treated differently than the other player(s)):

Definition 7.1.5. Suppose Ai = Aj for all i and j. A strategy profile

is strongly symmetric if

si (ht ) = sj (ht ) ht H, i, j.

Represent strategy profiles by automata, (W , w 0 , f , ), where

W is set of states,

w 0 is initial state,
2
This is a statement about the strategies as functions, i.e., for all h H,
(
si (h ), if at = EE for all t 0 = 0, 1, . . . , t 1,
0

si|(a0 ,...,at1 ) (h ) =
si (h ), otherwise.
160 CHAPTER 7. REPEATED GAMES

f : W A is output function (decision rule),3 and

: W A W is transition function.
Any automaton (W , w 0 , f , ) induces a pure strategy profile as
follows: First, extend the transition function from the domain W
A to the domain W (H\{}) by recursively defining


(w, h t ) = (w, h t1 ), a t1 .
With this definition, the strategy s induced by the automaton is
given by s() = f (w 0 ) and
s(ht ) = f ((w 0 , ht )), ht H\{}.
Conversely, it is straightforward that any strategy profile can be
represented by an automaton. Take the set of histories H as the set
of states,

the null history as the initial

state, f (ht ) = s(ht ), and
h t , a = h t+1 , where h t+1 h t , a is the concatenation of the
history h t with the action profile a.
A strongly symmetric automaton has fi (w) = fj (w) for all w
and all i and j.
This representation leaves us in the position of working with
the full set of histories H. However, strategy profiles can often be
represented by automata with finite sets W . The set W is then
a partition on H, grouping together those histories that prompt
identical continuation strategies.
Advantage of automaton representation clearest when W can be
chosen finite, but also has conceptual advantages.
Example 7.1.2 (Grim trigger in the repeated PD, cont.).
Grim trigger profile has automata representation, (W , w 0 , f , ),
with W = {wEE , wSS }, w 0 = wEE , f (wEE ) = EE and f (wSS ) = SS,
and (
wEE , if w = wEE and a = EE,
(w, a) =
wSS , otherwise.
The automaton is illustrated in Figure 7.1.2. Note the notation con-
3
A profile of behavior strategies (b1 , . . . , bn ), bi : H (Ai ), can also be
represented by an automaton. The outputQfunction now maps into profiles of
mixtures over action profiles, i.e., f : W i (Ai ).
January 18, 2016 161

EE EE, SE, ES, SS

ES, SE, SS
w0 wEE wSS

Figure 7.1.2: The automaton from Example 7.1.2.

vention: the subscript on the state indicates the action profile spec-
ified by the output function (i.e., f (wa ) = a, and we may have
f (wa ) = a and f (wa ) = a); it is distinct from the transition func-
tion.

If s is represented by (W , w 0 , f , ), the continuation strategy

profile after a history ht , s |ht is represented by the automaton
(W , (w 0 , ht ), f , ), where (w 0 , ht ) is the result of recursively ap-
plying to ht , i.e., if ht = (ht1 , at1 ), then
(w 0 , ht ) = ((w 0 , ht1 ), at1 ).
Lemma 7.1.1. The strategy profile with representing automaton (W , w 0 , f , )
is a subgame perfect equilibrium iff for all ht H the strategy pro-
file represented by (W , (w 0 , ht ), f , ) is a Nash eq of the repeated
game.
Given an automaton (W , w 0 , f , ), let Vi (w) be is value from
being in the state w W , i.e.,
Vi (w) = (1 )ui (f (w)) + Vi ((w, f (w))).
Note that if W is finite, Vi solves a finite set of linear equations (see
Problem 7.6.3).
It is an implication of Theorem 7.1.1 and Lemma 7.1.1 that if
(W , w 0 , f , ) represents a pure strategy subgame perfect equilib-
rium, then for all states w W , and all i,
p
Vi (w) vi

Compare the following definition with Definition 5.1.3, and the
proofs of Theorem 7.1.3 with that of Theorem 5.3.2.
162 CHAPTER 7. REPEATED GAMES

Definition 7.1.6. Player i has a profitable one-shot deviation from

(W , w 0 , f , ), if there is some history ht and some action ai Ai
such that (where w = (w 0 , ht ))

Vi (w) < (1 )ui (ai , fi (w)) + Vi ((w, (ai , fi (w))).

Another instance of the one-shot deviation principle (recall The-

orems 5.1.1 and 5.3.2):
Theorem 7.1.3. A strategy profile is subgame perfect iff there are
no profitable one-shot deviations.
Proof. Clearly, if a strategy profile is subgame perfect, then there
are no profitable deviations.
We need to argue that if a profile is not subgame perfect, then
there is a profitable one-shot deviation.
Suppose (s1 , . . . , sn ) (with representing automaton (W , w 0 , f , ))
0
is not subgame perfect. Then there exists some history ht and
player i such that si |ht0 is not a best reply to si |ht0 . That is, there
exists si such that

0 < Ui (si , si|ht0 ) Ui (si|ht0 , si|ht0 ) .

For simplicity, define sj = sj |ht0 . Defining M 2 maxi,a |ui (a)|,

suppose T is large enough so that T M < /2, and consider the
strategy for i defined by
(
si (ht ), t < T ,
si (ht ) =
si (ht ), t T .

Then,
|Ui (si , si ) Ui (si , si )| T M < /2,
so that

Ui (si , si|ht0 ) Ui (si|ht0 , si|ht0 ) Ui (si , si ) Ui (si , si ) > /2 > 0.

Note that si is a profitable T -period deviation from si .

Let hT 1 be the outcome path up to and including period T 1
0
(history) induced by (si , si ), and let w = (w 0 , ht hT 1 ). Note that
0
Ui (si|hT 1 , si|hT 1 ) = Vi ((w 0 , ht hT 1 )) = Vi (w)
January 18, 2016 163

and

Ui (si|hT 1 , si|hT 1 )
= (1 )ui (si (hT 1 ), fi (w)) + Vi ((w, (si (hT 1 ), fi (w)))).
Hence, if Ui (si|hT 1 , si|hT 1 ) > Ui (si|hT 1 , si|hT 1 ), then we are done,
since player i has a profitable one-shot deviation from (W , w 0 , f , ).
Suppose not, i.e., Ui (si|hT 1 , si|hT 1 ) Ui (si|hT 1 , si|hT 1 ). For the
strategy si defined by
(
t si (ht ), t < T 1,
si (h ) =
si (ht ), t T 1,

we have
TX
2
Ui (si , si ) =(1 ) t ui (at (si , si )) + T 1 Ui (si|hT 1 , si|hT 1 )
t=0
TX
2
(1 ) t ui (at (si , si )) + T 1 Ui (si|hT 1 , si|hT 1 )
t=0
=Ui (si , si ) > Ui (si , si ).
That is, the (T 1)-period deviation is profitable. But then either the
one-shot deviation in period T 1 is profitable, or the (T 2)-shot
deviation is profitable. Induction completes the argument.

Note that a strategy profile can have no profitable one-shot

deviations on the path of play, and yet not be Nash, see Ex-
ample 7.1.4/Problem 7.6.5 for a simple example.

See Problem 7.6.4 for an alternative (and perhaps more enlight-

ening) proof.
Corollary 7.1.1. Suppose the strategy profile s is represented by
(W , w 0 , f , ). Then s is subgame perfect if, and only if, for all
w W (satisfying w = (w 0 , ht ) for some ht H), f (w) is a
Nash eq of the normal form game with payoff function g w : A Rn ,
where
giw (a) = (1 )ui (a) + Vi ((w, a)).
164 CHAPTER 7. REPEATED GAMES

Example 7.1.3 (continuation of grim trigger). We clearly have V1 (wEE ) =

2 and V1 (wSS ) = 0, so that the normal form associated with wEE is

E S
E 2, 2 (1 ), 3(1 ) ,
S 3(1 ), (1 ) 0, 0

while the normal form for wSS is

E S
E 2(1 ), 2(1 ) (1 ), 3(1 ) .
S 3(1 ), (1 ) 0, 0

As required, EE is a (but not the only!) Nash eq of the wEE normal

form, while SS is a Nash eq of the wSS normal form.

Example 7.1.4. Stage game:

A B C
A 4, 4 3, 2 1, 1
B 2, 3 2, 2 1, 1
C 1, 1 1, 1 1, 1

Stage game has a unique Nash eq: AA. Suppose 2/3. Then
there is a subgame perfect equilibrium of G() with outcome path
(BB) : (W , w 0 , f , ), where W = {wBB , wCC }, w 0 = wBB , fi (wa ) =
ai , and
(
wBB , if w = wBB and a = BB, or w = wCC and a = CC,
(w, a) =
wCC , otherwise.
The automaton is illustrated in Figure 7.1.3.
Values of the states are
Vi (wBB ) =(1 )2 + Vi (wBB ),
and Vi (wCC ) =(1 ) (1) + Vi (wBB ).
January 18, 2016 165

BB
BB

w0 wBB wCC CC

CC

Figure 7.1.3: The automaton for Example 7.1.4.

Solving,

Vi (wBB ) =2,
and Vi (wCC ) =3 1.

Player 1s payoffs in the normal form associated with wBB are

A B C
A 4(1 ) + (3 1) 3(1 ) + (3 1) 1 + (3 1)
,
B 2(1 ) + (3 1) 2 1 + (3 1)
C 1 + (3 1) 1 + (3 1) (1 ) + (3 1)

and since the game is symmetric, BB is a Nash eq of this normal

form only if
2 3(1 ) + (3 1),
i.e.,
0 1 4 + 32 a 0 (1 )(1 3),
or 1/3.
Player 1s payoffs in the normal form associated with wCC are

A B C
A 4(1 ) + (3 1) 3(1 ) + (3 1) 1 + (3 1)
,
B 2(1 ) + (3 1) 2(1 ) + (3 1) 1 + (3 1)
C 1 + (3 1) 1 + (3 1) (1 ) + 2
166 CHAPTER 7. REPEATED GAMES

and since the game is symmetric, CC is a Nash eq of this normal

form only if
(1 ) + 2 1 + (3 1),

i.e.,
0 2 5 + 32 a 0 (1 )(2 3),

or 2/3. This completes our verification that the profile repre-

sented by the automaton in Figure 7.1.3 is subgame perfect.
Note that even though there is no profitable deviation at wBB if
1/3, the profile is only subgame perfect if 2/3. Moreover,
the profile is not Nash if [1/3, 1/2) (Problem 7.6.5).

7.2 Modeling Competitive Agents (Small/Short-

Lived Players)
Example 7.2.1 (Product-choice game).

c s
H 3, 3 0, 2
L 4, 0 2, 1

Player I (row player) is long-lived, choosing high (H) or low (L) ef-
fort, player II (the column player) is short lived, choosing the cus-
tomized (c) or standardized (s) product.
Subgame perfect equilibrium described by the two state automa-
ton given in Figure 7.2.1. The action profile Ls is a static Nash equi-
librium, and since wLs is an absorbing state, we trivially have that
Ls is a Nash equilibrium of the associated one-shot game, g wLs .
Note that V1 (wHc ) = 3 and V1 (wLs ) = 2. Since player 2 is short-
lived, he must myopically optimize in each period. The one-shot
game from Corollary 7.1.1 has only one player. The one-shot game
January 18, 2016 167

Hc, Hs

Lc, Ls
w0 wHc wLs

Figure 7.2.1: The automaton for the equilibrium in Example 7.2.1.

g wHc associated with wHc is given by

c
H (1 )3 + 3
L (1 )4 + 2
and player I finds H optimal if 3 4 2, i.e., if 1/2.
Thus the profile is a subgame perfect equilibrium if, and only if,
1/2.

Example 7.2.2.
c s
H 3, 3 2, 2
L 4, 0 0, 1
The action profile Ls is no longer a static Nash equilibrium, and
so Nash reversion cannot be used to discipline player Is behavior.
Subgame perfect equilibrium described by the two state automa-
ton in Figure 7.2.2.
Since player 2 is short-lived, he must myopically optimize in
each period, and he is.
Note that V1 (wHc ) = 3 and V1 (wLs ) = (1 )0 + 3 = 3. There
are two one shot games we need to consider. The one-shot game
g wHc associated with wHc is given by

c
H (1 )3 + 3
L (1 )4 + 32
168 CHAPTER 7. REPEATED GAMES

Hc, Hs
Lc, Ls

w0 wHc wL` Hc, Hs

Lc, Ls

Figure 7.2.2: The automaton for the equilibrium in Example 7.2.2.

and player I finds H optimal if 3 4 4 + 32 a 0 (1 )(1

3) a 1/3.
The one-shot game g wLs associated with wLs is given by

s
H (1 )2 + 32
L (1 )0 + 3

and player I finds L optimal if 3 2 2 + 32 a 0 (1 )(2

3)) a 2/3.
Thus the profile is a subgame perfect equilibrium if, and only if,
2/3.

Example 7.2.3. Stage game: Seller chooses quality, H or L, and

announces price.
Cost of producing H = cH = 2.
Cost of producing L = cL = 1.
Demand: (
10 p, if H, and
x(p) =
4 p, if L.
5 3 9
If L, maxp (4 p)(p cL ) p = 2 x = 2 , L = 4 .
If H, maxp (10 p)(p cH ) p = 6 x = 4, H = 16.
Quality is only observed after purchase.
Model as a game: Strategy space for seller {(H, p), (L, p 0 ) : p, p 0
R+ }.
January 18, 2016 169

Continuum of (long-lived) consumers of mass 10, each consumer

buys zero or one unit of good. Consumer i [0, 10] values one unit
of good as follows
(
i, if H, and
vi =
max{0, i 6}, if L.

Strategy space for consumer i is {s : R+ {0, 1}}, where 1 is buy

and 0 is not buy.
Strategy profile is ((Q, p), ), where (i) is consumer is strat-
egy. Write i for (i). Consumer is payoff function is


i p,
if Q = H and i (p) = 1,
ui ((Q, p), ) = max{0, i 6} p, if Q = L and i (p) = 1, and


0, if i (p) = 0.

Firms payoff function is

((Q, p), ) =(p cQ )x(p, )

Z 10
(p cQ ) i (p)di
0
=(p cQ ){i [0, 10] : i (p) = 1},

where is Lebesgue measure. [Note that we need to assume that

is measurable.]
Assume firm only observes x(p, ) at the end of the period, so
that consumers are anonymous.
Note that x(p, ) is independent of Q, and that the choice (L, p)
strictly dominates (H, p) whenever x(p, ) 0.
If consumer i believes the firm has chosen Q, then is best re-
Q
sponse to p is i (p) = 1 only if ui ((Q, p), ) 0. Let i (p) denote
the maximizing choice of consumer i when the consumer observes
price p and believes the firm also chose quality Q. Then,
(
1, if i p, and
iH (p) =
0, if i < p,
170 CHAPTER 7. REPEATED GAMES

R 10
so x(p, H ) = p di = 10 p. Also,
(
1, if i p + 6, and
iL (p) =
0, if i < p + 6,
R 10
so x(p, L ) = p+6 di = 10 (p + 6) = 4 p.
5
Unique subgame perfect equilibrium of stage game is ((L, 2 ), L ).
Why isnt the outcome path ((H, 6), H (6)) consistent with sub-
game perfection? Note that there are two distinct deviations by
the firm to consider: an unobserved deviation to (L, 6), and an ob-
served deviation involving a price different from 6. In order to deter
an observed deviation, we specify that consumers believe that, in
response to any price different from 6, the firm had chosen Q = L,
leading to the best response i given by
(
1, if p = 6 and i p, or p 6 and i p + 6,
i (p) =
0, otherwise,
implying aggregate demand
(
4, if p = 6,
x(p, ) =
max{0, 4 p}, p 6.
Clearly, this implies that observable deviations by the firm are not
profitable. Consider then the profile ((H, 6), ): the unobserved
deviation to (L, 6) is profitable, since profits in this case are (10
6)(6 1) = 20 > 16. Note that for the deviation to be profitable,
firm must still charge 6 (not the best response to H ).
Eq with high quality: buyers believe H will be produced as long
as H has been produced in the past. If ever L is produced, then L
is expected to always be produced in future. See Figure 7.2.3.
It only remains to specify the decision rules:
(
(H, 6), if w = wH , and
f1 (w) =
(L, 25 ), if w = wL .
and (
, if w = wH , and
f2 (w) =
L , if w = wL .
January 18, 2016 171

(Hp, x)
(Lp, x) (Qp, x)
w0 wH wL

Figure 7.2.3: Grim trigger in the quality game. Note that the transitions
are only a function of Q.

Since the transitions are independent of price, the firms price is

myopically optimal in each state.
Since the consumers are small and myopically optimizing, in
order to show that the profile is subgame perfect, it remains to
verify that the firm is behaving optimally in each state. The firm
value in each state is V1 (wQ ) = Q . Trivially, L is optimal in wL .
Turning to wH , we have
9 16
(1 )20 + 16 a .
4 71
There are many other equilibria.

Remark 7.2.1 (Short-lived player). We can model the above as a

game between one long-lived player and one short-lived player. In
the stage game, the firm chooses p, and then the firm and con-
sumer simultaneously choose quality Q {L, H}, and quantity
x [0, 10]. If the good is high quality, the consumer receives a
utility of 10x x 2 /2 from consuming x units. If the good is of
low quality, his utility is reduced by 6 per unit, giving a utility of
4x x 2 /2.4 The consumers utility is linear in money, so his payoffs
are
(4 p)x x 2 , if Q = L, and
uc (Q, p) = 2
x 2
(10 p)x , if Q = H.
2

Since the period t consumer is short-lived (a new consumer replaces

him next period), if he expects L in period t, then his best reply is
4
Note that for x > 4, utility is declining in consumption. This can be avoided
by setting his utility equal to 4x x 2 /2 for x 4, and equal to 8 for all x > 4.
This does not affect any of the relevant calculations.
172 CHAPTER 7. REPEATED GAMES

to choose x = x L (p) max{4 p, 0}, while if he expects H, his

best reply is choose x = x H (p) max{10 p, 0}. In other words,
his behavior is just like the aggregate behavior of the continuum of
consumers.
This is in general true: a short-lived player can typically repre-
sent a continuum of long-lived anonymous players.

7.3 Applications
7.3.1 Efficiency Wages I
Consider an employment relationship between a worker and firm.
Within the relationship, (i.e., in the stage game), the worker (player
I) decides whether to exert effort (E) for the firm (player II), or to
shirk (S) for the firm. Effort yields output y for sure, while shirking
yields output 0 for sure. The firm chooses a wage w that period. At
the end of the period, the firm observes output, equivalently effort
and the worker observes the wage. The payoffs in the stage game
are given by
(
w e, if aI = E and aII = w,
uI (aI , aII ) =
w, if aI = S and aII = w,

and (
y w, if aI = E and aII = w,
uII (aI , aII ) =
w, if aI = S and aII = w.
Suppose
y > e.
Note that the stage game has (S, 0) as the unique Nash equilib-
rium, with payoffs (0, 0). This can also be interpreted as the pay-
offs from terminating this relationship (when both players receive
a zero outside option).
Grim trigger at the wage w is illustrated in Figure 7.3.1.
January 18, 2016 173

{Ew : w w }

{Ew : w < w } or Sw
w 0 wEw wS0

Figure 7.3.1: Grim Trigger for the employment relationship in Section

7.3.1. The transition from wEw labelled {Ew : w w }
means any action profile in which the worker exerted effort
and the firm paid at least w ; the other transition from wEw
occurs if either the firm underpaid (w < w ), or the worker
shirked (S).

Grim trigger is an equilibrium if

w e (1 )w w e (7.3.1)
and y w (1 )y y w . (7.3.2)
Combining the worker (7.3.1) and firm (7.3.2) incentive constraints
yields bounds on the equilibrium wage w :
e
w y, (7.3.3)

Note that both firm and worker must receive a positive surplus
from the relationship:
(1 )
w e e>0 (7.3.4)

and y w (1 )y > 0. (7.3.5)
Inequality (7.3.1) (equivalently, (7.3.4)) can be interpreted as in-
dicating that w is an efficiency wage: the wage is strictly higher
than the disutility of effort (if workers are in excess supply, a naive
market clearing model would suggest a wage of e). We return to
this idea in Section 7.5.1.
Suppose now that there is a labor market where firms and work-
ers who terminate one relationship can costlessly form a employ-
ment relationship with a new partner (perhaps there is a pool of
unmatched firms and workers who costlessly match).
174 CHAPTER 7. REPEATED GAMES

Ew

wEw

E0 Sw

S0
w0 wE0 wN

Figure 7.3.2: A symmetric profile for the employment relationship in Sec-

tion 7.3.1, where the firm is committed to paying w in every
period except the initial period while the relationship lasts
(and so transitions are not specified for irrelevant wages).
The state wN means start a new relationship; there are no
transitions from this state in this relationship (wN corre-
sponds to wE0 in a new relationship).

In particular, new matches are anonymous: it is not possible to

treat partners differently on the basis of behavior of past behavior
(since that is unobserved).
A specification of behavior is symmetric if all firms follow the
same strategy and all workers follow the same strategy in an em-
ployment relationship. To simplify things, suppose also that firms
commit to wage strategy (sequence) at the beginning of each em-
ployment relationship. Grim trigger at a constant wage w satisfy-
ing (7.3.3) is not a symmetric equilibrium: After a deviation by the
worker, the worker has an incentive to terminate the relationship
and start a new one, obtaining the surplus (7.3.4) (as if no deviation
had occurred).
Consider an alternative profile illustrated in Figure 7.3.2. Note
that this profile has the flavor of being renegotiation-proof. The
firm is willing to commit at the beginning of the relationship to
paying w in every period (after the initial period, when no wage is
January 18, 2016 175

paid) as long as effort is exerted if

y w 0.

The worker has two incentive constraints. In state wE0 , the value to
the worker is

(1 )e + (w e) = w e =: VI (wE0 ).

The worker is clearly willing to exert effort in wE0 if

VI (wE0 ) 0 (1 ) + VI (wE0 ),

that is
VI (wE0 ) 0.

The worker is willing to exert effort in wEw if

w e (1 )w + VI (wE0 )
= (1 + 2 )w e,

which is equivalent

w e VI (wE0 ) 0.

A critical feature of this profile is that the worker must invest

in the relationship: Before the worker can receive the ongoing sur-
plus of the employment relationship, he/she must pay an upfront
cost so the worker does not have an incentive to shirk in the current
relationship and then restart with a new firm.
If firms must pay the same wage in every period including the
initial period (for example, for legal reasons or social norms), then
some other mechanism is needed to provide the necessary disin-
centive to separate. Frictions in the matching process (involuntary
unemployment) is one mechanism. For more on this and related
ideas, see Shapiro and Stiglitz (1984); MacLeod and Malcomson
(1989); Carmichael and MacLeod (1997).
176 CHAPTER 7. REPEATED GAMES

7.3.2 Collusion Under Demand Uncertainty

Oligopoly selling perfect substitutes. Demand curve is given by

Q = min{p1 , p2 , . . . , pn },

where s is the state of demand and pi is firm is price. Demand is

evenly divided between the lowest pricing firms. Firms have zero
constant marginal cost of production.
The stage game has unique Nash equilibrium, in which firms
price at 0, yielding each firm profits of 0, which is their minmax
payoff.
In each period, the state is an independent and identical draw
from the finite set , according to the distribution q ().
The monopoly price is given by p m () := /2, with associated
monopoly profits of 2 /4.
Interested in the strongly symmetric equilibrium in which firms
maximize expected profits. A profile (s1 , s2 , . . . , sn ) if for all his-
tories ht , si (ht ) = sj (ht ), i.e., after all histories (even asymmetric
ones where firms have behaved differently), firms choose the same
action.
Along the equilibrium path, firms set a common price p(),
and any deviations are punished by perpetual minmax, i.e., grim
trigger.5
Let v be the common expected payoff from such a strategy
profile. The necessary and sufficient conditions for grim trigger to
be an equilibrium are, for each state ,
1
(1 )p()( p()) + v sup (1 )p( p)
n p<p()

= (1 )p()( p()),
(7.3.6)
5
Strictly speaking, this example is not a repeated game with perfect monitor-
ing. An action for firm i in the stage game is a vector (pi ()) . At the end
of the period, firms only observe the pricing choices of all firms at the realized
. Nonetheless, the same theory applies. Subgame perfection is equivalent to
one-shot optimality: a profile is subgame perfect if, conditional on each informa-
tion set (in particular, conditional on the realized ), it is optimal to choose the
specified price, given the specified continuation play.
January 18, 2016 177

where
1 X
v = p(0 )(0 p(0 ))q(0 ).
n 0
Inequality (7.3.6) can be written as
nv
p()( p())
(n 1)(1 )
X
= p(0 )(0 p(0 ))q(0 ).
(n 1)(1 ) 0

If there is no uncertainty over states (i.e, there exists 0 such

that q(0 ) = 1), this inequality is independent of states (and the
price p()), becoming

n1
1 . (7.3.7)
(n 1)(1 ) n
Suppose there are two equally likely states, L < H. In order to
support collusion at the monopoly price in each state, we need :
( )
L2 L2 H 2
+ (7.3.8)
4 (n 1)(1 )2 4 4
( )
H2 L2 H 2
and + . (7.3.9)
4 (n 1)(1 )2 4 4

Since H 2 > L2 , the constraint in the high state (7.3.9) is the relevant
one, and is equivalent to

2(n 1)H 2
,
L2 + (2n 1)H 2
a tighter bound than (7.3.7), since the incentive to deviate is un-
changed, but the threat of loss of future collusion is smaller.
Suppose
n1 2(n 1)H 2
<< 2 ,
n L + (2n 1)H 2
so that colluding on the monopoly price in each state is inconsis-
tent with equilibrium.
178 CHAPTER 7. REPEATED GAMES

Since the high state is the state in which the firms have the
strongest incentive to deviate, the most collusive equilibrium sets
p(L) = L/2 and p(H) to solve the following incentive constraint
with equality:
( )
L2
p(H)(H p(H)) + p(H)(H p(H)) .
(n 1)(1 )2 4

In order to fix ideas, suppose n(1 ) > 1. Then, this inequality

implies
( )
1 L2
p(H)(H p(H)) + p(H)(H p(H)) ,
2 4

that is,
 
L2 L L
p(H)(H p(H)) = L .
4 2 2
L
Note that setting p(H) = 2 violates this inequality (since H > L).
Since profits are strictly concave, this inequality thus requires a
lower price, that is,
L
p(H) < .
2
In other words, if there are enough firms colluding (n(1 ) >
1), collusive pricing is counter-cyclical! This counter-cyclicality of
prices arises with also with more states and a less severe require-
ment on the number of firms (Mailath and Samuelson, 2006, 6.1.1).

7.4 Enforceability, Decomposability, and The

Folk Theorem
Definition 7.4.1. An action profile a0 A is enforced by the contin-
uation promises : A Rn if a0 is a Nash eq of the normal form
game with payoff function g : A Rn , where

gi (a) = (1 )ui (a) + i (a).
January 18, 2016 179

A payoff v is decomposable on a set of payoffs V if there exists

an action profile a0 enforced by some continuation promises : A
V satisfying, for all i,

vi = (1 )ui (a0 ) + i (a0 ).

The notion of enforceability and decomposition plays a central

role in the construction of subgame perfect equilibria, see Problem
7.6.8.

Theorem 7.4.1 (The Folk Theorem). Suppose F has nonempty in-

terior in Rn . For all v F , there exists a sufficiently large discount
factor 0 , such that for all 0 , there is a subgame perfect equi-
librium of the infinitely repeated game whose average discounted
value is v.

Example 7.4.1 (Symmetric folk theorem for PD). We restrict atten-

tion to strongly symmetric strategies, i.e., for all w W , f1 (w) =
f2 (w). When is {(v, v) : v [0, 2]} a set of equilibrium payoffs?
Since we have restricted attention to strongly symmetric equilibria,
we can drop player subscripts. Note that the set of strongly sym-
metric equilibrium payoffs cannot be any larger than [0, 2], since
[0, 2] is the largest set of feasible symmetric payoffs.
Two preliminary calculations (important to note that these pre-
liminary calculations make no assumptions about [0, 2] being a set
of eq payoffs):

1. Let W EE be the set of player 1 payoffs that could be decom-

posed on [0, 2] using EE (i.e., W EE is the set of player 1 payoffs
that could enforceably achieved by EE followed by appropri-
ate symmetric continuations in [0, 2]). Then v W EE iff

v =2(1 ) + (EE)
3(1 ) + (SE),

for some (EE), (SE) [0, 2]. The largest value for (EE)
is 2, while the incentive constraint implies the smallest is (1
)/, so that W EE = [3(1 ), 2]. See Figure 7.4.1 for an
illustration.
180 CHAPTER 7. REPEATED GAMES

v = SS

3 0

1 v = 2(1 ) + EE

1

W SS
W EE

v
2 v0 2

3 3

Figure 7.4.1: An illustration of the folk theorem. The continuations that

enforce EE are labelled EE , while those that enforce SS are
labelled SS . The value v 0 is the average discounted value
of the equilibrium whose current value/continuation value
is described by one period of EE, followed by the cycle 1
2 3 1. In this cycle, play follows EE, (EE, EE, SS) . The
2 98
Figure was drawn for = 3 ; v 0 = 57 .
Many choices of v 0 will not lead to a cycle.
January 18, 2016 181

2. Let W SS be the set of player 1 payoffs that could be decom-

posed on [0, 2] using SS. Then v W SS iff

v =0 (1 ) + (SS)
(1)(1 ) + (ES),

for some (SS), (ES) [0, 2]. Since the inequality is satis-
fied by setting (SS) = (ES), the largest value for (SS) is
2, while the smallest is 0, and so W SS = [0, 2].

Observe that

[0, 2] W SS W EE = [0, 2] [3(1 ), 2].

Lemma 7.4.1 (Necessity). Suppose [0, 2] is the set of strongly sym-

metric strategy equilibrium payoffs. Then,

[0, 2] W SS W EE .

Proof. Suppose v is the payoff of some strongly symmetric strategy

equilibrium s. Then either s 0 = EE or SS. Since the continuation
equilibrium payoffs must lie in [0, 2], we immediately have that if
s 0 = EE, then v W EE , while if s 0 = SS, then v W SS . But this
implies v W SS W EE . So, if [0, 2] is the set of strongly symmetric
strategy equilibrium payoffs, we must have

[0, 2] W SS W EE .

So, when is
[0, 2] W SS W EE ?
This holds iff 2 3(1 ) (i.e., 35 ).

Lemma 7.4.2 (Sufficiency). If

[0, 2] = W SS W EE ,

then [0, 2] is the set of strongly symmetric strategy equilibrium pay-

offs.
182 CHAPTER 7. REPEATED GAMES

Proof. Fix v [0, 2], and define a recursion as follows: set 0 = v,

and
(
t / if t W SS = [0, 2], and
t+1 =
( t 2(1 ))/ if t W EE \ W SS = (2, 2].
Since [0, 2] W SS W EE , this recursive definition is well defined
for all t. Moreover, since 53 , t [0, 2] for all t. The recursion
thus yields a bounded sequence of continuations { t }t . Associated
with this sequence of continuations is the outcome path {at }t :
(
t EE if t W EE \ W SS , and
a =
SS if t W SS .
Observe that, by construction,
t = (1 )(ui (at ) + t+1 .
Consider the automaton (W , w 0 , f , ) where
W = [0, 2];
w 0 = v;
the output function is
(
EE if w W EE \ W SS , and
f (w) =
SS if w W SS , and

the transition function is

if w W EE \ W SS and a = f (w),
(w 2(1 ))/

(w, a) = w/ if w W SS and a = f (w), and


0, if a f (w).

The outcome path implied by this strategy profile is {at }t . More-

over,
v = 0 =(1 )ui (a0 ) + 1

=(1 )ui (a0 ) + (1 )ui (a1 ) + 2
PT 1
=(1 ) t=0 t ui (at ) + T T
P
=(1 ) t=0 t ui (at )
January 18, 2016 183

(where the last equality is an implication of < 1 and the sequence

{ T }T being bounded). Thus, the payoff of this outcome path is
exactly v, that is, v is the payoff of the strategy profile described
by the automaton (W , w 0 , f , ) with initial state w 0 = v.
Thus, there is no profitable one-deviation from this automaton
(this is guaranteed by the constructions of W SS and W EE ). Conse-
quently the associated strategy profile is subgame perfect.

See Mailath and Samuelson (2006, chapter 2) for much more on

this.

7.5 Imperfect Public Monitoring

7.5.1 Efficiency Wages II
A slight modification of the example from Section 7.3.1.6 As before,
in the stage game, the worker decides whether to exert effort (E)
or to shirk (S) for the firm (player II). Effort has a disutility of e
and yields output y for sure, while shirking yields output y with
probability p, and output 0 with probability 1p. The firm chooses
a wage w R+ . At the end of the period, the firm does not observe
effort, but does observe output.
Suppose
y e > py
so it is efficient for the worker to exert effort.
The payoffs are described in Figure 7.5.1.
Consider the profile described by the automaton illustrated in
Figure 7.5.2.
The value functions are

V1 (wS0 ) = 0, V1 (wEw ) = w e,

V2 (wS0 ) = py, V2 (wEw ) = y w ,

6
This is also similar to example in Gibbons (1992, Section 2.3.D), but the firm
also faces an intertemporal trade-off.
184 CHAPTER 7. REPEATED GAMES

E Nature

[p] [1 p]

II

w [0, y] w [0, y] w [0, y]

(w e, y w) (w, y w) (w, w)

Figure 7.5.1: The extensive form and payoffs of the stage game for the
game in Section 7.5.1.

{(y, w) : w w }

{(y, w) : w < w } or (0, w)

w 0 wEw wS0

Figure 7.5.2: The automaton for the strategy profile in the repeated game
in Section 7.5.1. The transition from the state wEw labelled
{(y, w) : w w } means any signal profile in which output is
observed and the firm paid at least w ; the other transition
from wEw occurs if either the firm underpaid (w < w ), or
no output is observed (0).
January 18, 2016 185

In the absorbing state wS0 , play is the unique eq of the stage

game, and so incentives are trivially satisfied.
The worker does not wish to deviate in H if

V1 (wEw ) (1 )w + {pV1 (wEw ) + (1 p) 0},

i.e.,
(1 p)w (1 p)e
or
1 p e p(1 )
w e= + e.
(1 p) (1 p)
To understand the role of the imperfect monitoring, compare
this with the analogous constraint when the monitoring is perfect
(7.3.1), which requires w e/..
The firm does not wish to deviate in wEw if

V2 (wEw ) (1 )y + py,

i.e.,
y w (1 )y + py a (1 p)y w .
So, the profile is an equilibrium if

1 p
(1 p)y w e.
(1 p)

In fact, it is implication of the next section that the profile is a

perfect public equilibrium.

7.5.2 Basic Structure

As before, action space for i is Ai , with typical action ai Ai . An
action profile is denoted a = (a1 , . . . , an ).
At the end of each period, rather than observing a, all players
observe a public signal y taking values in some space Y according
to the distribution Pr{y| (a1 , . . . , an )} (y| a).
Since the signal y is a possibly noisy signal of the action profile
a in that period, the actions are imperfectly monitored by the other
186 CHAPTER 7. REPEATED GAMES

players. Since the signal is public (and so observed by all players),

the game is said to have public monitoring.
Public history up to date t: ht (y 0 , . . . , y t1 ) Y t H t ; H 0
{}.
Assume Y is finite.
ui : Ai Y R, is ex post or realized payoff.
Stage game (ex ante) payoffs:
X
ui (a) u
i (ai , y)(y| a).
yY

Public histories:
t
H
t=0 Y ,

with ht (y 0 , . . . , y t1 ) being a t period history of public signals

(Y 0 {}).
Public strategies:
si : H Ai .

Definition 7.5.1. A perfect public equilibrium is a profile of public

strategies s that, after observing any public history ht , specifies a
Nash equilibrium for the repeated game, i.e., for all t and all ht Y t ,
|ht is a Nash equilibrium.


If y|a > 0 for all y and a, every public history arises with
positive probability, and so every Nash equilibrium in public strate-
gies is a perfect public equilibrium.
Automaton representation of public strategies: (W , w 0 , f , ),
where

W is set of states,

w 0 is initial state,

f : W A is output function (decision rule), and

: W Y W is transition function.

As before, Vi (w) is is value of being in state w.

January 18, 2016 187

y y
(3p2q)
(p+2q)
E (pq)
(pq)
3(1r ) 3r
S (qr )
(qr )

Figure 7.5.3: The ex post payoffs for the imperfect public monitoring ver-
sion of the prisoners dilemma from Figure 7.1.1.

Lemma 7.5.1. Suppose the strategy profile s is represented by (W , w 0 , f , ).

Then s is a perfect public eq if, and only if, for all w W (satisfying
w = (w 0 , ht ) for some ht H), f (w) is a Nash eq of the normal
form game with payoff function g w : A Rn , where
X
giw (a) = (1 )ui (a) + Vi ((w, y))(y| a).
y

See Problem 7.6.13 for the proof (another instance of the one-
shot deviation principle). .

Example 7.5.1 (PD as a partnership). Effort determines output {y, y}

stochastically:


p, if a = EE,

Pr{y| a} (y| a) = q, if a = SE or ES,


r , if a = SS,

where 0 < q < p < 1 and 0 < r < p.

The ex post payoffs (u i ) are given in Figure 7.5.3, so that ex ante
payoffs (ui ) are those given in Figure 7.1.1.

Example 7.5.2 (One period memory). Two state automaton: W =

{wEE , wSS }, w 0 = wEE , f (wEE ) = EE, f (wSS ) = SS, and
(
wEE , if y = y,
(w, y) =
wSS , if y = y.

188 CHAPTER 7. REPEATED GAMES

y y

w0 wEE wSS y

y

Value functions (I can drop player subscripts by symmetry):

V (wEE ) = (1 ) 2 + {pV (wEE ) + (1 p)V (wSS )}

and

V (wSS ) = (1 ) 0 + {r V (wEE ) + (1 r )V (wSS )}.

This is eq if

V (wEE ) (1 ) 3 + {qV (wEE ) + (1 q)V (wSS )}

and

V (wSS ) (1 ) (1) + {qV (wEE ) + (1 q)V (wSS )}.

Rewriting the incentive constraint at wEE ,

(1 ) 2 + {pV (wEE ) + (1 p)V (wSS )}

(1 ) 3 + {qV (wEE ) + (1 q)V (wSS )}

or
(p q){V (wEE ) V (wSS )} (1 ).
We can obtain an expression for V (wEE ) V (wSS ) without solv-
ing for the value functions separately by differencing the value re-
cursion equations, yielding

V (wEE ) V (wSS ) =(1 ) 2 + {pV (wEE ) + (1 p)V (wSS )}

{r V (wEE ) + (1 r )V (wSS )}
=(1 ) 2 + (p r ){V (wEE ) V (wSS )},

so that
2(1 )
V (wEE ) V (wSS ) = ,
1 (p r )
January 18, 2016 189

and so
1
. (7.5.1)
3p 2q r
Turning to wSS , we have

{r V (wEE ) + (1 r )V (wSS )}
(1 ) (1) + {qV (wEE ) + (1 q)V (wSS )}
or
(1 ) (q r ){V (wEE ) V (wSS )},
requiring
1
. (7.5.2)
p + 2q 3r
Note that (7.5.2) is trivially satisfied if r q (make sure you
understand why this is intuitive).
The two bounds (7.5.1) and (7.5.2) on are consistent if
p 2q r .
The constraint (0, 1) in addition requires
3r p < 2q < 3p r 1.
Solving for the value functions,
1
V (wEE ) 1 p (1 p) 2
=(1 )
V (wSS ) r 1 (1 r ) 0
(1 )
=


1 p (1 (1 r )) 2 1 p r


1 (1 r ) 1 p 2

r 1 p 0

(1 ) 2 (1 (1 r ))
=


(1 ) 1 p r 2r

1 2 (1 (1 r ))
=
.
1 pr 2r
 190 CHAPTER 7. REPEATED GAMES

Moreover, for fixed p and r ,

2r
lim V (wEE ) = lim V (wSS ) = ,
1 1 1p+r
and, for r > 0,
lim lim V (wEE ) = 2.
p1 1

Remark 7.5.1. The notion of PPE only imposes ex ante incentive

constraints. If the stage game has a non-trivial dynamic structure,
such as Problem 7.6.16, then it is natural to impose additional in-
centive constraints.

7.6 Problems
7.6.1. Suppose G {(Ai , ui )} is an n-person normal form game and GT
is itsQT -fold repetition (with payoffs evaluated as the average). Let
A i Ai . The strategy profile, s, is history independent if for all i
and all 1 t T 1, si (ht ) is independent of ht At (i.e., si (ht ) =
si (ht ) for all ht , ht At ). Let N(1) be the set of Nash equilibria
of G. Suppose s is history independent. Prove that s is a subgame
perfect equilibrium if and only if s(ht ) N(1) for all t, 0 t T 1
and all ht At (s(h0 ) is of course simply s 0 ). Provide examples to
show that the assumption of history independence is needed in both
directions.

7.6.2. Prove the infinitely repeated game with stage game given by match-
ing pennies does not have a pure strategy Nash equilibrium for any
.

7.6.3. Suppose (W , w 0 , f , ) is a (pure strategy representing) finite au-

tomaton with |W | = K. Label the states from 1 to K, so that
W = {1, 2, . . . , K}, f : {1, 2, . . . , K} A, and : {1, 2, . . . , K} A
{1, 2, . . . , K}. Consider the function : RK RK given by (v) =
(1 (v), 2 (v), . . . , K (v)), where

k (v) = (1 )ui (f (k)) + v(k,f (k)) , k = 1, . . . , K.

January 18, 2016 191

(a) Prove that has a unique fixed point. [Hint: Show that is a
contraction.]
(b) Given an explicit equation for the fixed point of .
(c) Interpret the fixed point.

7.6.4. A different (and perhaps more enlightening) proof of Theorem 7.1.3

is the following:
Suppose W and Ai are finite. Let V ei (w) be player is payoff from
the best response to (W , w, fi , ) (i.e., the strategy profile for the
other players specified by the automaton with initial state w). (The
best response is well-defined; you need not prove this.)

(a) Prove that

n o
ei (w) = max
V ei ((w, (ai , fi (w)))) .
(1 )ui (ai , fi (w)) + V
ai Ai

(b) Note that Vei (w) Vi (w) for all w. Denote by wi , the state that
maximizes V ei (w)Vi (w) (if there is more than one, choose one
arbitrarily).
Let aw
i be the action solving the maximization in 7.6.4(a), and
define

Vi (w) := (1 )ui (aw w
i , fi (w)) + Vi ((w, (ai , fi (w)))).

Prove that if the strategy profile with representing automaton

(W , w 0 , f , ) is not subgame perfect, then there exists a player
i for which

Vi (wi ) > Vi (wi ).
Relate this inequality to Theorem 7.1.3.
(c) Extend the argument to infinite W , compact Ai , and continuous
ui .

7.6.5. Prove that the profile described in Example 7.1.4 is not a Nash equi-
librium if [1/3, 1/2). [Hint: what is the payoff from always
playing A?] Prove that it is Nash if [1/2, 1).

7.6.6. Suppose two players play the infinitely repeated prisoners dilemma
displayed in Figure 7.6.1.
 192 CHAPTER 7. REPEATED GAMES

E S
E 1, 1 `, 1 + g
S 1 + g, ` 0, 0

Figure 7.6.1: A general prisoners dilemma, where ` > 0 and g > 0.

L R
U 2, 2 x, 0
D 0, 5 1, 1

Figure 7.6.2: The game for Problem 7.6.7.

(a) For what values of the discount factor is grim trigger a sub-
game perfect equilibrium?
(b) Describe a simple automaton representation of the behavior in
which player I alternates between E and S (beginning with E),
player II always plays E, and any deviation results in permanent
SS. For what parameter restrictions is this a subgame perfect
equilibrium?
(c) For what parameter values of `, g, and is tit-for-tat a subgame
perfect equilibrium?

7.6.7. Suppose the game in Figure 7.6.2 is infinitely repeated: Let de-
note the common discount factor for both players and consider the
strategy profile that induces the outcome path DL, U R, DL, U R, ,
and that, after any unilateral deviation by the row player speci-
fies the outcome path DL, U R, DL, U R, , and after any unilat-
eral deviation by the column player, specifies the outcome path
U R, DL, U R, DL, (simultaneous deviations are ignored. i.e., are
treated as if neither player had deviated).

(a) What is the simplest automaton that represents this strategy

profile?
(b) Suppose x = 5. For what values of is this strategy profile
subgame perfect?
 January 18, 2016 193

(c) Suppose now x = 4. How does this change your answer to part
7.6.7(b)?
(d) Suppose x = 5 again. How would the analysis in part 7.6.7(b)
be changed if the column player were short-lived (lived for only
one period)?

7.6.8. Fix a stage game G = {(Ai , ui )} and discount factor . Let E p ()

F p be the set of pure strategy subgame perfect equilibrium pay-
offs.

(a) Prove that every payoff v E p () is decomposable on E p ().

(b) Suppose : A E p () enforces the action profile a0 . Describe
a subgame perfect equilibrium in which a0 is played in the first
period.
(c) Prove that every payoff v F p decomposable on E p () is in
E p ().

7.6.9. Consider the prisoners dilemma from Figure 7.1.1. Suppose the
game is infinitely repeated with perfect monitoring. Recall that a
strongly symmetric strategy profile (s1 , s2 ) satisfies s1 (ht ) = s2 (ht )
for all ht . Equivalently, its automaton representation satisfies f1 (w) =
f2 (w) for all w. Let W = {v, v}, v > 0 to be determined, be the
set of continuation promises. Describe a strongly symmetric strat-
egy profile (equivalently, automaton) whose continuation promises
come from W which is a subgame perfect equilibrium for some val-
ues of . Calculate the appropriate bounds on and the value of v
(which may or may not depend on ).

7.6.10. Describe the five state automaton that yields v 0 as a strongly sym-
metric equilibrium payoff with the indicated cycle in Figure 7.4.1.

7.6.11. Consider the (asymmetric) prisoners dilemma in Figure 7.6.3. Suppose

the game is infinitely repeated with perfect monitoring. Prove that
1
for < 2 , the maximum (average discounted) payoff to player 1 in
1
any pure strategy subgame perfect equilibrium is 0, while for = 2 ,
there are equilibria in which player 1 receives a payoff of 1. [Hint:
First prove that, if 21 , in any pure strategy subgame perfect equi-
librium, in any period, if player 2 chooses E then player 1 chooses E
in that period.]
 194 CHAPTER 7. REPEATED GAMES

E S
E 1, 2 1, 3
S 2, 4 0, 0

Figure 7.6.3: The game for Problem 7.6.11.

L R
T 2, 3 0, 2
B 3, 0 1, 1

Figure 7.6.4: The game for Problem 7.6.12

7.6.12. Consider the stage game in Figure 7.6.4, where player 1 is the row
player and 2, the column player (as usual).

(a) Suppose the game is infinitely repeated, with perfect monitor-

ing. Players 1 and 2 are both long-lived, and have the same
discount factor, (0, 1). Construct a three state automaton
that for large is a subgame perfect equilibrium, and yields a
1
payoff to player 1 that is close to 2 2 . Prove that the automaton
has the desired properties. (Hint: One state is only used off the
path-of-play.)
(b) Now suppose that player 2 is short-lived (but maintain the as-
sumption of perfect monitoring, so that the short-lived player
in period t knows the entire history of actions up to t). Prove
that player 1s payoff in any pure strategy subgame perfect
equilibrium is no greater than 2 (the restriction to pure strategy
is not neededcan you prove the result without that restric-
tion?). For which values of is there a pure strategy subgame
perfect equilibrium in which player 1 receives a payoff of pre-
cisely 2?

7.6.13. Fix a repeated finite game of imperfect public monitoring (as usual,
assume Y is finite). Say that a player has a profitable one-shot devi-
ation from the public strategy (W , w 0 , f , ) if there is some history
 January 18, 2016 195

ht Y t and some action ai Ai such that (where w = (w 0 , ht ))

X
Vi (w) < (1)ui (ai , fi (w))+ Vi ((w, y))(y | (ai , fi (w))).
y

(a) Prove that a public strategy profile is a perfect public equilib-

rium if and only if there are no profitable one-shot deviations.
(This is yet another instance of the one-shot deviation princi-
ple).
(b) Prove Lemma 7.5.1.

7.6.14. Consider the prisoners dilemma game in Example 7.5.1.

(a) The grim trigger profile is described by the automaton (W , w 0 , f , ),

where W = {wEE , wSS }, w 0 = wEE , f (wa ) = a, and
(
wEE , if w = wEE and y = y,
(w, y) =
wSS , otherwise.

For what parameter values is the grim-trigger profile an equi-

librium?
(b) An example of a forgiving grim trigger profile is described by
the automaton (W c , w 0 , f, ), where W
c = {wEE , w 0 , wSS }, w 0 =
EE
wEE , f(wa ) = a, and

0
wEE , if w = wEE or wEE , and y = y,

0
(w, y) = wEE , if w = wEE and y = y,


w , otherwise.
SS

For what parameter values is this forgiving grim-trigger profile

an equilibrium? Compare the payoffs of grim trigger and this
forgiving grim trigger when both are equilibria.

7.6.15. Player 1 (the row player) is a firm who can exert either high effort (H)
or low effort (L) in the production of its output. Player 2 (the column
player) is a consumer who can buy either a high-priced product, h,
or a low-priced product `. The actions are chosen simultaneously,
and payoffs are given in Figure 7.6.5. Player 1 is infinitely lived, dis-
counts the future with discount factor , and plays the above game
in every period with a different consumer (i.e., each consumer lives
only one period). The game is one of public monitoring: while the
 196 CHAPTER 7. REPEATED GAMES

h `
H 4, 3 0, 2
L x, 0 3, 1

Figure 7.6.5: The game for Problem 7.6.15.

actions of the consumers are public, the actions of the firm are not.
Both the high-priced and low-priced products are experience goods
of random quality, with the distribution over quality determined by
the effort choice. The consumer learns the quality of the product af-
ter purchase (consumption). Denote by y the event that the product
purchased is high quality, and by y the event that it is low qual-
quality signal). Assume the
ity (in other words, y {y, y} is the

distribution over quality is independent of the price of the product:
(
p, if a1 = H,
Pr(y | a) =
q, if a1 = L,

with 0 < q < p < 1.

(a) Describe the ex post payoffs for the consumer. Why can the ex
post payoffs for the firm be taken to be the ex ante payoffs?
(b) Suppose x = 5. Describe a perfect public equilibrium in which
the patient firm chooses H infinitely often with probability one,
and verify that it is an equilibrium. [Hint: This can be done with
one-period memory.]
(c) Suppose now x 8. Is the one-period memory strategy profile
still an equilibrium? If not, can you think of an equilibrium in
which H is still chosen with positive probability?

7.6.16. A financial manager undertakes an infinite sequence of trades on

behalf of a client. Each trade takes one period. In each period, the
manager can invest in one of a large number of risky assets. By ex-
erting effort (a = E) in a period (at a cost of e > 0), the manager can
identify the most profitable risky asset for that period, which gen-
erates a high return of R = H with probability p and a low return
R = L with probability 1 p. In the absence of effort (a = S), the
January 18, 2016 197

manager cannot distinguish between the different risky assets. For

simplicity, assume the manager then chooses the wrong asset, yield-
ing the low return R = L with probability 1; the cost of no effort is
0. In each period, the client chooses the level of the fee x [0, x] to
be paid to the financial manager for that period. Note that there is
an exogenous upper bound x on the fee that can be paid in a period.
The client and financial manager are risk neutral, and so the clients
payoff in a period is
uc (x, R) = R x,
while the managers payoff in a period is
(
x e, if a = E,
um (x, a) =
x, if a = S.
The client and manager have a common discount factor . The client
observes the return on the asset prior to paying the fee, but does not
observe the managers effort choice.
(a) Suppose the client cannot sign a binding contract committing
him to pay a fee (contingent or not on the return). Describe the
unique sequentially rational equilibrium when the client uses
the manager for a single transaction. Are there any other Nash
equilibria?
(b) Continue to suppose there are no binding contracts, but now
consider the case of an infinite sequence of trades. For a range
of values for the parameters (, x, e, p, H, and L), there is a
perfect public equilibrium in which the manager exerts effort
on behalf of the client in every period. Describe it and the
restrictions on parameters necessary and sufficient for it to be
an equilibrium.
(c) Compare the fee paid in your answer to part 7.6.16(b) to the fee
that would be paid by a client for a single transaction,
i. when the client can sign a legally binding commitment to a
fee schedule as a function of the return of that period, and
ii. when the client can sign a legally binding commitment to a
fee schedule as a function of effort.
(d) Redo question 7.6.16(b) assuming that the clients choice of fee
level and the managers choice of effort are simultaneous, so
that the fee paid in period t cannot depend on the return in
period t. Compare your answer with that to question 7.6.16(b).
 198 CHAPTER 7. REPEATED GAMES

7.6.17. In this question, we revisit the partnership game of Example 7.5.1.

Suppose 3p 2q > 1. This question asks you to prove that for
sufficiently large , any payoff in the interval [0, v], is the payoff of
some strongly symmetric PPE equilibrium, where
(1 p)
v = 2 ,
(p q)
and that no payoff larger than v is the payoff of some strongly sym-
metric PPE equilibrium. Strong symmetry implies it is enough to
focus on player 1, and the player subscript will often be omitted.

(a) The action profile SS is trivially enforced by any constant con-

tinuation [0, ] independent of y. Let W SS be the set of
values that can be obtained by SS and a constant continuation
[0, ], i.e.,

W SS = (1 )u1 (SS) + : [0, ] .
Prove that W SS = [0, ]. [This is almost immediate.]
(b) Recalling Definition 7.4.1, say that v is decomposed by EE on
y
[0, ] if there exists y , [0, ] such that
y
v =(1 )u1 (EE) + {p y + (1 p) } (7.6.1)
y y
(1 )u1 (SE) + {q + (1 q) }. (7.6.2)
y
(That is, EE is enforced by the continuation promises y ,
and implies the value v.) Let W EE be the set of values that can
be decomposed by EE on [0, ]. It is clear that W EE = [ 0 , 00 ],
for some 0 and 00 . Calculate 0 by using the smallest possible
y
choices of y and in the interval [0, ] to enforce EE. (This
will involve having the inequality (7.6.2) holding with equality.)
(c) Similarly, give an expression for 00 (that will involve ) by us-
y
ing the largest possible choices of y and in the interval
[0, ] to enforce EE. Argue that < 00 .
(d) As in Example 7.4.1, we would like all continuations in [0, ] to
be themselves decomposable using continuations in [0, ], i.e.,
we would like
[0, ] W SS W EE .
Since < 00 , we then would like 00 . Moreover, since
we would like [0, ] to be the largest such interval, we have
= 00 . What is the relationship between 00 and v?
January 18, 2016 199

(e) For what values of do we have [0, ] = W SS W EE ?

(f) Let (W , w 0 , f , ) be the automaton given by W = [0, v], w 0
[0, v], (
EE, if w W EE ,
f (w) =
SS, otherwise,
and (
y (w), if w W EE ,
(w, y) =
w/, otherwise,
where y (w) solves (7.6.1)(7.6.2) for w = v and y = y, y. For
our purposes here, assume that V (w) = w, that is, thevalue
to a player of being in the automaton with initial state w is
precisely w. (From the argument of Lemma 7.4.2, this should
be intuitive.) Given this assumption, prove that the automaton
describes a PPE with value w 0 .
200 CHAPTER 7. REPEATED GAMES
Chapter 8

Topics in Dynamic Games1

8.1 Dynamic Games and Markov Perfect Equi-

libria
Set of players: {1, . . . , n}.
Action space for i is Ai .
Set of states S, with typical state s S.
Payoffs for each i:
ui : S A R,

with future flow payoffs discounted at rate (0, 1).

State transitions:
q : S A S,

and initial state s0 S. (More generally, can have random transi-

tions from S A into (S), but deterministic transitions suffice for
an introduction.)

Example 8.1.1. Suppose players 1 and 2 fish from a common area

(pool). In each period t, the pool contains a stock of fish of size
s t R+ . This is the state of the game.
In period t, player i attempts to extract ati 0 units of fish. In
particular, if player i attempts to extract ati , then player i actually
1
Copyright January 18, 2016 by George J. Mailath

201
202 CHAPTER 8. TOPICS IN DYNAMIC GAMES

extracts
ati , if at1 + at2 s t ,
ati = ati

a1 +at2
t st, if at1 + at2 > s t .

It will turn out that in equilibrium, at1 + at2 < s t , so we can ig-
nore the rationing rule (see Problem 8.4.1(a)), and assume player i
derives payoff
ui (s, a) = log ai ,
from (s, a) for all values of (s, a), with the choices restricted to
those satisfying ai < s.
The transition rule is

q(s, a) = 2 max{0, s a1 a2 },

that is, it is deterministic and doubles any leftover stock after ex-
traction. The initial stock is fixed at some value s 0 .

State is public and perfect monitoring of actions, so history to

period t is

ht = (s 0 , a0 , s 1 , a1 , . . . , s t1 , at1 , s t ) (S A)t S.

Let H t denote the set of all feasible t-period histories (so that s is
consistent with (s 1 , a1 ) for all 1 t). A pure strategy for i
is a mapping
i : t H t Ai .
For any history ht , write the function that identifies the last state
s t by s(ht ). Let G(s) denote the dynamic game with initial state s.
As usual, we have:
Definition 8.1.1. The profile is a subgame perfect equilibrium if
for all ht , |ht := (1 |ht , . . . , n |ht ) is a Nash equilibrium of the dy-
namic game G(s(ht )).
Different histories that lead to the same state are effectively
payoff equivalent. Loosely, a strategy is said to be Markov if at
different histories that are effectively payoff equivalent, the strat-
egy specifies identical behavior. See Maskin and Tirole (2001) for a
discussion of why this may be a reasonable restriction. There are
January 18, 2016 203

no general strong foundations for the restriction, though it is pos-

sible to provide one in some settings (Bhaskar, Mailath, and Morris,
2013).
Definition 8.1.2. A strategy i : t H t Ai is Markov if for all histo-
ries ht and ht , if s(ht ) = s(ht ), , then

i (ht ) = i (ht ).
If the above holds for histories ht and h of possibly different length
(so that t is allowed), the strategy is stationary.
Restricting equilibrium behavior to Markov strategies:
Definition 8.1.3. A Markov perfect equilibrium is a strategy profile
that is a subgame perfect equilibrium, and for which each i is
Markov.

Note that while there is a superficial similarity between

Markov states s and automata states used in the theory of
repeated games, they are very different.
In particular, a repeated game has a trivial set of Markov
states (since different histories lead to strategically equiv-
alent subgames, see Remark 7.1.2), and the only Markov
perfect equilibria involve specifying static Nash equilibria in
each period.

Example 8.1.2 (Example 8.1.1 continued). Fix a symmetric station-

ary MPE. Let V (s) denote the common equilibrium value from the
state s (in an MPE, this must be independent of other aspects of the
history).
The common eq strategy is a1 (s) = a2 (s).
One-shot deviation principle holds here, and so for each player
i, ai (s) solves, for any s S, the Bellman equation:
ai (s) arg max (1 ) log(ai ) + V (max{0, 2(s ai aj (s))}).
ai Ai , ai s

Assuming V is differentiable, that the optimal choices satisfy

s > ai + aj (s) > 0,
204 CHAPTER 8. TOPICS IN DYNAMIC GAMES

and imposing ai = aj (s) = a(s) after differentiating, the implied

first order condition is

(1 )
= 2V 0 (2(s 2ai (s))).
ai (s)

To find an equilibrium, suppose

ai (s) = ks,

for some k. Then we have

s t+1 = 2(s t 2ks t ) = 2(1 2k)s t .

Given an initial stock s, in period t, ati = k[2(1 2k)]t s, and so

X
V (s) =(1 ) t log{k[2(1 2k)]t s}
t=0
X
=(1 ) t log{k[2(1 2k)]t } + log s.
t=0

This implies V is indeed differentiable, with V 0 (s) = 1/s.

1
Solving the first order condition, k = 2 , and so

1
ai (s) = s.
2

Example 8.1.3 (Asynchronous move games). Consider the repeated

prisoners dilemma, but where player 1 moves in odd periods only
and player 2 moves in even periods only. The game starts with E1
exogenously and publicly specified for player 1. The stage game is
(x > 0):
E2 S2
E1 2, 2 x, 3
S2 3, x 0, 0
January 18, 2016 205

This fits into the above formulation of a repeated game: S =

{E1 , S1 , E2 , S2 }, s 0 = E1 ,
(
a1 , if s {E2 , S2 },
q(s, a) =
a2 , if s {E1 , S1 },
(
g1 (a1 , s), if s {E2 , S2 },
u1 (s, a) =
g1 (s, a2 ), if s {E1 , S1 },
where gi describes the stage game payoffs from the PD, and
(
g2 (s, a2 ), if s {E1 , S1 },
u2 (s, a) =
g2 (a1 , s), if s {E2 , S2 }.

In particular, when the current state is player 1s action (i.e., we

are in an even period), 1s choice is irrelevant and can be ignored.
Grim Trigger:
(
GT t Ei , if ht = E1 or always E,
i (h ) =
Si , otherwise.

Need to check two classes of information sets: when players are

supposed to play Ei , and when they are supposed to play Si :
1. Optimality of Ei after all Es:
2 3(1 ) + 0
1
.
3

2. The optimality of Si after any S1 or S2 is trivially true for all :

0 (x)(1 ) + 0.

This equilibrium is not an equilibrium in Markov strategies (and

so is not an MPE).
Supporting effort using Markov pure strategies requires a tit-
for-tat like behavior:
(
Ei , if s t = Ej ,
i (ht ) = (8.1.1)
Si , if s t = Sj .
206 CHAPTER 8. TOPICS IN DYNAMIC GAMES

For t 1, everything is symmetric. The value when the current

state is Ej is
Vi (Ej ) = 2,
while the payoff from a one-shot deviation is

3(1 ) + 0 = 3(1 ),
1
and so the deviation is not profitable if (as before) 3 .
The value when the current state is Sj is

Vi (Sj ) = 0,

while the payoff from a one-shot deviation is (since under the Markov
strategy, a deviation to Ei triggers perpetual E1 E2 ; the earlier devia-
tion is forgiven)

x(1 ) + 2 = (2 + x) x.

The deviation is not profitable if

(2 + x) x 0
x
.
2+x
Note that
x 1
x 1.
2+x 3
Thus, is an MPE (inducing the outcome path (E1 E2 ) ) if x 1
and
1 x
. (8.1.2)
3 2+x

8.2 Disappearance of Monopoly Power and

the Coase Conjecture
January 18, 2016 207

Uninformed seller selling to informed buyerone sided offers,

seller makes all the offers.
Sellers cost (value) is zero.
Buyer values good at v. Assume v is uniformly distributed on
[0, 1], so that there are buyers with valuations arbitrarily close to
the sellers valuation.

8.2.1 One and Two Period Example

Buyer accepts a take-it-or-leave-it offer of p if v > p and rejects if
v < p.
Seller chooses p to maximize

p Pr{sale} = p(1 p),

i.e., chooses p = 1/2, for a payoff of 1/4. This is the optimal seller
mechanism (this can be easily shown using standard mechanism
design techniques).
Suppose now two periods, with common discount factor
(0, 1). If seller chose p 0 = 1/2 in the first period, and buyers with
v > 1/2 buy in period 0, then buyers with value v [0, 1/2] are
left, and then optimal for seller to price p 1 = 1/4. But then buyer
v = 1/2 strictly prefers to wait till period 1 (and so by continuity
so do some buyers with v > 1/2.
Suppose seller makes offers p t in period t, t = 0, 1, and buyers
v < dont buy in period 0. Then, p 1 = /2. If < 1, then
should be indifferent between purchasing in period 0 and period 1,
so that

p 0 = ( p 1 )
= /2
= p 0 = (1 /2) < .

The sellers payoff (as a function of ) is

(1 /2)(1 ) + 2 /4.
208 CHAPTER 8. TOPICS IN DYNAMIC GAMES

The first order condition is

(1 /2) 2(1 /2) + /2 = 0
2
= = (< 1)
4 3
(2 )2 1
= p 0 = < .
8 6 2
The resulting payoff is
(2 )2 1
< .
4(4 3) 4

8.2.2 Infinite Horizon

Seller makes offers p t in period t = 0, 1, . . . , T , T .
After each offer, buyer Accepts or Rejects.
If agreement in period t at price p t , payoff to seller is
us = t p t ,
and payoff to buyer is


ub = t v p t .
Interested in equilibrium of following form:
If p t offered in period t, types v p t accept and types v < p t
reject, where > 1.
If at time t, sellers posterior beliefs are uniform on [0, ], seller
offers p t () = , where < 1.
Natural to treat as a state variable. Resulting equilibrium is a
Markov equilibrium.
Under this profile, p 0 = and sellers posterior entering pe-
riod 1 is [0, ], so in order for profile to be well defined, < 1.
Thus, p 1 = () = 2 and sellers posterior entering period 2 is
[0, 2 2 ]. Prices are thus falling exponentially, with p t = t+1 t .
Let Us () be the discounted expected value to the seller, when
his posterior beliefs are uniform on [0, ]. Then
(
)
p p

Us () = max p+ Us p ,
p
January 18, 2016 209

or


Ws () = max p p + Ws p , (8.2.1)
p

where Ws () = Us (). If Ws is differentiable, then p() solves the

first order condition,

2p() + Ws0 (p()) = 0.

The envelope theorem applied to (8.2.1) gives

Ws0 () = p() = ,

so that
Ws0 (p()) = p(p()) = p() = 2 .
Substituting,
2 + 2 2 = 0,
or
1 2 + 2 2 = 0. (8.2.2)
Turning to the buyers optimality condition, a buyer with valuation
v = p must be indifferent between accepting and rejecting, so


p p = p p ,

or
1 = (1 ) . (8.2.3)
Solving (8.2.2) for yields

2 4 4 1 1
= = .
2
Since we know < 1, take the negative root,

1 1
= .

Substituting into (8.2.3),
p
= + 1 ,
210 CHAPTER 8. TOPICS IN DYNAMIC GAMES

or
1
= ,
1
so that
 
p 1 1 1 (1 )
= 1 = .

Note that in this equilibrium, there is skimming: higher valua-
tion buyers buy before lower valuation buyers.
Equilibrium is not unique. It is the only stationary equilibrium.
Let denote real time, the length of a period, and r the rate
of time discount, so that = er . If buyer with valuation v buys
at or after , his utility is no more than er v. Buying in period 0,
he earns v (), and so for close to 1, buyer v buys before .
Note that this is not a uniform statement (since for all and all
there exists v such that v purchases after ).
The Coase conjecture is:
As time between offers shrinks, price charged in first period con-
verges to competitive price, and trade becomes efficient.

8.3 Reputations
Recall the stage game of the the chain store paradox from example
2.2.1, reproduced in Figure 8.3.1.
Two Nash equilibria: (In, Accommodate) and (Out, Fight). Latter
violates backward induction.
Chain store, play the game twice, against two different entrants
(E1 and E2 ), with the second entrant E2 observing outcome of first
interaction. Incumbent receives total payoffs.
Chain store paradox only backward induction (subgame per-
fect) outcome is that both entrants enter (play In), and incumbent
always accommodates.
But, now suppose incumbent could be tough, t : such an incum-
bent receives a payoff of 2 from fighting and only 1 from accom-
modating. Other incumbent is normal, n . Both entrants prior
1
assigns prob (0, 2 ) to the incumbent being t . In second
January 18, 2016 211

Entrant

Out In

Incumbent
0
4 Fight Accommodate

1 1
1 2

Figure 8.3.1: The stage game for the chain store.

market, normal incumbent accommodates and tough fights. Con-

ditional on entry in the first market, result is the signaling game
illustrated in Figure 8.3.2.
Note first that there are no pure strategy equilibria.
There is a unique mixed strategy equilibrium: n plays F +
(1 ) A, t plays F for sure. E2 enters for sure after A, and plays
E + (1 ) S after F .
E2 is willing to randomize only if his posterior after F that the
incumbent is t equals 12 . Since that posterior is given by

Pr{F | t } Pr{t }
Pr{t | F } =
Pr{F }

= ,
+ (1 )

solving
1
=
+ (1 ) 2
gives

= ,
1
1
where < 1 since < 2 .
212 CHAPTER 8. TOPICS IN DYNAMIC GAMES

4, 1 I I 3, 1
F t A
[]
6, 0 O O 5, 0

E2 E2

3, 1 I I 4, 1
F n A
[1 ]
5, 0 O O 6, 0

Figure 8.3.2: A signaling game representation of the subgame reached by

E1 entering.

Type n is willing to randomize if

4
|{z} = 3 + 5(1 ),
| {z }
Payoff from A Payoff from F

i.e.,
1
=
.
2
Entrant E1 thus faces a probability of F given by
+ (1 ) = 2.
1
Hence, if < 4 , E1 faces F with sufficiently small probability that
1 1
he enters. However, if ( 4 , 2 ), E1 faces F with sufficiently high
probability that he stays out.

8.3.1 Infinite Horizon

Suppose now infinite horizon with incumbent discounting at rate
(0, 1) and a new potential entrant in each period.
Note first that in the complete information game, the outcome
in which all entrants enter (play In) and the incumbent accommo-
dates in every period is an equilibrium. Moreover, the profile in
January 18, 2016 213

which all entrants stay out, any entry is met with F is a subgame
perfect equilibrium, supported by the threat that play switches
to the always-enter/always-accommodate equilibrium if the incum-
bent ever responds with A. The automaton representation is given
in Figure 8.3.3.

O, IF

IA
w0 wOF wIA

Figure 8.3.3: Automaton representation of Nash reversion in the complete

information game.

Note that the relevant incentive constraint for the incumbent is

conditional on I in state wOF (since I does not make a decision
when the entrant chooses O),2 i.e.,

(1 ) + 4 2,

i.e.,
1
.
3
We now consider the reputation game, where the incumbent may
be normal or tough.
The profile in which all entrants stay out, any entry is met with
F is a subgame perfect equilibrium, supported by the threat that
the entrants believe that the incumbent is normal and play switches
2
This is similar to the collusion example discussed in Section 7.3.2. The stage
game is not a simultaneous move game, and so the repeated game does not have
imperfect monitoring. In particular, the incumbents choice between F and A is
irrelevant if the entrant plays O (as the putative equilibrium requires). Subgame
perfection, however, requires that the incumbents choice of F be optimal, given
that the entrant had played I. The principle of one-shot optimality applies here:
the profile is subgame perfect if, conditional on I, it is optimal for the incumbent
to choose F , given the specified continuation play (this is the same idea as that
in footnote 5 in Section 7.3.2).
214 CHAPTER 8. TOPICS IN DYNAMIC GAMES

to the always-enter/always-accommodate equilibrium if the incum-

bent ever responds with A.
Theorem 8.3.1. Suppose the incumbent is either of type n or type
t , and that type t has prior probability less than 1/2. Type n
must receive a payoff of at least (1 ) 1 + 4 = 1 + 3 in any
pure strategy Nash equilibrium in which t always plays F .
If type t has prior probability greater than 1/2, trivially there
is never any entry and the normal has payoff 4.

Proof. In a pure strategy Nash equilibrium, either the incumbent

always plays F , (in which case, the entrants always stay out and the
incumbents payoff is 4), or there is a first period (say ) in which
the normal type accommodates, revealing to future entrants that
he is the normal type (since the tough type plays F in every period).
In such an equilibrium, entrants stay out before (since both types
of incumbent are choosing F ), and there is entry in period . After
observing F in period , entrants conclude the firm is the t type,
and there is no further entry. An easy lower bound on the normal
incumbents equilibrium payoff is then obtained by observing that
the normal incumbents payoff must be at least the payoff from
mimicking the t type in period . The payoff from such behavior
is at least as large as
1
X 0
(1 ) 4 + (1
| )

{z 1}
{z =0
0
| }
payoff in from playing F when
0
payoff in < from pooling
A is myopically optimal
with t type

X 0
+ (1 ) 4
0 =+1
| {z }
0
payoff in > from being treated as the t type

=(1 )4 + (1 ) + +1 4
=4 3 (1 )
4 3(1 ) = 1 + 3.
January 18, 2016 215

For > 1/3, the outcome in which all entrants enter and the
incumbent accommodates in every period is thus eliminated.

8.3.2 Infinite Horizon with Behavioral Types

In the reputation literature (see Mailath and Samuelson (2006, Chap-
ter 15) and Mailath and Samuelson (2014) for an extensive introduc-
tion), it is standard to model the tough type as a behavioral type.
In that case, the tough type is constrained to necessarily choose
F . Then, the result is that in any equilibrium, 1 + 3 is the lower
bound on the normal types payoff. (The type t from earlier in
this section is an example of a payoff type.)
In fact, irrespective of the presence of other types, if the entrants
assign positive probability to the incumbent being a tough behav-
ioral type, for close to 1, player Is payoff in any Nash equilibrium
is close to 4 (this is an example of a reputation effect):
Suppose there is a set of types for the incumbent. Some of
these types are behavioral. One behavioral type, denoted 0 ,
is the Stackelberg, or tough, type, who always plays F . The normal
type is n . Other types may include behavioral k , who plays F in
every period before k and A afterwards. Suppose the prior beliefs
over are given by .
Lemma 8.3.1. Consider the incomplete information game with types
for the incumbent. Suppose the Stackelberg type 0 receives
positive prior probability 0 > 0. Fix a Nash equilibrium. Let ht be
a positive probability period-t history in which every entry results in
F . The number of periods in ht in which an entrant entered is no
larger than
log 0
k := .
log 2
Proof. Denote by q the probability that the incumbent plays F in
period conditional on h if entrant plays I. In equilibrium, if
entrant does play I, then
1
q .
2
216 CHAPTER 8. TOPICS IN DYNAMIC GAMES

1
(If q > 2 , it is not a best reply for the entrant to play I.) An upper
bound on the number of periods in ht in which an entrant entered
is thus
1
k(t) := #{ : q 2 },

1
the number of periods in ht where q 2 . (This is an upper bound,
and not the actual number, since the entrant is indifferent if q =
1
2
.)
Let := Pr{0 |h } be the posterior probability assigned to 0
after h , where < t (so that h is an initial segment of ht ). If
entrant does not enter, +1 = . If entrant does enter in ht ,
then the incumbent fights and3

Pr{0 , F |h }
+1 = Pr{0 |h , F } =
Pr{F |h }
Pr{F |0 , h } Pr{0 |h }
=
Pr{F |h }

= .
q

Defining

(
q , if there is entry in period ,
q =
1, if there is no entry in period ,

we have, for all t,

= q +1 ,

1
Note that q < 1 = q = q 2 .

3
Since the entrants action is a function of h only, it is uninformative about
the incumbent and so can be ignored in the conditioning.
January 18, 2016 217

Then,

0 = q0 1 = q0 q1 2
t1
Y
= t q
=0
Y
= t q
{:qt 12 }
 k(t)
1
.
2
1
Taking logs, log 0 k(t) log 2 , and so

log 0
k(t) .
log 2

The key intuition here is that since the entrants assign prior pos-
itive probability (albeit small) to the Stackelberg type, they cannot
be surprised too many times (in the sense of assigning low prior
probability to F and then seeing F ). Note that the upper bound is
independent of t and , though it is unbounded in 0 .

Theorem 8.3.2. Consider the incomplete information game with types

for the incumbent. Suppose the Stackelberg type 0 receives
positive prior probability 0 > 0. In any Nash equilibrium, the nor-

mal types expected payoff is at least 1 + 3k . Thus, for all > 0,
there exists such that for all (, 1), the normal types payoff in
any Nash equilibrium is at least 4 .

Proof. The normal type can guarantee histories in which every entry
results in F by always playing F when an entrant enters. Such be-
havior yields payoffs that are no larger than the incumbents Nash
equilibrium payoffs in any equilibrium (if not, the incumbent has an
incentive to deviate). Since there is positive probability that the in-
cumbent is the Stackelberg type, the history resulting from always
playing F after entry has positive probability. Applying Lemma
 218 CHAPTER 8. TOPICS IN DYNAMIC GAMES

8.3.1 yields a lower bound on the normal types payoff of

1
kX
X

(1 ) 1 + (1 ) 4 = 1 k + 4k = 1 + 3k .
=0 =k

This can be made arbitrarily close to 4 by choosing close to 1.

8.4 Problems
8.4.1. (a) Suppose (1 , 2 ) is an MPE of the fisheries game from Example
8.1.1 satisfying 1 (s) + (s) < s for all s. Prove that the profile
remains an MPE of the dynamic game where payoffs are given
by
log ai ,
ui (s, a) = n o if a1 + a2 s,
log ai
s , if a1 + a2 > s.
a1 +a2

(b) Prove that

1
ai (s) = s, i = 1, 2,
2
does indeed describe an MPE of the fisheries game described in
Example 8.1.1.

8.4.2. What is the symmetric MPE for the fisheries game of Example 8.1.2
when there are n players, and the transition function is given by
 P
q(s, a) = max 0, s i ai ,

where > 1?

8.4.3. (a) In the MPE calculated in Example 8.1.2, for what values of the
discount factor does the stock of fish grow without bound, and
for which values does the stock decline to extinction?
(b) This MPE is inefficient, involving excess extraction. To see this,
calculate the largest symmetric payoff profile that can by achieved
when the firms choose identical Markov strategies, and prove
that the efficient solution extracts less than does the MPE.
(c) Describe an efficient subgame perfect equilibrium for this game
(it is necessarily non-Markov).
 January 18, 2016 219

coalition 1s payoff 2s payoff 3s payoff

{1, 2} 9 3 0
{2, 3} 0 9 3
{1, 3} 3 0 9

Figure 8.4.1: Payoffs to players in each pairwise coalition for Problem

8.4.5. The excluded player receives a payoff of 0.

8.4.4. Consider the asynchronous move prisoners dilemma from Section

8.1.

(a) Suppose x 1. For some values of , there is a Markov perfect

equilibrium in which players randomize at E between E and S,
and play S for sure at S. Identify the bounds on and the prob-
ability of randomization for which the described behavior is an
MPE.

(b) Suppose that the initial action of player 1 is not exogenously

fixed. The game now has three states, the initial null state and
E and S. At the initial state, both players choose an action, and
then thereafter player 1 chooses an action in odd periods and
player 2 in even periods. Suppose x > 1 and satisfies (8.1.2).
Prove that there is no pure strategy MPE in which the players
choose E.

8.4.5. (A simplification of Livshits (2002).) There are three players. In the

initial period, a player i is selected randomly and uniformly to pro-
pose a coalition with one other player j, who can accept or reject.
If j accepts, the game is over with payoffs given in Figure 8.4.1. If
j rejects, play proceeds to the next period, with a new proposer
randomly and uniformly selected. The game continues with a new
proposer randomly and uniformly selected in each period until a
proposal is accepted. Thus, the game is potentially of infinite hori-
 220 CHAPTER 8. TOPICS IN DYNAMIC GAMES

zon, and if no coalition is formed (i.e., there is perpetual rejection),

all players receive a payoff of 0.

(a) Suppose < 3/4. Describe a stationary pure strategy Markov

perfect equilibrium. [Hint: in this equilibrium, every proposal
is immediately accepted.]
(b) Suppose > 3/4. Prove there is no Markov perfect equilibrium
in stationary pure strategies. There is a stationary Markov per-
fect equilibrium in behavior strategies. What is it? [Hint: The
randomization is on the part of the responder.]
p
(c) Suppose 3/4 < < 3/4. There are two nonstationaryppure
strategy Markov equilibria. What are they? [Hint: if < 3/4,
then 2 < 3/4.]
p
(d) Suppose 3/4 < < 3/4. Construct a non-Markov perfect
equilibrium in which in the first period, if 1 is selected, then
1 chooses 3.

8.4.6. Consider the model of Section 8.2, but assume the buyers valuation
v can only take on two values, 2 and 3. Moreover, the sellers beliefs
assign probability to the value 2. The sellers cost (value) is zero,
and the buyer and seller have a common discount factor (0, 1].

(a) What are the unique perfect Bayesian equilibria (PBE) of the one
period model (in this model, the seller makes a take-it-or-leave-
it offer to the buyer)?

Consider now the two period model, that is, if the buyer rejects the
offer in the first period, then the seller make a final offer in period
2, after which the game ends.

(b) What restrictions on period 2 pricing are implied by the almost

perfect Bayesian equilibrium concept.

For the rest of this question, interpret PBE as almost PBE.

(c) Prove that, in any PBE, both types of buyer must accept any first
period price strictly smaller than 2.
(d) Prove that, in any PBE, if a first period price p1 > 2 is rejected,
then the sellers posterior in the beginning of the second period
must assign probability at least to the low value buyer.
 January 18, 2016 221

1
(e) Suppose = 2 . Describe the unique pure strategy PBE.
1
(f) Suppose = 4 . Prove that there is no pure strategy PBE. [Hint:
Suppose p1 is the first period price in a candidate pure strategy
PBE. How should the seller respond to a rejection of a deviation
to a price p1 p1 ?]
1
(g) Suppose = 4 . Suppose 6/7. Describe the unique PBE
(from part 8.4.6(f), it is necessarily in mixed strategies).4

8.4.7. As in the model of Section 8.2, there is an uninformed seller with

cost zero facing a buyer with value uniformly distributed on [0, 1].
Suppose the seller has a rate of continuous time discounting of rS
(so the sellers discount factor is S = erS , where > 0 is the time
between offers), while the buyer has a rate of continuous time dis-
counting of rB (so the buyers discount factor is B = erB . Solve
for an equilibrium of the infinite horizon game in which the unin-
formed sellers makes all the offers. What happens to the initial offer
as 0?
8.4.8. Reconsider the two period reputation example (illustrated in Figure
8.3.2) with > 12 . Describe all of the equilibria. Which equilibria
survive the intuitive criterion?
8.4.9. Describe the equilibria of the three period version of the reputation
example.
8.4.10. Consider the following stage game where player 1 is the row player
and 2, the column player (as usual). Player 1 is one of two types n
and 0 . Payoffs are:

L R L R
T 2, 3 0, 2 T 3, 3 1, 2
B 3, 0 1, 1 B 2, 0 0, 1
n 0
4
A natural state variable in this game is the posterior probability assigned to
the low value buyer; but with this notion of state, this game does not have a
Markov perfect equilibrium. The notion of weak Markov equilibrium was intro-
duced by Fudenberg, Levine, and Tirole (1985) to accommodate problems like
this. A similar issue arises in the chain store paradox (see Kreps and Wilson
(1982a) and Remark 17.3.1 in Mailath and Samuelson (2006)).
 222 CHAPTER 8. TOPICS IN DYNAMIC GAMES

The stage game is played twice, and player 2 is short-lived: a differ-

ent player 2 plays in different periods, with the second period player
2 observing the action profile chosen in the first period. Describe all
the equilibria of the game. Does the intuitive criterion eliminate any
of them?

8.4.11. This is a continuation of Problem 7.6.12. Suppose now that the game
with the long-lived player 1 and short-lived player 2s is a game of
incomplete information. With prior probability (0, 1), player 1
is a behavioral type who chooses T in every period, and with prob-
ability 1 , he is a strategic or normal type as described above.
Suppose > 21 . Describe an equilibrium in which the normal type of
player 1 has a payoff strictly greater than 2 for large .

8.4.12. (Based on Hu (2014).) Reconsider the infinite horizon reputation

game of Section 8.3.2. In addition to the endogenous signal of F and
A, there is an exogenous signal z {z0 , z1 }, with

1 > Pr{z = z0 | 0 ) := > =: Pr{z = z0 | 0 ) > 0.

In period , entrant observes the history of entry decisions, the

behavior of the incumbent in any period in which there was entry,
and realizations of the exogenous signal.
Fix a Nash equilibrium.

(a) Prove that in any period in which there is entry, the probability
of F , conditional on the incumbent not being 0 is no larger
1
than 2 . Denote this probability by .
(b) Prove that if the incumbent is not 0 , and the entrants never
enter, than with probability one, the entrants posterior proba-
bility on 0 converges to zero.
(c) Let ht be a positive probability period-t history in which every
entry results in F , and there is entry in the last period. Provide
an upper bound on the fraction of periods in ht in which an
entrant enters. [Hint: Express the odds ratio in period + 1
after entry results in F in terms of the odds ratio in period ,
and use part 8.4.12(a).]
Chapter 9

Bargaining1

9.1 Axiomatic Nash Bargaining

A bargaining problem is a pair < S, d >, S R2 compact and con-
vex, d S, and s S such that si > di for i = 1, 2. Let B denote
the collection of bargaining problems. While d is often interpreted
as a disagreement point, this is not the role it plays in the axiomatic
treatment. It only plays a role in INV (where its role has the flavor
of a normalization constraint) and in SYM. The appropriate inter-
pretation is closely linked to noncooperative bargaining. It is not
the value of an outside option!

Definition 9.1.1. A bargaining solution is a function f : B R2 such

that f (S, d) S.

9.1.1 The Axioms

1. INV (Invariance to Equivalent Utility Representations)
Given < S, d >, and a pair of constants (i , i ) with i > 0
for each individual i = 1, 2, let < S 0 , d0 > be the bargaining
problem given by

S 0 = {(1 s1 + 1 , 2 s2 + 2 ) : (s1 , s2 ) S}
1
Copyright January 18, 2016 by George J. Mailath

223
224 CHAPTER 9. BARGAINING

and
d0i = i di + i , i = 1, 2.
Then
fi (S 0 , d0 ) = i fi (S, d) + i , i = 1, 2.

2. SYM (Symmetry)
If d1 = d2 and (s1 , s2 ) S = (s2 , s1 ) S, then

f1 (S, d) = f2 (S, d).

3. IIA (Independence of Irrelevant Alternatives)

If S T and f (T , d) S, then

f (S, d) = f (T , d).

4. PAR (Pareto Efficiency)

If s S, t S, ti > si , i = 1, 2, then

f (S, d) s.

9.1.2 Nashs Theorem

Theorem 9.1.1 (Nash). If f : B R2 satisfies INV, SYM, IIA, and
PAR, then

f (S, d) = arg max (s1 d1 )(s2 d2 ) f N (S, d).

(d1 ,d2 )(s1 ,s2 )S

If s1 + s2 1, then player Is Nash share is

1 + d1 d2
s1 =
2

(which is (9.4.1)).
January 18, 2016 225

Proof. Leave as an exercise that f N satisfies the four axioms.

Suppose that f satisfies the four axioms. Fix < S, d >.
Step 1: Let z = f N (S, d). Then zi > di , i = 1, 2. Apply the
following affine transformations to move d to the origin and z to
(1/2, 1/2):
1 di
i = ; i = .
2(zi di ) 2(zi di )
Denote the transformed problem < S 0 , 0 >.
INV implies
fi (S 0 , 0) = i fi (S, d) + i
and
1
fiN (S 0 , 0) = i fiN (S, d) + i = .
2
Note that fi (S, d) = fiN (S, d) if and only if fi (S 0 , 0) = 1/2.
Step 2: Claim - (s10 , s20 ) S 0 such that s10 + s20 > 1.
Suppose not. Then convexity of S 0 implies t = (1)(1/2, 1/2)+
s 0 S 0 . Moreover, for small, t1 t2 > 1/4, contradicting f N (S 0 , 0) =
(1/2, 1/2) (see Figure
 9.1.1).    
Step 3: Let T = (s1 , s2 ) R2 : s1 + s2 1, |si | max s10  , s20  : s 0 S 0
(see Figure 9.1.2). Then, SYM and PAR, f (T , 0) = (1/2, 1/2).
Step 4: Since S 0 T , IIA implies f (S 0 , 0) = (1/2, 1/2).

9.2 Rubinstein (1982) Bargaining

Two agents bargain over [0, 1]. Time is indexed by t, t = 1, 2, . . .. A
proposal is a division of the pie (x, 1 x), x 0. The agents take
turns to make proposals. Player I makes proposals on odd t and II
on even t. If the proposal (x, 1 x) is agreed to at time t, Is payoff
is t1 t1
1 x and IIs payoff is 2 (1x). Perpetual disagreement yields
a payoff of (0, 0). Impatience implies i < 1.
Histories are ht [0, 1]1 .
Strategies for player I, I1 : t odd [0, 1]t1 [0, 1], I2 : t even [0, 1]t
{A, R}, and for player II, II1 : t2 even [0, 1]t1 [0, 1] and II2 :
t odd [0, 1]t {A, R}.
226 CHAPTER 9. BARGAINING

(s10 , s20 )

( 12 , 12 )

1
xy = 4

x+y =1

Figure 9.1.1: Illustration of step 2.

Need to distinguish between histories in which all proposals

have been rejected, and those in which all but the last have been
rejected and the current one is being considered.

9.2.1 The Stationary Equilibrium

All the subgames after different even length histories of rejected
proposals are strategically identical. A similar comment applies
to different odd length histories of rejected proposals. Finally, all
the subgames that follow different even length histories of rejected
proposals followed by the same proposal on the table are strate-
gically identical. Similarly, all the subgames that follow different
odd length histories of rejected proposals followed by the same
proposal on the table are strategically identical.
Consider first equilibria in history independent (or stationary)
strategies. Recall that a strategy for player I is a pair of mappings,
(I1 , I2 ). The strategy I1 is stationary if, for all ht [0, 1]t1 and
January 18, 2016 227

 
1 1
S0 ,
2 2

max{|s10 |, |s20 | : s 0 S 0 }

Figure 9.1.2: The bargaining set T .

228 CHAPTER 9. BARGAINING

ht [0, 1]t1 , I1 (ht ) = I1 (ht ) (and similarly for the other strate-
gies). Thus, if a strategy profile is a stationary equilibrium (with
agreement), there is a pair (x , z ), such that I expects x in any
subgame in which I moves first and expects z in any subgame
in which II moves first. In order for this to be an equilibrium, Is
claim should make II indifferent between accepting and rejecting:
1 x = 2 (1 z ), and similarly I is indifferent, so z = 1 x .
[Consider the first indifference. Player I wont make a claim that
II strictly prefers to 1 z next period, so 1 x 2 (1 z ). If
II strictly prefers (1 z ) next period, she rejects and gets 1 z
next period, leaving I with z . But I can offer II a share 1 z this
period, avoiding the one period delay.] Solving yields

x = (1 2 )/(1 1 2 ),

and
z = 1 (1 2 )/(1 1 2 ).
The stationary subgame perfect equilibrium (note that backward
induction is not well defined for the infinite horizon game) is for I
to always claim x and accept any offer z , and for II to always
offer z and always accept any claim x . Note the implicit appeal
to the one-shot deviation principle, which also holds here (the proof
is along similar lines, but simpler than, the proof of Theorem 7.1.3).

9.2.2 All Equilibria

While in principal, there could be nonstationary equilibria, it turns
out that there is only one subgame perfect equilibrium. The anal-
ysis will also accommodate the possibility of delay in agreement
(in other words, we do not want to assume that equilibrium offers
necessarily result in agreement).
Denote by i/j the game in which i makes the initial proposal to
j. Define

Mi = sup is discounted expected payoff

in any subgame perfect eq of i/j
January 18, 2016 229

and

mi = inf is discounted expected payoff

in any subgame perfect eq of i/j .

Claim 9.2.1. mj 1 i Mi .

Proof. Note first that i must, in equilibrium, accept any offer >
i Mi .
The claim is proved by proving that in every equilibrium, player
js payoff in j/i is at least 1 i Mi . Suppose not, that is, suppose
there exists an equilibrium yielding a payoff uj < 1 i Mi to j.
But this is impossible, since j has a profitable deviation in such an
equilibrium: offer i Mi + , small. Player i must accept, giving j a
payoff of 1 i Mi > uj , for sufficiently small.

Claim 9.2.2. Mj 1 i mi .

Proof. If j makes an equilibrium offer that i accepts, then i must

be getting at least i mi (since otherwise, i should reject the offer
to receive a guaranteed discounted value of at least i mi ). This
implies that j gets no more than 1 i mi .
The other possibility is that j makes an equilibrium offer that
i rejects (in equilibrium). Then, in equilibrium, in the game i/j, i
cannot offer j more than j Mj . In this case, js payoff is no more
than 2j Mj .
So,
2
Mj max{1 i mi , j Mj }
| {z } | {z }
if i accepts if i rejects

= Mj 1 i mi .

The first claim implies

Mj 1 i (1 j Mj )
(1 i ) (1 j )
= Mj , Mi .
(1 i j ) (1 i j )
230 CHAPTER 9. BARGAINING

This implies
(1 i ) (1 j )
mi 1 j =
(1 i j ) (1 i j )

and so
(1 j )
mi = Mi = .
(1 i j )

9.2.3 Impatience
In order to investigate the impact of reducing the bargaining fric-
tion intrinsic in impatience, we do the following:
Time is continuous, with each round of bargaining taking
units of time. If player i has discount rate ri ,

i = eri .

Player 1s share is then

1 2 1 er2
x () = =
1 1 2 1 e(r1 +r2 )
and so
1 er2
lim x () = lim
0 0 1 e(r1 +r2 )
r2 er2
= lim
0 (r1 + r2 )e(r1 +r2 )
r2
= ,
r1 + r2
where lHopitals rule was used to get to the second line.
Note that the first mover advantage has disappeared (as it should).
The bargaining is determined by relative impatience.

9.3 Outside Options

Player II has an outside option of value (0, b).
January 18, 2016 231

x 1 [0, 1]

II

O
R A
II
(0, b) (x 1 , 1 x 1 )
x 2 [0, 1]

R A
I
(1 x 2 , 2 (1 x 2 ))
x 3 [0, 1]

Figure 9.3.1: The first two periods when II can opt out only after rejecting
Is proposal.

9.3.1 Version I
Suppose player II can only select outside option when rejecting Is
proposal, and receives b in that period. See Figure 9.3.1 for the
extensive form.
Claim 9.3.1. m2 1 1 M1 .
Proof. Same argument as Claim 9.2.1.
Claim 9.3.2. M1 1 b, M1 1 2 m2 .
Proof. M1 1 b (since II can always opt out). M1 1 2 m2
follows as in case without outside option (Claim 9.2.2).
232 CHAPTER 9. BARGAINING

Claim 9.3.3. m1 1 max {b, 2 M2 }, M2 1 1 m1 .

Proof. If b 2 M2 , then the argument from Claim 9.2.1 shows that

m1 1 2 M2 . If b > 2 M2 , then II takes the outside option rather
than rejecting and making a counterproposal. Thus, IIs acceptance
rule is accept any proposal of a share > b, and take the outside
option for any proposal < b. Thus, Is payoffs is 1 b.
M2 1 1 m1 follows as in case without outside option (Claim
9.2.2).

Claim 9.3.4. b 2 (11 )/ (1 1 2 ) = mi (1j )/ (1 1 2 )

Mi .

Proof. Follows from the Rubinstein shares being equilibrium shares.

Claim 9.3.5. b 2 (11 )/ (1 1 2 ) = mi = (1j )/ (1 1 2 ) =

Mi

Proof. From Claim 9.3.2, 1 M1 2 m2 , and so from claim 9.3.1,

1 M1 2 (1 1 M1 ), and so M1 (1 2 ) / (1 1 2 ) and so we
have equality.
From Claim 9.3.1, m2 11 (1 2 ) / (1 1 2 ) = (1 1 ) / (1 1 2 ),
and so equality again.
From Claim 9.3.4, 2 M2 2 (1 1 )/ (1 1 2 ) b, and so
by Claim 9.3.3, m1 1 2 M2 1 2 (1 1 m1 ). Thus, m1
(1 2 ) / (1 1 2 ), and so equality.
Finally, from Claim 9.3.3,

M2 1 1 m1 = 1 1 (1 2 ) / (1 1 2 ) = (1 1 ) / (1 1 2 ) .

Thus, if b 2 (11 )/ (1 1 2 ), equilibrium payoffs are uniquely

determined. If b < 2 (1 1 )/ (1 1 2 ), then the subgame per-
fect equilibrium profile is also uniquely determined (player II never
takes the outside option). If b = 2 (1 1 )/ (1 1 2 ), then there
are multiple subgame perfect equilibrium profiles, which differ in
whether player II takes the outside option or not after an unaccept-
able offer.
January 18, 2016 233

Claim 9.3.6. b > 2 (1 1 )/ (1 1 2 ) = m1 1 b M1 , m2

1 1 (1 b) M2 .

Proof. Follows from the following being an equilibrium: I always

proposes 1b, and accepts any offer of at least 1 (1 b); II always
proposes 1 (1 b) and accepts any claim of no more than 1 b,
opting out if the claim is more than 1 b.

Claim 9.3.7. b > 2 (1 1 )/ (1 1 2 ) = m1 = 1 b = M1 , m2 =

1 1 (1 b) = M2 .

Proof. From Claim 9.3.2, 1 M1 b, i.e., M1 1 b, and so we have

equality.
From Claim 9.3.1, m2 1 1 (1 b),and so we have equality.
From Claim 9.3.6, 1 b m1 and so (1 2 )) / (1 1 2 ) > m1 .
We now argue that 2 M2 b. If 2 M2 > b, then m1 1 2 M2
12 (1 1 m1 ) and so m1 (1 2 ) / (1 1 2 ), a contradiction.
Thus, 2 M2 b.
From Claim 9.3.6 and 9.3.3, 1 b m1 1 max{b, 2 M2 } =
1 b and so m1 = 1 b.
Finally, this implies M2 1 1 m1 = 1 1 (1 b), and so
equality.
Thus, if b > 2 (11 )/ (1 1 2 ), equilibrium payoffs are uniquely
determined. Moreover, the subgame perfect equilibrium profile is
also uniquely determined (player II always takes the outside option
after rejection):

b > 2 (1 1 ) / (1 1 2 ) b > 2 [1 1 (1 b)] .

9.3.2 Version II
If II can only select outside option after I rejects (receiving b in that
period, see Figure 9.3.2), then there are multiple equilibria. The
equilibrium construction is a little delicate in this case. In fact, for
many parameter constellations, there is no pure strategy Markov
perfect equilibrium.
There is, however, a Markov perfect equilibrium in behavior strate-
gies. To illustrate the role of the Markov assumption it is useful to
234 CHAPTER 9. BARGAINING

x 1 [0, 1]

II

R A
II
(x 1 , 1 x 1 )
x 2 [0, 1]

R
II A
O
O

I
(0, 2 b) (1 x 2 , 2 (1 x 2 ))
x 3 [0, 1]

Figure 9.3.2: The first two periods when II can opt out only after I rejects
his proposal.
January 18, 2016 235

discuss the construction. Note first that, in contrast to the game

in Figure 9.3.1, there is only one state at which II can take the out-
side option. In a Markov strategy profile, I claims x 1 in every odd
period, and II offers x 2 in every even period. If in a putative pure
strategy MPE, the outside option is not taken, then we must have,
as in Rubinstein,

1 x 1 = 2 (1 x 2 )
and x 2 = 1 x 1 ,

implying the Rubinstein shares for the two players. This can only be
an equilibrium if it is indeed optimal for II not to take the outside
option:
22 (1 1 )
b 2 (1 x 1 ) = .
(1 1 2 )

Suppose now II always takes the outside option. Then, x 2 = 0

and x 1 = 1 2 . This is an equilibrium only if it is indeed optimal
for II to take the outside option:

b 2 (1 x 1 ) = 22 .

Suppose
22 (1 2 )
< b < 22 . (9.3.1)
(1 1 2 )

Then, there is no pure strategy MPE. Let (0, 1) be the probability

that the outside option is not taken. Then, if (x 1 , x 2 , ) describes
an MPE, we must have (make sure you understand why we need
each equality):

1 x 1 = 2 (1 x 2 ),
x 2 = 1 x 1 ,
and b = 2 (1 x 1 ).
236 CHAPTER 9. BARGAINING

We then have
b
x1 = 1 ,
2
b
x2 = 1 2 ,
2
22 b
and = .
1 2 (2 b)
It is easy to check that (0, 1) under (9.3.1).
Note the critical role played by the timing of the decision on the
outside option. In particular, even though I strictly prefers that II
never take the outside option, I is not able to prevent it, since Is
offer is only made after II decides not to take it. In Figure 9.3.1, on
the other hand, I always has the option of claiming 1 x 1 > b.

9.4 Exogenous Risk of Breakdown

Suppose that after any rejection, there is a probability 1 of
breakdown and the outcome (d1 , d2 ) is implemented. With prob-
ability , bargaining continues to the next round. No discounting.
Note that since always rejecting is a feasible strategy, di mi Mi .
Claim 9.4.1. mj 1 Mi (1 ) di .
Proof. Note first that i must, in equilibrium, accept any offer >
Mi + (1 ) di . Suppose mj < 1 Mi (1 ) di . Then there
would exists an eq yielding a payoff uj < 1 Mi (1 ) di to
j. But j can deviate in such an eq, and offer Mi + (1 ) di + ,
small, which i accepts. This gives j a payoff of 1Mi (1 ) di
> uj , for sufficiently small.
Claim 9.4.2. Mj 1 mi (1 ) di .
Proof. In eq, i rejects any offer < mi + (1 ) di and then i offers
no more than Mj + (1 ) dj . So,
  
Mj max 1 mi (1 ) di , Mj + (1 ) dj + (1 ) dj
= Mj 1 mi (1 ) di ,


since Mj < 2 Mj + 1 2 dj Mj < dj
January 18, 2016 237

The first claim implies

Mj 1 1 Mj (1 ) dj (1 ) di



1 + dj di 1 + di dj
= Mj , Mi .
(1 + ) (1 + )

This implies


1 + dj di 1 + di dj
mi 1 (1 ) dj = = Mi
(1 + ) (1 + )

and so
1 + di dj
mi = Mi = .
(1 + )
Now, we are interested in the payoffs as 1, and

1 + di dj
mi ,
2

so that Is share is
1 + d1 d2
x = . (9.4.1)
2
For much more on bargaining, see Osborne and Rubinstein (1990).

9.5 Problems
9.5.1. A three person bargaining problem is a pair < S, d >, where S
R3 is compact and convex, d S, and s S such that si > di
for i = 1, 2, 3. Let B denote the collection of bargaining problems,
and as for the two person case, a bargaining solution is a function
f : B R3 such that f (S, d) S. The Nash axioms extend in the
obvious manner. Prove that if a bargaining solution satisfies INV,
SYM, IIA, and PAR, then

f (S, d) = arg max (s1 d1 )(s2 d2 )(s3 d3 ).

(s1 ,s2 ,s3 )S,
di si ,i=1,2,3
 238 CHAPTER 9. BARGAINING

9.5.2. Two agents bargain over [0, 1]. Time is indexed by t, t = 1, 2, . . . , T ,

T finite. A proposal is a division of the pie (x, 1 x), x 0. The
agents take turns to make proposals. Player I makes proposals on
odd t and II on even t. If the proposal (x, 1 x) is agreed to at
time t, Is payoff is t1 t1
1 x and IIs payoff is 2 (1 x). Perpetual
disagreement yields a payoff of (0, 0). Impatience implies i < 1.
The game ends in period T if all previous proposals have been re-
jected, with each receiving a payoff of zero.

(a) Suppose T odd, so that I is the last player to make a proposal.

If T = 1, the player I makes a take-it-or-leave-it offer, and so in
equilibrium demands the entire pie and II accepts. Prove that
in the unique backward induction equilibrium, if there are k
periods remaining, where k is odd and k 3, Is proposal is
given by
1
X r
xk = (1 2 ) (1 2 ) + (1 2 ) , = (k 1)/2.
r =0

[Hint: First calculate x1 (the offer in the last period), x2 , and

x3 . Then write out the recursion, and finally verify that the
provided expression satisfies the appropriate conditions.]
(b) What is the limit of xT as T ?
(c) Suppose now that T is even, so that II is the last player to make
a proposal. Prove that in the unique backward induction equi-
librium, if there are k periods remaining, where k is even and
k 2, Is proposal is given by
1
X r
yk = (1 2 ) (1 2 ) , = k/2.
r =0

(d) What is the limit of yT as T ?

9.5.3. (a) Give the details of the proof of Claim 9.3.4. (A few sentences
explaining why it works is sufficient.)
(b) Give the details of the proof of Claim 9.3.6. (A few sentences
explaining why it works is sufficient.)

9.5.4. We will use the finite horizon bargaining result from question 9.5.2
to give an alternative proof of uniqueness in the Rubinstein model.
 January 18, 2016 239

(a) Prove that in any subgame perfect equilibrium of the game in

which I offers first, Is payoff is no more than xk , for all k odd.
[Hint: Prove by induction (the result is clearly true for k = 1).]
(b) Prove that in any subgame perfect equilibrium of the game in
which I offers first, Is payoff is no less than yk , for all k even.
(c) Complete the argument.

9.5.5. Let G(n) denote Rubinsteins alternating offer bargaining game with
discounting with the following change: the share going to player
1 must be an element of the finite set A(n) = {0, n 1
, . . . , (n1)
n , 1},
where n {1, 2, 3, . . .}. Suppose the two players have identical dis-
count factors .

(a) Show that for any n, there exists 0 (0, 1) such that for any
(0 , 1), the continuum case proof that there is a unique
partition of the pie achievable in a subgame perfect equilibrium
fails for G(n). Show that any share to player 1 in A(n) can be
achieved in a subgame perfect equilibrium of G(n).
(b) Prove or provide a counterexample to the following claim: For
any (0, 1) and any > 0, there exists n0 such that for
any n > n0 the share going to player 1 in any subgame perfect
equilibrium is within of the Rubinstein solution.
(c) Suppose
1 k
k.
1+ n
Prove that this game does not have a symmetric stationary equi-
librium. Describe the two stationary equilibria that, for large n,
approximates the Rubinstein solution.

9.5.6. There is a single seller who has a single object to sell (the sellers
reservation utility is zero). There are two potential buyers, and they
each value the object at 1. If the seller and buyer i agree to a trade at
price p in period t, then
the seller receives a payoff of t1 p, buyer
i a payoff of t1 1 p , and buyer j i a payoff of zero. Con-
sider alternating offer bargaining, with the seller choosing a buyer
to make an offer to (name a price to). If the buyer accepts, the
game is over. If the buyer rejects, then play proceeds to the next pe-
riod, when the buyer who received the offer in the preceding period
makes a counter-offer. If the offer is accepted, it is implemented.
 240 CHAPTER 9. BARGAINING

If the offer is rejected, then the seller makes a new proposal to ei-
ther the same buyer or to the other buyer. Thus, the seller is free
to switch the identity of the buyer he is negotiating with after every
rejected offer from the buyer.

(a) Suppose that the seller can only make proposals in odd-numbered
periods. Prove that the sellers subgame perfect equilibrium
payoff is unique, and describe it. Describe the subgame perfect
equilibria. The payoffs to the buyers are not uniquely deter-
mined. Why not?
(b) Now consider the following alternative. Suppose that if the
seller rejects an offer from the buyer, he can either wait one
period to make a counteroffer to this buyer, or he can immedi-
ately make an offer to the other buyer. Prove that the sellers
subgame perfect equilibrium payoff is unique, and describe it.
Describe the subgame perfect equilibria. [Cf. Shaked and Sut-
ton (1984).]

9.5.7. Extend alternating offer bargaining to three players as follows. Three

players are bargaining over a pie of size 1. The game has perfect in-
formation and players take turns (in the order of their index). A pro-
posed division (or offer) is a triple (x1 , x2 , x3 ), with xi being player
is share. All players must agree on the division. Play begins in
period 1 with player 1 making a proposal, which player 2 either ac-
cepts or rejects. If 2 accepts, then 3 either accepts or rejects. If both
accept, the offer is implemented. If either reject, the game proceeds
to period 2, where player 2 makes a proposal, which player 3 either
accepts or rejects. If 3 accepts, then 1 either accepts or rejects. If
both accept, the offer is implemented in period 2. If either reject,
then play proceeds to period 3, where 3 makes a proposal and 1 and
2 in turn accept or reject. Players rotate proposals and responses
like this until agreement is reached. Player i has discount factor
i (0, 1).
(a) What is the stationary Markov equilibrium? Is it unique?
(b) [Hard] This game has many non-Markovian equilibria. Describe
one.
9.5.8. (Based on Fernandez and Glazer (1991).2 ) A firm and its employee
union are negotiating a new wage contract. For simplicity assume
2
See Busch and Wen (1995) for a more general treatment.
January 18, 2016 241

the new contract will never be renegotiated. Normalize the value of

output produced in each period to 1. The current wage is w [0, 1].
The union and the firm alternate in making wage offers over discrete
time: t = 1, 2, . . . . In each odd period t, the union proposes a wage
offer x. The firm then responds by either accepting (A) or rejecting
(R). If the firm accepts the offer, negotiations are over and the newly
agreed upon wage is implemented: w = x and = 1 x for all
t (where w is the wage and is the profit in period ). If the
firm rejects the wage offer, the union decides whether to strike (s) or
not to strike (n). If the union decides not to strike, workers work and
receive the old wage wt = w, and the firm receives t = 1 w. If the
union decides to strike, workers get wt = 0 and the firm gets t = 0.
After the unions strike decision, negotiations continue in the next
period. In every even period, the firm offers the union a wage offer
z. The union then responds by either accepting (A) or rejecting (R).
Once again acceptance implies the newly agreed upon wage holds
thereafter. If the union rejects, the union again must decide whether
to strike (s) or not (n). After the union strike decision, negotiations
continue in the next period.
Both the firm and workers discount future flow payoffs with the
same discount factor (0, 1). The unions objective is to maxi-
mize workers utility:

X
(1 ) t1 wt .
t=1

The firms objective is to maximize its discounted sum of profits:

X
(1 ) t1 t .
t=1

(a) If the union is precommitted to striking in every period in which

there is a disagreement (i.e., the union does not have the option
of not striking), then there is a unique subgame perfect equi-
librium of the game. Describe it (you are not asked to prove
uniqueness).
(b) Explain why the option of not striking is not an outside option
(in the sense of Section 9.3).
(c) Describe a stationary Markov equilibrium.
242 CHAPTER 9. BARGAINING

2
(d) Suppose 0 < w 1+ . There is a subgame perfect equilib-
rium which gives the same equilibrium outcome as that in part
9.5.8(a). Describe it and verify it is an equilibrium. (Hint: Use
the equilibrium you found in part 9.5.8(c).)
(e) Describe an equilibrium with inefficient delay.
Chapter 10

Appendices1

10.1 Proof of Theorem 2.4.1

It will simplify notation to now assume that each action can only be
taken at one information set (i.e., A(h) A(h0 ) = for all h h0 ).
With this assumption, behavior strategies can be defined as follows:

Definition 10.1.1. A behavior strategy for player i is a function bi :

Ai [0, 1] such that, for all h Hi ,
X
bi (a) = 1.
aA(h)

We then have that the behavior strategy corresponding to a pure

strategy si is given by

bi (si (h)) = 1, h Hi ,

and
bi (a) = 0, a hHi {si (h)}.
Natures move is now denoted by b0 . Given b = (b0 , b1 , . . . , bn ),
the game is played as follows: play begins with player (t0 ) ran-

domly selects an action in A(h) according to the distribution b(h) (a) aA(h) .
All distributions are independent. The induced distribution on Z is
calculated as follows. Define recursively p n (t) = p(p n1 (t)) for
1
Copyright January 18, 2016 by George J. Mailath

243
244 CHAPTER 10. APPENDICES

t t0 , and `(t) solves p `(t) (t) = t0 . Then

 the unique path to z Z 
`(z) `(z)
starts at t0 = p (z) and is given by p (z), p `(z)1 (z), . . . , p(z), z .


The action leading to p j1 (z) is p j1 (z) , which is taken by
(p j (z))with probability
  
b(pj (z)) p j1 (z) .

Thus, the probability that z occurs is

`(z)
Y   
b
P (z) = b(pj (z)) p j1 (z) .
j=1

If ih is the number of actions available to i at h, a mixed strat-

Q
egy for i is a point in the i (= hHi ih 1)-dimensional simplex.
A behavior strategy, on the other hand involves  specifying
 i =
P h h
hHi (i 1) real numbers, one point on a i 1 -dimensional
simplex for each h Hi . In general, i >> i .

Definition 10.1.2. The mixed strategy i corresponding to a behav-

ior strategy bi is given by
Y
i (si ) = bi (si (h)).
hHi

Definition 10.1.3. Two strategies for a player are realization equiv-

alent if, for all specifications of strategies for the opponents, the two
strategies induce the same distribution over terminal nodes.

It is easy to check that the a behavior strategy and the mixed

strategy corresponding to it are realization equivalent.

Definition 10.1.4. A node t T is reachable under a mixed strat-

0
egy i (Si ) for i if there exists i such that the probability of
0
reaching t under (i , i ) is positive. An information set h Hi is
reachable under i if some node t h is reachable under i .

Remark 10.1.1. Perfect recall implies that if h Hi is reachable for

i under i , then all t h are reachable under i .
January 18, 2016 245

Define R(i ) = {h Hi : h is reachable for i under i }.

Definition 10.1.5. The behavior strategy, bi , generated by i is
X , X




i (si ) i (si ) , if h R(i ),



si Si s.t. si Si s.t.

hR(si ) and si (h)=a hR(si )
bi (a) =

X



i (si ), if h R(i ),




si Si s.t.
si (h)=a

for all h Hi and for all a A(h).

P
Note that h R(i ) implies si Si s.t. hR(si ) i (si ) 0. If h
R(i ), bi (a) is the probability that a will be chosen, conditional
on h being reached. The specification of bi (a) for h R(i ) is
arbitrary.
Theorem 10.1.1 (Kuhn, 1953). Fix a finite extensive game with per-
fect recall. For all mixed strategies, i (Si ), the behavior strategy
bi generated by i is realization equivalent to i .
Proof. Fix z Z. Let t1 , . . . , t` be is decision nodes (in order) in the
path to z, and ak is the action at tk required to stay on the path.
Independence across players means that it is enough to show
i : = Pr{ak is chosen at tk k|i }
= Pr{ak is chosen at tk k|bi } = bi .

Let hk be the information set containing tk .

Suppose there exists k such that hk R(i ). Then there exists
k0 < k such that Pr (ak0 |i ) = 0 and so i = 0. Letting k0 be the
smallest first k such that Pr (ak0 |i ) = 0, we have bi (ak0 ) = 0, and
so bi = 0.
Suppose hk R(i ) for all k. Then
X
i = i (si ).
si Si s.t.
si (hk )=ak , k`
246 CHAPTER 10. APPENDICES

Perfect recall implies that every information set owned by i is made

0
reachable by a unique sequence of is actions. Since si (hk ) = ak0
k0 < k implies hk R(si ),

hk+1 R(si ) hk R(si ), si (hk ) = ak .

Thus,
n o n o
k k k+1
si Si : h R(si ), si (h ) = ak = si Si : h R(si )

and
Y
bi = bi (ak )
k=1
,
Y X X
= i (si ) i (si )
k=1 si Si s.t. si Si s.t.
hk R(si ) and si (hk )=ak hk+1 R(si )
,
X X
= i (si ) i (si ) .
si Si : si Si :
h` R(si ),si (h` )=a` h1 R(si )

But denominator equals 1, so

X
bi = i (si )
si Si :
h` R(si ),si (h` )=a`
X
= i (si ) = i .
si Si :
si (hk )=ak ,1k`

10.2 Trembling Hand Perfection

10.2.1 Existence and Characterization
This subsection proves the equivalence between Definition 2.5.1
and Seltens (1975) original definition.
January 18, 2016 247

P
Let : Si (0, 1) be a function satisfying si Si (si ) < 1 for
all i. The associated perturbed game, denoted (G, ), is the normal

form game {(R1 , v1 ), . . . , (Rn , vn )} where

Ri = {i (Si ) : i (si ) (si ), si Si }

and vi is expected payoffs. Note that is a Nash equilibrium of

(G, ) if and only if for all i, si , si0 Si ,


vi (si , i ) < vi si0 , i = i (si ) = (si ) .

Definition 10.2.1 (Selten (1975)). An equilibrium of a normal

form game G is (normal form) trembling hand perfect if there ex-
ists a sequence {k }k such that k (si ) 0 s k k and an
 i kas
associated sequence of mixed strategy profiles k with a Nash
equilibrium of (G, k ) such that k as k .

Theorem 10.2.1. Every finite normal form game has a trembling

hand perfect equilibrium.

Proof. Clearly (G, ) has a Nash equilibrium for all . Suppose {m }

is a sequence such that m (si ) 0 si as m . Let m be an
equilibrium
Q of (G, m ). Since { m } is a sequence in the compact set
i (Si ), it has a convergent subsequence. Its limit is a trembling
hand perfect equilibrium of G.

Remark 10.2.1. If is not trembling hand perfect, then there exists

> 0 such that for all sequences {k }k satisfying k 0, eventually
all Nash equilibria of the associated perturbed games are bounded
away from by at least  (i.e., K so that k K and k equilib-
rium of (G, k ),  k  ).

Definition 10.2.2 (Myerson (1978)). The mixed strategy profile

is an -perfect equilibrium of G if it is completely mixed (i (si ) >
0 si Si ) and satisfies

si BRi (i ) = i (si ) .
248 CHAPTER 10. APPENDICES

Theorem 10.2.2. Suppose is a strategy profile of the normal form

game G. The following are equivalent:

1. is a trembling hand perfect equilibrium of G;

2. there exists a sequence {k : k 0} and an associated sequence

of k -perfect equilibria converging to ; and

3. there exists a sequence k of completely mixed strategy pro-
k
files converging to such that i is a best reply to i , for all
k.

Proof. (1) (2). Take k = maxsi Si , i k (si ).


(2) (3). Let k be the sequence of k -perfect equilibria.
Suppose si receives positive probability under i . Need to show
k
that si is a best reply to i . Since k 0 and ik (si ) i (si ) > 0,
there exists k (si ) such that k > k (s i ) implies
 ik (si ) > i (si )/2 >
k
k . But k is k -perfect, so si BRi i . The desired sequence is
n o
k : k > k , where k max {k (si ) : i (si ) > 0, i}.
(3) (1). Define k as

k (si ), if i (si ) = 0,
i
k (si ) =
1/k, if i (si ) > 0.

P
Since k , there exists k0 such that k > k0 implies si Si k (si ) <
1 for all i.
Let m = min  {i (si ) : i (si) > 0, i}. There exists k such that,
00
 
for all k > k00 , ik (si ) i (si ) < m/2. Suppose k > max {k0 , k00 , 2/m}.
Then ik (si ) k (si ). [If i (si ) = 0, then immediate. If i (si ) > 0,
then ik (si ) > i (si ) m/2 > m/2 > 1/k.]
k k
Since i is a best reply to i , if si is not a best reply to i ,
k
then i (si ) = 0. But this implies that is an equilibrium of (G, k )
(since si is played with minimum probability).
January 18, 2016 249

10.2.2 Extensive form trembling hand perfection

Definition 10.2.3. An equilibrium b (b1 , . . . , bn ) of a finite exten-
sive from game is extensive
form trembling hand perfect if there
exists a sequence bk k of completely mixed behavior strategy pro-
files converging to b such that for all players i and information sets
h Hi , conditional on reaching h, for all k, bi (h) maximizes player
k
is expected payoff, given bi and bik (h0 ) for h0 h.
Theorem 10.2.3. If b is an extensive form trembling hand perfect
equilibrium of the finite extensive from game , then it is subgame
perfect.
Proof. This is a corollary of Theorem 10.2.4.
Remark 10.2.2. When each player only has one information set
(such as in Seltens horse), trembling hand perfect in the normal
and extensive form coincide. In general, they differ. Moreover,
trembling hand perfect in the normal form need not imply subgame
perfection (see problems 2.6.23 and 10.3.1).

Remark 10.2.3. Note that trembling hand perfect in the extensive

form requires players to be sensitive not only to the possibility of
trembles by the other players, but also their own trembles (at other
information sets). The notion of quasi-perfect drops the latter re-
quirement:
Definition 10.2.4 (van Damme (1984)). A behavior strategy profile
b is quasi-perfect if it is the limit of a sequence of completely mixed
behavior profiles bn , and if for each player i and information set h
owned by that player, conditional on reaching h, bi is a best response
n
to bi for all n.

Theorem 10.2.4. Suppose b is a extensive form trembling hand per-

fect equilibrium of a finite extensive from game . Then, bis sequen-
tially rational given some consistent system of beliefs , and so is
sequential.
 250 CHAPTER 10. APPENDICES

Proof. Suppose b is trembling hand perfect in the extensive form.

Let {bk }k be the trembles, i.e., the sequence of completely mixed be-
havior strategy profiles converging to b. Let k be the system of be-
liefs implied by Bayes rule by bk (since bk is completely Q mixed, k is
well-defined). Since the collection of systems of beliefs, hi Hi (h),
is compact, the sequence { k } has a convergent subsequence with
limit , and so is consistent. Moreover, a few minutes of reflec-
k
tion reveals that for each i, (bi , bi ) is sequentially rational at every
information set owned by i, i.e., for all h Hi , given k . Since best
replies are hemicontinuous, for each i, b is sequentially rational at
every information set owned by i, i.e., for all h Hi , given . That
is, b is sequentially rational given .

10.3 Problems
10.3.1. By explicitly presenting the completely mixed trembles, show that
the profile L` is normal form trembling hand perfect in the following
game (this is the normal form from Example 2.3.4):

` r
L 2, 0 2, 0
T 1, 1 4, 0
B 0, 0 5, 1

Show that there is no extensive form trembling hand perfect equi-

librium with that outcome in the first extensive form presented in
Example 2.3.4.
Bibliography

Aumann, R. J. (1987): Correlated Equilibrium as an Expression of

Bayesian Rationality, Econometrica, 55(1), 118.

Ben-El-Mechaiekh, H., and R. W. Dimand (2011): A Simpler Proof

of the Von Neumann Minimax Theorem, American Mathematical
Monthly, 118(7), 636641.

Bernheim, B. D. (1984): Rationalizable Strategic Behavior, Econo-

metrica, 52(4), 10071028.

Bhaskar, V., G. J. Mailath, and S. Morris (2013): A Foundation

for Markov Equilibria in Sequential Games with Finite Social Mem-
ory, Review of Economic Studies, 80(3), 925948.

Busch, L.-A., and Q. Wen (1995): Perfect Equilibria in a Negotiation

Model, Econometrica, 63(3), 545565.

Carlsson, H., and E. van Damme (1993): Global Games and Equi-
librium Selection, Econometrica, 61(5), 9891018.

Carmichael, H. L., and W. B. MacLeod (1997): Gift Giving and the

Evolution of Cooperation, International Economic Review, 38(3),
485509.

Cho, I.-K., and D. Kreps (1987): Signaling Games and Stable Equi-
libria, Quarterly Journal of Economics, 102(2), 179221.

Elmes, S., and P. J. Reny (1994): On the Strategic Equivalence of

Extensive Form Games, Journal of Economic Theory, 62(1), 123.

251
252 BIBLIOGRAPHY

Fernandez, R., and J. Glazer (1991): Striking for a Bargain be-

tween Two Completely Informed Agents, American Economic Re-
view, 81(1), 24052.

Fudenberg, D., and D. K. Levine (1998): The Theory of Learning in

Games. MIT Press, Cambridge, MA.

Fudenberg, D., D. K. Levine, and J. Tirole (1985): Infinite-

Horizon Models of Bargaining with One-Sided Incomplete Infor-
mation, in Game-Theoretic Models of Bargaining, ed. by A. E.
Roth, pp. 7398. Cambridge University Press, New York.

Gibbons, R. (1992): Game Theory for Applied Economists. Princeton

University Press, Princeton, NJ.

Govindan, S., P. J. Reny, and A. J. Robson (2003): A Short Proof of

Harsanyis Purification Theorem, Games and Economic Behavior,
45(2), 369374.

Gul, F. (1998): A Comment on Aumanns Bayesian View, Econo-

metrica, 66(4), 923927.

Harris, C. J., P. J. Reny, and A. J. Robson (1995): The existence

of subgame-perfect equilibrium in continuous games with almost
perfect information: A case for public randomization, Economet-
rica, 63(3), 507544.

Harsanyi, J. C. (1973): Games with Randomly Disturbed Payoffs:

A New Rationale for Mixed-Strategy Equilibrium Points, Interna-
tional Journal of Game Theory, 2(1), 123.

Hu, J. (2014): Reputation in the presence of noisy exogenous learn-

ing, Journal of Economic Theory, 153, 6473.

Jackson, M. O., L. K. Simon, J. M. Swinkels, and W. R. Zame (2002):

Communication and Equilibrium in Discontinuous Games of In-
complete Information, Econometrica, 70(5), 17111740.

Kohlberg, E., and J.-F. Mertens (1986): On the Strategic Stability

of Equilibria, Econometrica, 54(5), 10031037.
January 18, 2016 253

Kreps, D., and R. Wilson (1982a): Reputation and Imperfect Infor-

mation, Journal of Economic Theory, 27(2), 253279.
(1982b): Sequential Equilibria, Econometrica, 50(4), 863
894.
Kuhn, H. W. (1953): Extensive Games and the Problem of Informa-
tion, in Contributions to the Theory of Games, vol. II of Annals of
Mathematical Studies 28. Princeton University Press.
Livshits, I. (2002): On Non-Existence of Pure Strategy Markov Per-
fect Equilibrium, Economics Letters, 76(3), 393396.
MacLeod, W. B., and J. M. Malcomson (1989): Implicit Contracts,
Incentive Compatibility, and Involuntary Unemployment, Econo-
metrica, 57(2), 447480.
Mailath, G. J. (1987): Incentive Compatibility in Signaling Games
with a Continuum of Types, Econometrica, 55(6), 13491365.
(1998): Do People Play Nash Equilibrium? Lessons From
Evolutionary Game Theory, Journal of Economic Literature, 36(3),
13471374.
Mailath, G. J., and L. Samuelson (2006): Repeated Games and Rep-
utations: Long-Run Relationships. Oxford University Press, New
York, NY.
(2014): Reputations in Repeated Games, in Handbook of
Game Theory, volume 4, ed. by H. P. Young, and S. Zamir. North
Holland.
Mailath, G. J., L. Samuelson, and J. M. Swinkels (1993): Ex-
tensive Form Reasoning in Normal Form Games, Econometrica,
61(2), 273302.
(1997): How Proper Is Sequential Equilibrium?, Games and
Economic Behavior, 18, 193218, Erratum, 19 (1997), 249.
Mailath, G. J., and E.-L. von Thadden (2013): Incentive Com-
patibility and Differentiability: New Results and Classic Applica-
tions, Journal of Economic Theory, 148(5), 18411861.
254 BIBLIOGRAPHY

Maskin, E., and J. Tirole (2001): Markov Perfect Equilibrium I.

Observable Actions, Journal of Economic Theory, 100(2), 191
219.

Morris, S. (2008): Purification, in The New Palgrave Dictionary

of Economics Second Edition, ed. by S. Durlauf, and L. Blume, pp.
779782. Macmillan Palgrave.

Morris, S., and H. S. Shin (2003): Global Games: Theory and Ap-
plications, in Advances in Economics and Econometrics (Proceed-
ings of the Eighth World Congress of the Econometric Society), ed.
by M. Dewatripont, L. Hansen, and S. Turnovsky. Cambridge Uni-
versity Press.

Myerson, R. B. (1978): Refinements of the Nash Equilibrium Con-

cept, International Journal of Game Theory, 7, 7380.

(1991): Game Theory: Analysis of Conflict. Harvard Univ.

Press., Cambridge, MA.

Myerson, R. B., and P. J. Reny (2015): Sequential equilibria of

multi-stage games with infinite sets of types and actions, Uni-
versity of Chicago.

Ok, E. A. (2007): Real Analysis with Economic Applications. Prince-

ton University Press.

Osborne, M. J., and A. Rubinstein (1990): Bargaining and Markets.

Academic Press, Inc., San Diego, CA.

Owen, G. (1982): Game Theory. Academic Press, second edn.

Pearce, D. (1984): Rationalizable Strategic Behavior and the Prob-

lem of Perfection, Econometrica, 52(4), 102950.

Rubinstein, A. (1982): Perfect Equilibrium in a Bargaining Model,

Econometrica, 50(1), 97109.

(1989): The Electronic Mail Game: Strategic Behavior under

Almost Common Knowledge, American Economic Review, 79(3),
385391.
January 18, 2016 255

Samuelson, L. (1997): Evolutionary Games and Equilibrium Selec-

tion. MIT Press, Cambridge, MA.

Selten, R. (1975): Reexamination of the Perfectness Concept for

Equilibrium Points in Extensive Games, International Journal of
Game Theory, 4, 2255.

Shaked, A., and J. Sutton (1984): Involuntary Unemployment as

a Perfect Equilibrium in a Bargaining Model, Econometrica, 52(6),
13511364.

Shapiro, C., and J. Stiglitz (1984): Equilibrium Unemployment

as a Worker Discipline Device, American Economic Review, 74(3),
433444.

Simon, L. K., and W. R. Zame (1990): Discontinuous Games and

Endogenous Sharing Rules, Econometrica, 58(4), 861872.

van Damme, E. (1984): A Relation between Perfect Equilibria in

Extensive Form Games and Proper Equilibria in Normal Form
Games, International Journal of Game Theory, 13, 113.

(1991): Stability and Perfection of Nash Equilibria. Springer-

Verlag, Berlin, second, revised and enlarged edn.

Vohra, R. V. (2005): Advanced Mathematical Economics. Routledge,

London and New York.

Weibull, J. W. (1995): Evolutionary Game Theory. MIT Press, Cam-

bridge.

Weinstein, J., and M. Yildiz (2007): A Structure Theorem for Ra-

tionalizability with Application to Robust Predictions of Refine-
ments, Econometrica, 75(2), 365400.
256 BIBLIOGRAPHY
Index

automaton, 159 equivalence

realization, 40
backward induction, 24 strategic, 14
Bayesian-Nash equilibrium, 72
best reply, 19 focal points, 42
common prior assumption, 73 game
extensive form, 10
decomposable, 179
normal form, 2
dominant strategy, 5
reduced, 14
domination
strategic form, 2
strict, 4, 34
weak, 4 information
iterated deletion, 9 incomplete, 71
enforced, 178 perfect, 12
equilibrium intuitive criterion, 137
Bayesian-Nash, 72
Markov perfect equilibrium, 203
Markov perfect, 203
Markov strategy, 203
Nash, 17
minmax
mixed strategies, 37
mixed strategy, 157
perfect Bayesian, 122
pure strategy, 156
almost, 122
mixed strategy, 33
signaling games, 135, 140
formal treatment, 89
weak, 121
perfect public, 186 Nash equilibrium, 17
sequential, 125 mixed strategies, 37
subgame perfect, 28
trembling hand perfect one-shot deviation, 117
extensive form, 129, 249 principle, 119, 128, 162, 187,
normal form, 41 195

257
258 INDEX

profitable, 117, 162 payoff, 215

outcome, 11, 12
outcome path, 11

payoff dominance, 42
perfect Bayesian equilibrium, 122
almost, 122
signaling games, 135, 140
weak, 121
perfect information, 12
perfect public equilibrium, 186
purification, 54, 80
Harsanyi, 55

renegotiation, 43
Riley outcome, 142
risk dominance, 43

security level, 34
sequential equilibrium, 125
sequential rationality, 117
strategy
behavior, 39
extensive form, 11
Markov, 203
mixed, 33
formal treatment, 89
reduced normal form, 14
strongly symmetric, 159
subgame, 28
subgame perfect equilibrium, 28
system of beliefs, 115

trembling hand perfect equilib-

rium
extensive form, 129, 249
normal form, 41
type, 71
behavioral, 215