Read 5 of 5

Playing repeatedly

Can the shadow of the future make rivals cooperate?

The cafés' prisoner's dilemma was played once. But two cafés across the street from each other set prices every day, and they'll be there tomorrow.

This reading asks how that changes players' behavior. Repetition lets each player make today's choice depend on what happened in the past, so a deviation today can be punished tomorrow. Whether that threat sustains cooperation depends on how much the players care about the future.

5.1

The same rivals, every day

In a , the same players play the same game, the , again and again. Each period's payoffs add up, and each player can condition what they do today on what happened in the past.

In this reading the stage game is the prisoner's dilemma below. It has the same structure as the cafés' pricing game. If a café charges $6, we call it cooperating, and if it charges $4, we call it defecting. Defect is a dominant strategy for both Alice and Bob: against Cooperate it pays 5 instead of 3, and against Defect it pays 1 instead of 0. So (Defect, Defect) is the only Nash equilibrium of the stage game, although both would be better off at (Cooperate, Cooperate).

Repetition changes what a strategy can be. In the one-shot game, Alice just picks an action. In the repeated game she can follow a rule such as "I'll cooperate as long as you do" or "if you defect, I'll defect from then on." Rules like these reward cooperation and punish deviation, and they make the future part of today's decision.

We'll suppose the game goes on indefinitely: neither player knows of a last period. The last section of this reading explains why that matters.

5.2

Valuing a stream of payoffs

Payoffs now arrive over time, and a dollar tomorrow is worth less than a dollar today. Today's dollar can be invested to earn interest, prices may rise, and there's always some chance that tomorrow's payment never comes, for example because the relationship ends.

We summarize all of this with a \(\delta\), between 0 and 1: one dollar received one period from now is worth \(\delta\) dollars today.

You may have seen the same idea as a . The discount rate \(r\) is the percentage rate used to bring future payments back to their present value, and it reflects interest, inflation, and risk. One dollar today grows to \(1+r\) dollars next period, so one dollar next period is worth \(1/(1+r)\) today. The two are linked by \(\delta=\dfrac{1}{1+r}\). A discount rate of 10% gives \(\delta=1/1.1\approx0.91\).

A dollar two periods from now is worth \(\delta^2\), and so on. What is an infinite stream of one dollar every period, starting today, worth? Call its value \(V\):

\(V\)\(=\)\(1+\delta+\delta^2+\delta^3+\cdots\)One dollar every period, discounted.\(\delta V\)\(=\)\(\delta+\delta^2+\delta^3+\cdots\)Multiply both sides by \(\delta\).\(V-\delta V\)\(=\)\(1\)Subtract the second line from the first: every term but the first cancels.\(V\)\(=\)\(\dfrac{1}{1-\delta}\)

With \(\delta=0.9\), a dollar every period is worth \(1/0.1=10\) today. With \(\delta=0.5\) it's worth 2. The larger \(\delta\) is, the more the future counts.

What if the stream stops after period \(T\)?

Suppose the dollar arrives in periods \(0, 1, \ldots, T\) and then stops. Call its value \(V_T\). It's the infinite stream minus the part that starts in period \(T+1\). That part is itself an infinite stream of one dollar a period, only pushed \(T+1\) periods into the future, so today it's worth \(\delta^{T+1}V\):

\(V_T\)\(=\)\(1+\delta+\delta^2+\cdots+\delta^T\)One dollar in each of periods 0 to \(T\).\(=\)\(V-\delta^{T+1}V\)The whole stream minus the part from period \(T+1\) on.\(=\)\(\dfrac{1-\delta^{T+1}}{1-\delta}\)Use \(V=1/(1-\delta)\).

With \(\delta=0.9\) and \(T=9\), ten payments are worth \((1-0.9^{10})/0.1\approx6.51\). As \(T\) grows, \(\delta^{T+1}\) shrinks toward 0 and \(V_T\) approaches \(1/(1-\delta)\).

5.3

Grim trigger

Repetition makes cooperation possible, but not automatic. The players need strategies that make cooperating pay. The idea behind all of them is a simple deal: keep cooperating and you'll be rewarded with cooperation tomorrow; deviate and you'll be punished. The question is whether such a deal can be a Nash equilibrium of the repeated game, so that neither player wants to break it. We'll study two strategies that can do this, grim trigger and tit-for-tat. They differ in how hard they punish.

Suppose Alice follows : cooperate as long as both players have always cooperated; after the first period in which anyone defects, defect forever. Should Bob keep cooperating?

Suppose Bob considers deviating in some period \(t\). Up to period \(t-1\) both have cooperated, and nothing Bob does now changes those payoffs. So we compare what he gets from period \(t\) on, valued in period \(t\).

If Bob keeps cooperating, he earns 3 every period:

\(V_{\text{cooperate}}\)\(=\)\(3+3\delta+3\delta^2+\cdots\)\(=\)\(\dfrac{3}{1-\delta}\)

If Bob deviates in period \(t\), he earns 5, because Alice still cooperates in that period. From period \(t+1\) on, Alice defects forever, and Bob's best reply is to defect too, so he earns 1 a period:

\(V_{\text{deviate}}\)\(=\)\(5+\delta+\delta^2+\cdots\)\(=\)\(5+\dfrac{\delta}{1-\delta}\)1 a period, starting in period \(t+1\).

Bob keeps cooperating if cooperating is worth at least as much as deviating:

\(\dfrac{3}{1-\delta}\)\(\geq\)\(5+\dfrac{\delta}{1-\delta}\)\(3\)\(\geq\)\(5(1-\delta)+\delta\)Multiply both sides by \(1-\delta\).\(3\)\(\geq\)\(5-4\delta\)\(\delta\)\(\geq\)\(\dfrac{1}{2}\)

Valued from period 0 instead, both streams would be multiplied by \(\delta^t\), which doesn't change the comparison. So the condition is the same in every period: if Bob doesn't want to deviate in period 0, he doesn't want to deviate later either.

The same holds for Alice. So if both follow grim trigger and \(\delta\geq\frac{1}{2}\), both cooperate in every period, and neither gains by deviating. The gain from deviating is 2 in period \(t\). The cost is 2 a period forever after, starting in period \(t+1\). With enough patience, the cost wins.

Is the punishment itself believable? After a deviation, grim trigger has both players defect forever. Since (Defect, Defect) is a Nash equilibrium of the stage game, neither player can do better by deviating from the punishment. So the threat behind grim trigger is credible in the sense of Reading 4.

Before you drag, predict the smallest discount factor at which cooperation survives. The top chart compares Bob's two payoff streams when he deviates in period \(t\). The bottom chart plots their discounted values against \(\delta\): cooperation is sustained where the green curve is above the red one.

5.4

Tit-for-tat

Grim trigger never forgives. is gentler: cooperate in the first period, then copy whatever the other player did in the previous period. A deviation is punished for one period, and if the deviator returns to cooperating, so does the punisher.

Against tit-for-tat, Bob has two ways to deviate in period \(t\), and cooperating has to beat both.

Deviate forever. Bob earns 5 in period \(t\) and, since Alice then defects every period, 1 a period after that. That's the same stream as under grim trigger, so it needs \(\delta\geq\frac{1}{2}\).

Deviate once, then return. In period \(t\) Bob earns 5. In period \(t+1\) Alice punishes by defecting while Bob is back to cooperating, so Bob earns 0. From period \(t+2\) on, both cooperate again. The two streams are the same from period \(t+2\) on, so compare periods \(t\) and \(t+1\):

\(3+3\delta\)\(\geq\)\(5+0\times\delta\)Cooperate against deviate once.\(3\delta\)\(\geq\)\(2\)\(\delta\)\(\geq\)\(\dfrac{2}{3}\)

Both conditions must hold, so tit-for-tat sustains cooperation when \(\delta\geq\frac{2}{3}\). That's more patience than grim trigger needs.

That's the general pattern. Grim trigger is the harshest punishment available, so it sustains cooperation for the widest range of discount factors. A forgiving strategy like tit-for-tat punishes less, so it never needs less patience than grim trigger and sometimes needs more. What it offers in return is a way back to cooperation after a mistake, which grim trigger never allows.

5.5

What the shadow of the future can and can't do

The result is a general one. In a repeated prisoner's dilemma, cooperation can be sustained when players are patient enough, and how patient they must be depends on the payoffs. A bigger gain from deviating today, or a milder punishment, needs a larger \(\delta\).

What makes \(\delta\) large in practice? Frequent interaction, so that the next period comes soon, and a relationship that is likely to last. Rivals that set prices daily and expect to compete with each other for years are in a good position to cooperate. A business that expects to close next month, or a rival that's about to be bought out, isn't.

Two limits matter.

A known last period unravels cooperation. Suppose both players know that the game ends after exactly 100 periods. In period 100 there's no tomorrow to protect, so the game is a one-shot prisoner's dilemma, and both defect. In period 99, both know that period 100 will be (Defect, Defect) whatever happens, so nothing today can be rewarded or punished tomorrow, and both defect again. Backward induction runs all the way back to period 1. That's why we assumed the game goes on with no known end.

Cooperation is possible, not guaranteed. Even when \(\delta\geq\frac{1}{2}\), both players defecting in every period is still an equilibrium of the repeated game. Repetition adds cooperative equilibria; it doesn't remove the bad one.

5.6

Looking ahead to Week 7

This week's games had two or three strategies per player. In Week 7, firms choose any price or any quantity, and the same tools give us the equilibria of oligopoly markets. The prisoner's dilemma comes back too: a cartel is a group of firms trying to keep prices high when each would gain by cutting its own price, and the repeated game tells us when that can last.

Summary

In a repeated game, today's choice can be rewarded or punished tomorrow. An infinite stream of one dollar a period is worth \(1/(1-\delta)\), where \(\delta\) is the discount factor. Under grim trigger, the players in the prisoner's dilemma keep cooperating if \(3/(1-\delta)\geq5+\delta/(1-\delta)\), that is \(\delta\geq\frac{1}{2}\). Tit-for-tat punishes less and needs \(\delta\geq\frac{2}{3}\). Cooperation needs a future that matters and no known last period, and even then the bad equilibrium remains.

Concept atlas