Game Theory and Cooperation

The previous module treated a single mind deciding under its own limits. But few decisions are made in isolation. The moment two bounded agents must each anticipate the other, a new layer of structure appears — one with its own logic, its own traps, and its own surprising escapes. Game theory is the formal study of that layer, and in the last decade its oldest puzzles have acquired a new kind of player: the machine.

Strategic Interaction: Game Theory Meets AI

Two burglars are caught near the scene and interrogated in separate cells; neither can hear what the other says. Each has the same choice: stay silent — cooperate with your partner — or betray them: defect. Scored in the points the lab below uses (higher is better), the four possible outcomes look like this:

Read one cell to learn the table: if you defect while your partner cooperates, that is the lower-left cell — 5 points for you, 0 for them. Now check your own best move. If your partner cooperates, defecting pays 5 against cooperating’s 3; if your partner defects, defecting pays 1 against 0. Whatever the other does, betrayal scores higher — so two rational players end at 1 point each, while 3 each sat on the table. Neither can improve by changing strategy alone: mutual defection is the game’s Nash equilibrium. Can you trust your partner? That is why it is called the Prisoner’s Dilemma. And the trap generalizes: whenever your best action depends on what someone else does, you are in strategic interdependence, and game theory is its formal study. (Game theorists name the four payoffs Temptation 5, Reward 3, Punishment 1, and Sucker’s payoff 0; any game with T > R > P > S is a Prisoner’s Dilemma.)

The dilemma looks inescapable — but real people have memory, and they meet again. So which strategy is best for a game repeated indefinitely? Robert Axelrod ran that question as a tournament in the 1980s, and the winning strategy, Tit-for-Tat, was simple: cooperate first, then mirror whatever the opponent did last. Nice, retaliatory, forgiving, and clear.

The AI era brought dramatic developments. AlphaGo (2016) defeated the world Go champion using moves that human experts found alien but brilliant. AlphaZero (2017) mastered chess, Go, and shogi from scratch through pure self-play — learning entirely by playing millions of games against copies of itself, with no human examples — discovering strategies that centuries of human play had missed. Pluribus (2019) beat professional poker players — the first AI to handle imperfect-information games, where players cannot see each other’s full state, with multiple players. (For the full AI story, see The AI Revolution.)

But when large language models — AI systems like ChatGPT that generate text by predicting the most likely next words, examined in depth in The AI Revolution — entered game-theoretic settings, the results were more nuanced. Decades of lab experiments had established the human baseline: most people are conditional cooperators and altruistic punishers — they cooperate when others do, and pay real costs to punish cheaters. Research showed GPT-4 cooperates in the Prisoner’s Dilemma about 79% of the time — higher than typical human rates. However, it plays what researchers call an “unforgiving” strategy: it cooperates until the first defection, then permanently retaliates. No forgiveness, no recovery. This makes it “too rational for its own good” — optimizing for not being exploited at the cost of losing all future cooperative gains.

This behavior is prompt-dependent — it shifts with how the question is posed to the model (the prompt). Social Chain-of-Thought prompting — asking the AI to reason about the other player’s perspective before deciding — significantly increases cooperation and forgiveness. The architecture is the same; only the framing changes. This echoes Gigerenzer’s ecological rationality: the “environment” (prompt) determines whether the same system cooperates or defects.

Before you run the tournament below, commit to two guesses: over hundreds of rounds, does Always Defect — which never loses a single exchange — finish first or last? And does GPT-4’s unforgiving strategy beat Tit-for-Tat’s forgiving one?

Does Structure Rescue Cooperation?

If you guessed Always Defect finishes last, the standings just corrected you: in this small pool it lands at or near the top, and the unforgiving strategy outscores the forgiving one — because the lab’s pool contains an unconditional cooperator and a random player to farm. That is Axelrod’s real finding, sharper than the slogan: whether cooperation wins is a property of the population, not of the strategy — Tit-for-Tat won the 1980s tournaments because the pool was thick with retaliators, and even there it never beat a single opponent head-to-head. Cooperation needs structure. Reputation systems, repeated interaction, and institutional design create the conditions for cooperative equilibria. (Temptation is not even required: in the Stag Hunt — hunt a stag together and feed ten, or hares alone and feed two — cooperation fails from sheer coordination risk, the fear that the other will not show up.) And structure is literal: who plays whom matters as much as the payoffs. But which way? Do the long-range shortcuts that make networks efficient — Module 2’s small worlds — help cooperation spread, or help defection invade? Commit to a guess, then try all three structures below.

Cooperation, then, is an achievement of structure, not a default of nature: clusters let cooperators meet one another often enough to out-score the defectors on their boundary. The next question follows directly: what happens when the structures that mediate human interaction — the platforms, feeds, and recommendation systems that now sit between us — are redesigned around a different objective entirely? That is the subject of the next module: the contest for attention, and what it does to collective intelligence.