Module III of VI

Priors & base rates

A cab sideswipes a pedestrian at night and drives off. In this city, 85 percent of cabs are green and 15 percent are blue. A witness says the cab was blue, and careful testing shows she identifies colors correctly 80 percent of the time under those conditions. How likely is it the cab was really blue?

The famous answer — famous because Daniel Kahneman and Amos Tversky used this puzzle to show how reliably we get it wrong — is not 80 percent. Count 100 late-night cabs. Fifteen are blue; the witness correctly calls about 12 of them blue. Eighty-five are green; she miscalls about 17 of those blue. So “blue” gets said about 29 cabs, and only 12 of them are blue: roughly 41 percent. The witness is good. The base rate is against her.

Almost everyone answers “80 percent,” letting the witness’s reliability swallow the base rate whole. The error even has a name — base-rate neglect — and once you can see it, it turns up everywhere: in hiring (“she interviewed brilliantly”), in medicine (Module II), in fraud detection, in your own snap judgments about strangers.

Panel III·1 — Same evidence, different city

prior slider · fixed evidence
before (witness says “blue” not yet seen)
15 %
after “witness says “blue””
41 %
Of 1,000 cabs: 150 where the cab was blue — witness says “blue” in 120 of them, and falsely in 170 others → 120 / 290 = 41 %
Try this Slide the base rate down with the evidence held fixed, and find the point where the witness's 'blue' is only a coin flip. Then switch scenarios — the costume changes, the arithmetic doesn't.

The base rate is your prior: what you believed before this particular piece of evidence showed up. And here a philosophical objection usually arrives on schedule — isn’t starting from a prior just importing bias? It’s a fair worry pointed in the wrong direction. The prior for “blue cab” isn’t a prejudice about this driver; it’s the compressed record of every cab in the city. Refusing to use it doesn’t make you neutral. It silently replaces an informed starting point with an uninformed one, which is also a prior — just a worse one, chosen by not choosing.

Where do good priors come from? From asking how often is this kind of thing true in this kind of situation — the reference class — before looking at the shiny specific detail in front of you. What fraction of startups at this stage succeed? How often do projects with this scope finish on time? The outside view first, then adjust for the particulars. People who forecast well do this almost compulsively.

Priors also explain a slogan you already know. “Extraordinary claims require extraordinary evidence” is Bayesian arithmetic in a trench coat: a claim with a one-in-a-million prior needs evidence strong enough to overcome odds of a million to one, and an anecdote won’t lift that weight.

Retell itBefore admiring the detail in front of you, ask how often this kind of thing is true at all — the base rate is half of every answer, and the half everyone skips.

How to measure the strength of evidence — what “strong enough” means, exactly — is Module IV’s whole business, and it opens with a corpse in a library.