Part IV · Chapter 14

Conclusions

Since the defects described in the first thirteen chapters cannot be removed from the analyst, Part IV puts the remedy in the procedure and the organization that sets the conditions for it.

The checklist for analysts

Heuer lists the usual proposals — collect more, manage differently, hire more analysts, sharpen the writing — and grants each may matter. His objection is that analysis is above all a mental process, and analysts at every level devote little attention to how they think (p. 173). His answer is a checklist built around six steps:

1. Defining the problem. Check you are being asked the right question, and settle depth against deadline at the outset.
2. Generating hypotheses. Brainstorm broadly, then cut to a workable number. Never screen one out merely because no evidence supports it — that reasoning disposes of the deception hypothesis automatically.
3. Collecting information. What arrives at your desk will not be enough. Collect against all the reasonable hypotheses.
4. Evaluating hypotheses. Build the arguments against each one, and test how far the conclusion moves when your assumptions change.
5. Selecting the most likely hypothesis. Try to reject rather than confirm, and report the alternatives you considered and why you set them aside.
6. Ongoing monitoring. Specify in advance what would shift the probabilities, and treat surprise as data.

Paraphrased from Heuer’s checklist, Ch. 14 · p. 173–177 — the wording is this site’s, the substance his.

Two warnings

Step 5 restates the book’s governing principle — the likeliest hypothesis is the one with the least evidence against it, not the most for it (p. 176), which the ACH page quotes in full. Two warnings sit beside it, both on p. 176. Do not fill gaps by assuming another government will act as the US Government would — Heuer’s example is the 1989–90 judgment that Iraq would demobilize after the Iran-Iraq war and would see no interest in attacking a neighboring Arab country. And do not assume foreign actions follow from rational decisions in pursuit of identified goals; some are better explained by bargaining between bureaucracies, procedures applied in the wrong circumstances, confusion or accident.

What management can change

The closing section turns to the organization. Heuer groups the measures into four (p. 177): research into the cognitive processes behind intelligence judgment; training aimed at reasoning rather than at procedures and writing style; deliberate exposure to alternative mind-sets; and guiding analytical products so competing views survive into what is published.

The training point has the most teeth. Analysts who write clearly are assumed to be thinking clearly, yet one can follow a faulty process and still argue persuasively for the wrong judgment (p. 178). Clarity of product is no evidence about the reasoning behind it.