What to do on Monday
The staged playbook from chapter 8, expanded to working detail: three stages as gates, six steps as the checklist inside them. Every stage carries two lines that matter more than its contents: a condition for moving on, and a condition for stopping. A playbook without a stop condition is a sales pitch.
The sequencing is deliberate. The second stage is where the value and the risk both live, and the third is conditional on evidence you will not have for months, the evidence stage 2 exists to collect.
Stage 1 — Build a lit factory
0–3 monthsThis stage is about making agent output reviewable at volume, not about reducing review. Keep the human merge gate. All of it.
-
01
SPEC
Write the contract
Define the product surface, the acceptance criteria, the permitted side effects, and what must remain a human decision. Standardise on a spec-driven workflow so the unit of work is a reviewable artifact rather than a conversationprimary. Give agents a disciplined context layer as well: one file per repository stating how to build, test and check it, worker instructions that read that file instead of hard-coding toolsprimary.
-
02
SENSORS
Make the floor honest
Types, tests, links, paths, policy and smoke checks should prove only what they can truly decide. A floor that quietly stands in for taste is the first dishonest gate in the building. Isolate parallel work too, so runs cannot collide.
- Ready to move on when agent-authored changes routinely pass CI on the first or second attempt, and review time per change has dropped below your human-authored baseline.
- Stop if you are shipping more and nobody can describe what changed last week. That is comprehension debt arriving early, and the cure is smaller, better-scoped work rather than more tooling.
Stage 2 — Spend disproportionately on the verifier
2–6 months · the important oneThis is where the leverage is, and almost nobody budgets for it, because verification does not demo well. Layer computational sensors that run on every change; reserve inferential sensors (an LLM judge, an architecture review) for after integration, and keep quality left: a guide that prevents a class of mistake is worth more than a sensor that reports itprimary.
-
03
INDEPENDENCE
Separate maker and checker
Fresh context, regenerated evidence, explicit rubrics, and a judge at least as capable as the maker. A judge that grades the maker's own artifacts, or shares its failure distribution, is consistency dressed as verification. The case studies caught exactly this, in productionworkspace.
-
04
GRAPH
Own the control flow
Sanctioned nodes, retry caps, parked runs, and named terminal states. “Ask the user” is a valid lit gate; it is not an orchestration strategy, and it is not a dark policy at all.
-
05
HANDOFF
Pass evidence across the gate
A verdict, a runnable artifact, regenerated checks, unresolved findings, and a short review path — not “done”. A handoff that says all checks passed is a claim; a handoff that says what ran, what it found and what nothing checked is evidence.
Then measure the things nobody measures:
- Cost per useful review, not reviews performed. Agent review runs at cents per PR against tens of dollars for a human passreputable. But 20–40% of AI review comments are false positivesreputable, and a verifier that is mostly right is still expensive if the noise trains people to stop reading it.
- Marginal catch rate per gate — what each human gate still finds after the machines have signed off. This is the only number that can retire a gate later, and you have to start collecting it now, while the gates are still thereworkspace.
- Defect escape rate, because throughput without it is a vanity metric.
- Ready to move on when you have at least one loop whose oracle is cheap, immediate and impossible for the generator to game, plus months of data on what your humans still catch downstream of it.
- Stop if verification cost per change approaches what you saved on generation. The factory is then running at a loss; the fix is to narrow the work until it is testable, not to check less of it.
Stage 3 — Consider dark, narrowly
6+ months · conditionalNote the scope: consider it for a loop, never for an organisation. The published route to running this way at scale requires holdout scenarios the agents cannot read, behavioural clones of your dependencies so scenarios can run in bulk, and a tolerance for probabilistic rather than boolean successprimary, at a stated benchmark of at least $1,000 per engineer per day in tokensprimary.
Start where Stripe and Spotify started: flaky-test repair, mechanical migrations, dependency bumps, well-specified changes behind rich testsprimaryreputable. Low stakes, high coverage, small blast radius.
-
06
RETRO
Instrument the dimmer
Track what each gate catches, what escapes, review time, convergence rounds, and verifier cost. Darken only from that evidence, one gate at a time, on a named class of work, and run each candidate loop through the dimmer before you argue about it. Be strict about the third switch: whether the generator could game the oracle.
- Ready when the marginal catch rate for this class of work has been zero for enough consecutive runs that the zero means somethingworkspace.
- Stop if you cannot say who finds out when this loop is subtly wrong. That is not a tooling gap; it is the absence of an oracle.
Three things to hold across all of it
- Do not trust perceived speed — yours or your team’s. The one finding from the METR trial that survived its own caveats is the perception gap: developers misjudged their own AI-assisted productivity by roughly thirty-nine pointscontested. Measure; do not ask.
- Automate the disposable, guard the durable. Output that can be regenerated is a good candidate for darkness. The templates, rubrics and shared libraries that make it are where comprehension debt actually accumulates; the case studies keep edits to the factory itself human-gated for exactly this reasonworkspace.
- Prefer harness investment to framework shopping. The frameworks are commoditisingreputable. Your context, your checks and your rollback path are not.
Move through the stages on their gate conditions, not on the calendar ranges. The months are what the field reports, not what your evidence will say. If you cannot state a stage's stop condition from memory, you are not running the stage; you are running the sales pitch.