Back pressure

Generation was never the bottleneck. What it costs to check the work is, and that price sets every other limit.

Here is the rule, and it is the most portable idea in this manual:

You can hand a loop as much autonomy as you can cheaply and reliably verify, and not one inch more.

Generation is a wide mouth. Verification is a narrow neck. Speeding up the mouth does not widen the neck; it deepens the pile in front of it.

cheap and parallel expensive and serial — the constraint

Fig. 4.1Widening the mouth does not widen the neck. It deepens the pile in front of it.

Volume on its own is not the problem — plenty of teams merge a great deal of machine-written code without drama. A surplus of bad pull requests is the problem, and the difference between the two situations is entirely a property of the neck.

The price of the verifier

The economics are lopsided enough to plan against.

≈ $0.05 machine review, per pull request as of 2026 Vendor

$15–25 human review, per pull request as of 2026 Vendor

Three orders of magnitude. That gap is why every serious factory eventually tries to move work from the second row to the first, and why the interesting engineering in this field is increasingly about verification rather than generation. Werner Vogels named the accumulating cost of unchecked machine output verification debt Primary; it is comprehension debt seen from the accounting side.

But the cheap row is not free of cost, only of price:

20–40% of machine review comments that are noise as of 2026 Vendor

A verifier that cries wolf up to two times in five trains the humans downstream to stop reading it, which converts a cheap check into an expensive habit.

The evidence is unsettled

This is where a manual has to be careful, because the honest answer is unsatisfying and the dishonest answers are everywhere.

The most rigorous study of the period found the opposite of what everyone expected:

19% slower measured effect on 16 experienced developers across 246 tasks — WALKED BACK by METR in February 2026 as of 2025-07-10 Contested

Read that with its retraction attached, because METR itself withdrew the result in February 2026 as methodologically unreliable for the current period, and its stated position now is that we do not know. Contested The number is not evidence that AI slows developers down. It is evidence that nobody has cleanly measured the effect, and that includes the people best equipped to try.

The finding from that study that has not been withdrawn is the one worth carrying:

~39 points gap between developers’ perceived speed-up and the measured effect as of 2025-07-10 Contested

Developers in the trial forecast a speed-up, experienced a slow-down, and afterwards still believed they had been faster. Whatever the true effect is, your own sense of it is not a reliable instrument, and neither is your team’s.

The survey data points the same way from a different angle:

38% say reviewing AI code takes more effort than reviewing human code as of 2026-01-08 Reported

Nearly everyone reports not fully trusting machine-written code; fewer than half always verify it before committing. Reported The gap between those two numbers is where verification debt is created, one small skipped reading at a time.

And the benchmarks will not settle it for you

~73% → ~11% frontier agent resolution once the verified set is filtered to discriminative cases as of 2026 Reported

The published scores measure something real, but not the thing procurement decisions assume they measure. OpenAI put out its own note in February 2026 saying its verified benchmark no longer tracks frontier coding capability Primary, and benchmark-passing patches frequently would not be merged by the maintainers of the projects they patch.

From the floor

Back pressure is the reason the site you are reading has a three-round cap on its judges.

The arrangement is split in three. The floor is deterministic and binary — the build compiles, the internal links resolve, every page screenshots without a console error. It is never allowed to become a proxy for taste, which is why there are no pixel-diff tests in it: screenshot comparisons are flaky, and a flaky gate poisons trust in every other gate beside it. The quality layer is two independent judges over the rendered pages and the prose. The feel layer is a person.

The cap exists because judge rounds are the expensive row of the table. Three rounds says nothing about how many rounds produce a good site; it says how much verification this work is worth before a human’s attention becomes the cheaper instrument.

Evidence: website-factory/knowledge/quality-gates-and-judges.md

What to do with all this

Budget for the verifier explicitly, as a line item, the way you would budget for CI. Track cost per useful review rather than reviews performed. And treat the threshold seriously: if verification cost per change approaches what you saved on generation, the factory is running at a loss and the fix is to narrow the work until it is testable again, not to verify less.