Comprehension debt

The bill a dark factory runs up. It accrues silently, with the tests green, and it comes due late.

Comprehension debt is the widening gap between how much code exists and how much any human still understands.

A dark factory does not pay that debt down. It takes it on as fast as it possibly can, and the whole time the tests are green — which is precisely what makes it hard to notice. Nothing is failing. The dashboards are fine. The only thing happening is that the share of your own system anybody could explain from memory is falling every week.

The clearest published account of where that ends comes from Dex Horthy, who ran a fully automated code factory for about four months with no human reading the output. Primary It worked, until it did not. The failure that ended the experiment required painstaking manual debugging to locate — not to fix, to find. That is the shape of the reckoning: not a dramatic outage, but a day when something is wrong and there is no one who can say where.

What gets optimised instead

Underneath is a substitution that is easy to make and hard to see. Token utilisation is measurable, immediate and satisfying. It goes up. Comprehension is none of those things, and it goes down. Maximising the number that is easy to watch minimises the one that is not, and nothing in the dashboard registers the trade.

Thoughtworks’ Technology Radar has been naming the same creature from the organisational side Primary — cognitive debt, semantic diffusion, complacency with AI-generated code — and pushing, of all unfashionable things, a return to engineering fundamentals.

Why the timing fools people

Comprehension debt has a delay built into it. That delay is what makes the argument for going dark so persuasive in month one.

A weekend project never lives long enough to collect the debt. You will not maintain it, nobody will inherit it, and if it breaks you will throw it away. Going dark there costs you nothing, and the experience is genuinely wonderful, and it teaches you a lesson that does not transfer.

A decade-old system in production is made of comprehension debt already, most of it inherited from people who left. It is the environment where every additional unread change is expensive, and also the one where the throughput gain looks most tempting, because the backlog is enormous.

So the two situations where the decision feels most obvious point in opposite directions, and neither of them is telling you about the third case — the two-year-old system your team is still actively reasoning about — which is where most of the actual code is.

From the floor

The most useful thing four factory runs taught me about this debt is where it lives, and it was not where I expected.

The sites these factories generate are disposable. If one turns out to be wrong, the fix is to rewrite the brief and run it again — nobody needs to understand the generated Liquid. So comprehension debt on the output side is close to harmless, and that is a real argument for letting more of the output side go dark.

The factory itself is the opposite. Its knowledge documents, its templates, its judge rubrics — that is a small brownfield codebase, edited by hand, that has to stay understood for the next run to work. Which is why edits to the factory stay human-gated even as the work it does gets more automated. The rule that fell out of it: automate the disposable, guard the durable. They are usually not the same files, so be explicit about which is which before you start switching lights off.

Evidence: website-factory/skills/factory-retro/SKILL.md · specs/retro/2026-07-23.md

The one question to ask

Before any loop goes unattended, ask: if this produces something subtly wrong, who finds out, and how long does it take them?

If the answer is “a test, within a minute”, you are looking at a candidate. If the answer is “a person, eventually, once something else breaks”, you have just described the debt, and you are choosing to take it on.