Comprehension debt
The bill a dark factory runs up. It does not show in any dashboard, the tests stay green the whole time, and the reckoning is quiet and late.
Comprehension debt is the widening gap between how much code exists and how much any human still understandsprimary.
It is worth separating from its more famous sibling. Technical debt is a known shortcut: someone took it, someone can describe it, and the cost of repaying it can be estimated. Comprehension debt has no author who remembers. It is not a shortcut but an absence — nobody built the mental model, because nobody read the thing.
A dark factory does not pay this down. It takes it on as fast as it can, with the tests green the whole wayprimary.
Why it stays invisible
Every signal an engineering organisation watches keeps reading normal while the debt accrues. CI is green — the tests were generated alongside the code and agree with it. Velocity is up — that is the point. Incident rate is flat, until it is not.
Underneath sits a substitution that happens without anyone deciding it: maximising token utilisation, the number treated as progress, quietly minimises the share of the system any human still understandsprimary. Two quantities, one visible on a dashboard and one not, moving in opposite directions.
The reckoning arrives quiet and lateprimary. Horthy’s four-month dark factory did not end with a spectacular outage; it ended with a failure that required painstaking manual debugging to even locateprimary. That is what the debt buys you. The bug count may not even rise; what rises is the cost of locating any single bug, which becomes a research project.
The evidence problem
The code is only the first thing nobody understands. The second is whether any of this works.
The most rigorous study of the question is METR’s 2025 randomised controlled trial: 16 experienced developers, 246 tasks on repositories they knew well. It found that allowing AI increased completion time by 19%, while the same developers forecast a 24% speedup beforehand and estimated a 20% speedup afterwards — a perception gap of roughly 39 pointscontested.
That result was cited relentlessly through late 2025 as proof the field was fooling itself, and it deserves better handling than that, because of what happened next: in February 2026 METR itself said selection effects had made the follow-up unreliable, and that its honest position is now that it does not knowcontested.
So the finding is not “AI makes developers 19% slower.” The defensible summary is narrower and more uncomfortable: we do not have rigorous evidence that AI makes experienced developers faster on mature codebases, and the one serious attempt to measure it was withdrawn by its own authors. Anyone quoting the 19% as current fact is doing the thing this site exists to make harder.
What survives the retraction is the perception gap, and it is corroborated independently by the Sonar survey’s mismatch between what developers say they trust and what they shipreputable. Practitioners are poor instruments for measuring their own productivity. That much has held up.
Benchmarks measure the wrong thing well
The same problem in a different suit. SWE-bench — 2,294 real GitHub issues across 12 Python repositories — became the field’s standard measure, with SWE-bench Verified as OpenAI’s human-filtered subsetprimary. By mid-2026 the leaderboard sat around 80 to 94%reputable.
Three findings puncture that number from three directions.
IBM Research found that frontier agent capability drops from about 73% to about 11% on discriminative subsets — the tasks where a benchmark can tell real capability apart from memorisation. Resolution on genuinely novel issues runs around 18 to 20%primary. One reported 93.9% result had roughly 20% of its “solved” cases judged semantically wrong on inspectionreputable. And OpenAI published a note in February 2026 explaining why SWE-bench Verified no longer measures frontier coding capabilitiesprimary — a benchmark’s owner declaring it saturated.
The practitioner takeaway is compact: a high benchmark score is not a mergeable pull request. A wave of successors — Terminal-Bench, SWE-bench-Live, and othersprimary — exists because the field noticed.
What you can actually measure
Given all of the above, the temptation is to measure nothing and go on vibes. The alternative is to measure the things that resist the failure modes above.
- What the gate caught. Per gate, per period, what did this check actually stop? A gate with an empty column may be redundant; a gate you never asked is a mystery.
- What was caught after the gate passed. The marginal catch rate of every human step. This is the only real input to the question “can this gate go dark”, and almost nobody records it.
- Cost per useful review, not per reviewprimary.
- Time to locate a fault, which is the direct proxy for comprehension debt and the number Horthy’s four months would have movedprimary.
Thoughtworks’ Technology Radar, which has been unusually blunt about this, calls the same family of risks cognitive debt and pushes a return to engineering fundamentalsprimary.
Start recording marginal catch rate per gate now, before you need it. It costs a line per review and it is the only evidence that can ever justify turning a gate off — and it cannot be reconstructed after the fact, because the thing you would be measuring is what somebody noticed and did not write down.