Station5 / 7
Reading10 min
SourcesBirgitta Böckeler 2026-04-02
Geoffrey Huntley 2025-07
Dex Horthy 2025
Revised2026-07-24
Harness engineering
The discipline that appeared when people stopped iterating on the code and started iterating on the environment around the model.
The formulation that made this click for a lot of people is deliberately plain:
Agent = Model + Harness.
You do not control the first term. You control the second entirely, and by 2026 the evidence had accumulated that the second term explains more of the variance in outcomes than the first. Two teams on the same frontier model get wildly different results. The difference is not prompt wording — it is context, tools, checks and rollback.
Birgitta Böckeler gave the practice its name and, more usefully, its taxonomy. Primary
Guides and sensors
Controls come in two kinds, distinguished by when they act.
Guides are feedforward: they shape the agent before it does anything. A context file it must read, a skill that encodes a procedure, a template that makes the right shape the path of least resistance, a type signature that refuses the wrong call at the point of writing.
Sensors are feedback: they observe after the agent has acted, and let it correct itself. A linter, a test suite, a review agent.
Cross that with a second distinction — whether a control is computational (deterministic, fast, cheap) or inferential (a judgment call, slow, non-deterministic) — and you get the grid that decides how your factory behaves.
Context files the agent must read. Codemods and scaffolds that make the right shape the path of least resistance. Types and signatures that refuse the wrong call.
AGENTS.md · CLAUDE.md · project templates · codegen
Procedures written as prose, and reference documents the agent consults when it hits a decision the codebase does not settle.
SKILL.md procedures · Spec Kit's constitution.md · ADRs · craft-knowledge docs · worked examples
The checks that may gate, because they cannot be argued with: compilers, type checkers, linters, unit and property tests, structural fitness rules.
tsc · clippy · ArchUnit · mutation tests · link checks
The checks that may only advise, because they are judgment: a review agent with a rubric, a semantic critic, a screenshot judge.
LLM-as-judge · critic models · architecture review
The operative rule is in the columns, not the rows. Computational checks may gate. Inferential checks may only advise. A type error is not a matter of opinion, so it can stop the line without anyone feeling hard done by. An LLM judge is a probabilistic opinion; wire it as a hard gate and you will spend your afternoons arguing with a machine that cannot be appealed to, and eventually you will start overriding it, and then it is no longer a gate at all.
Spotify’s fleet illustrates the corollary neatly. Their judge was removed as the models improved, having been busy enough to matter:
~25% of sessions vetoed by the LLM judge before that judge was removed as models improved
Sensors are not sacred. They are instruments, and instruments get retired.
Owning your control flow
The other half of harness engineering is unglamorous and older than any of this: decide the paths in advance.
An agent walking a predefined directed graph — explicit nodes, explicit conditional edges — is back pressure drawn as a diagram. You trade some of the agent’s freedom for mandatory checks and legible failure points, and the return on that trade is that when a run dies you can point at the node that killed it. Most systems that work in production turn out to be mostly deterministic code with model calls at a few well-chosen points. The clever part stays inside each box; the paths between the boxes are sanctioned ahead of time. Primary
Length matters here for a reason that is empirical rather than aesthetic:
3–10 steps where agents hold up; past twenty the thread is gone
A short loop is verifiable because you can see all of it; a sprawling one hides its mistakes in the corners, and you find them by accident weeks later.
The Ralph loop, and why the dumbest version works
The counter-example everyone cites is Geoffrey Huntley’s Ralph loop Primary: one coding agent in an infinite shell loop, reading the same prompt file every iteration, using the filesystem and git history as its memory, starting each cycle with a completely fresh context window.
It is named after a cartoon character who is cheerfully, reliably dim. The dimness is the feature. A fresh context each iteration sidesteps context rot — the slow degradation as a window fills with an agent’s own history. Git history is better memory than conversation history because it is durable, inspectable and diffable by a human. The loop does not need to be clever if the environment remembers for it.
That is the thesis of harness engineering in one pattern: the intelligence you are engineering is not in the agent.