PageRegistry
entries16
Noterendered from _data/landscape.yml
Revised2026-07-24
Every row is a registry entry with an as-of date and a source. Nothing on this page is typed into prose, which is why a September revision costs one edit rather than a rewrite.
| Operation | Autonomy | What it is | As of | Evidence |
|---|---|---|---|---|
| StrongDM Software Factoryfactory | L5 | The reference dark factory. Two rules: code must not be written by humans, and must not be reviewed by humans. | 2026-02 | Primary |
| Stripe “Minions”factory | L4 | Unattended execution, attended approval. From Slack message to merged PR with no human keyboard — and a human reads every one. | 2026-02-20 | Primary |
| Spotify “Honk” / Fleet Managementfactory | L4 | A migration factory at scale. Deterministic scripts still do the bulk; the agent takes the edge cases. | 2026-03 | Reported |
| How long an agent holds the threadevidence | — | The empirical reason short loops are verifiable and sprawling ones are not. | 2026-06 | Primary |
| Cursor cloud agentsfactory | L4 | Agents produce merge-ready PRs with recorded evidence; review stays the human bottleneck. | 2026-04 | Vendor |
| Factory.ai Droidsagent | L4 | Model- and interface-agnostic autonomous agents. The performance claims are the company's own. | 2026-04-16 | Vendor |
| This site's own factoryfactory | L4 | A four-plugin suite run by one person. The evidence behind every “From the floor” block on this site. | 2026-07-23 | First-hand |
| METR randomised controlled trialevidence | — | The most rigorous productivity study of the period, and the reason nobody should quote productivity numbers with confidence. | 2025-07-10 | Contested |
| The cost of a reviewevidence | — | The asymmetry that decides whether a factory pays for itself. | 2026 | Vendor |
| Developer trust in AI-generated codeevidence | — | Adoption and trust have come apart, and the review burden is where it shows. | 2026-01-08 | Reported |
| Security findings in AI-generated codeevidence | — | Volume arrives at the security gate too. | 2025-09 | Reported |
| Benchmark inflationbenchmark | — | A high SWE-bench score is not a mergeable pull request. | 2026 | Primary |
| Harness engineeringharness | — | Agent = Model + Harness. Guides steer before the agent acts; sensors observe after it has. | 2026 | Primary |
| The Ralph loopharness | — | One agent, an infinite shell loop, one task per iteration, the filesystem as memory, a fresh context window each cycle. | 2026 | Primary |
| Spec-driven developmentframework | — | Specs as the executable source of truth, in place of prompt-by-prompt improvisation. | 2026 | Primary |
| Agents per workflowevidence | — | The same shape appears outside code: a few specialised agents, human approval at the write boundary. | 2026 | Reported |
The reference dark factory. Two rules: code must not be written by humans, and must not be reviewed by humans.
≥ $1,000/day/engineer stated token-spend benchmark for running this wayMade viable by machinery, not by trust in the model: scenario tests held outside the codebase like a machine-learning holdout set, behavioural clones of third-party services so thousands of scenarios can run per hour, and "satisfaction" — a probabilistic success metric — in place of a boolean "the tests passed".
Unattended execution, attended approval. From Slack message to merged PR with no human keyboard — and a human reads every one.
1,300+ PRs merged per week, zero human-written code, all human-reviewed 2 rounds CI limit before a stuck task bails out to a humanBuilt on a decade of prior investment in standardised devboxes, a three-million-test suite and internal tooling — the transferability caveat for anyone smaller.
A migration factory at scale. Deterministic scripts still do the bulk; the agent takes the edge cases.
~1,000 PRs merged every 10 days ~70 / 30 split between deterministic migration scripts and agent work ~25% of sessions vetoed by the LLM judge before that judge was removed as models improvedNotable in the other direction: a verifier retired rather than added.
The empirical reason short loops are verifiable and sprawling ones are not.
3–10 steps where agents hold up; past twenty the thread is goneAgents produce merge-ready PRs with recorded evidence; review stays the human bottleneck.
35% of internally merged PRs created by cloud agentsModel- and interface-agnostic autonomous agents. The performance claims are the company's own.
“31× faster feature delivery” company-reported, no independent measurementA four-plugin suite run by one person. The evidence behind every “From the floor” block on this site.
10 sites published through the full gated arc, 2026-07-11 to 07-23 2 of 3 judge rounds used on the most recent run before both judges passed 1 human feedback round that still caught something after both judges had passedOne operator, disposable output, no production traffic. It is evidence about gates and verification, not about engineering at scale.
The most rigorous productivity study of the period, and the reason nobody should quote productivity numbers with confidence.
19% slower measured effect on 16 experienced developers across 246 tasks — WALKED BACK by METR in February 2026 ~39 points gap between developers' perceived speed-up and the measured effectMETR's position since February 2026 is that selection effects made the follow-up unreliable and the honest answer is that we do not know. Both halves travel together, or neither does.
The asymmetry that decides whether a factory pays for itself.
≈ $0.05 machine review, per pull request $15–25 human review, per pull request 20–40% of machine review comments that are noiseAdoption and trust have come apart, and the review burden is where it shows.
96% of developers do not fully trust AI-generated code 48% always verify it before committing 38% say reviewing AI code takes more effort than reviewing human codeVolume arrives at the security gate too.
> 10,000/month new security findings attributed to AI-generated code by June 2025, a tenfold rise from December 2024A high SWE-bench score is not a mergeable pull request.
~73% → ~11% frontier agent resolution once the verified set is filtered to discriminative cases ~18–20% resolution on the live benchmark, which uses novel issuesOpenAI published its own note in February 2026 saying SWE-bench Verified no longer measures frontier coding capability.
Agent = Model + Harness. Guides steer before the agent acts; sensors observe after it has.
Separates computational regulation — linters, type checkers, structural tests: fast, cheap, deterministic — from inferential regulation: an LLM judge, a semantic review, slow and non-deterministic. The human's work moves to iterating the harness rather than the code.
One agent, an infinite shell loop, one task per iteration, the filesystem as memory, a fresh context window each cycle.
The dumb persistence is the feature: a fresh context each iteration sidesteps context rot, and git history carries the state that conversation history would lose.
Specs as the executable source of truth, in place of prompt-by-prompt improvisation.
The reference toolkit carries an immutable constitution file, a specify/plan split and slash-command templates. Heavier multi-agent frameworks simulate an entire delivery team and cost accordingly.
The same shape appears outside code: a few specialised agents, human approval at the write boundary.
3–7 agents per enterprise workflow, with human review at side-effect boundariesSomething out of date? That is expected — it is why this page
exists as data. The fix is one entry in _data/landscape.yml and one line in
the update log — no chapter is touched.