chapter 05 · the snapshot

05

The field, as of July 2026

as of 2026-07 · volatile — verify before citing

Everything in chapters 1–4 should age slowly. This chapter will not, which is why it carries a stamp and why the updates log exists. Every claim here carries its class: measured primary or peer-reviewed source · vendor-reported self-reported and unverified, rendered deliberately colorless · our experience observed first-hand in this workshop’s factories.

§ 5.1The numbers that frame the field

• measured 1,300+ PRs merged per week at Stripe carrying zero human-written code — every one human-reviewed. “Minions” is unattended execution, attended approval. stripe engineering · 2026-02-20
• measured ~1,000 PRs merged every 10 days by Spotify’s Honk, every merge still human-reviewed. Its LLM judge, initially vetoing a quarter of sessions, was removed in March 2026 as models improved. qcon london via infoq · 2026-03
• measured 35% of Cursor’s internally merged PRs are created by cloud agents in sandboxes. Review remains the human bottleneck, by their own account. cursor (truell) · 2026-04
• vendor-reported $20k per engineer per month in tokens, StrongDM’s self-stated spend benchmark for the reference dark factory. Willison flagged it as the make-or-break question. strongdm, self-reported · 2026-02
• vendor-reported 31× faster feature delivery, claimed by Factory.ai for its Droids. No independent measurement exists; treat as directional. factory.ai marketing · 2026
• measured 19% slower with AI: the METR randomized trial’s famous result (16 experienced devs, 246 tasks), which METR itself later called unreliable for the current period. The honest state of the productivity evidence: we don’t know. arxiv 2507.09089 · 2025-07 · walked back 2026-02

§ 5.2The stack that crystallized

The tooling settled into recognizable strata during 2025–2026. Spec-driven development frameworks sit on top: GitHub’s Spec Kit (80k+ stars, constitution-driven), BMAD (a simulated agile team of specialized agents, at a real token cost of $800–2,000 per developer per month on frontier models), OpenSpec for brownfield, and AWS’s Kiro IDE. Below them, the coding-agent harnesses: Claude Code (skills, subagents, worktrees; also the harness this site’s own factory runs on), OpenAI Codex, Google Antigravity, Devin, Factory.ai’s Droids, and open-source OpenHands. The money followed: Cognition reached a $26B valuation in May 2026 with ARR growing from $37M to $492M in twelve months measured (Series D announcements, 2026-05-27); Factory.ai raised at $1.5B in April. Multi-agent orchestration frameworks (LangGraph, CrewAI, AutoGen) commoditized fastest. The 2026 consensus across practitioner comparisons: the framework is rarely the differentiator; the harness is.

§ 5.3The reference dark factory

StrongDM’s Software Factory, publicly described in February 2026, is the one well-documented level-5 operation. Its rules are absolute: code must not be written by humans, and must not be reviewed by humans. What makes it work is not trust in the model but three verification inventions. Scenario testing as holdout sets: end-to-end user stories stored outside the codebase, like an ML holdout set, so agents cannot overfit to them. Satisfaction: a probabilistic success metric (what fraction of observed trajectories likely satisfy the user?) replacing boolean “tests pass.” And the Digital Twin Universe: behavioral clones of Okta, Jira, Slack, and Google Docs, built by agents from public API docs, enabling thousands of scenario runs per hour without rate limits. Every one of these is a verification investment. The reference dark factory going dark bought more verification, not less; that is the correct one-line summary of the state of the art.

§ 5.4What the numbers don’t say

Three caveats belong in any honest snapshot. Benchmark scores overstate capability: mid-2026 SWE-bench Verified leaders sit at 80–94%, but IBM Research found frontier agents dropping to ~11% on discriminative subsets, and resolution on genuinely novel issues runs 18–20% measured (SWE-bench-Live). Vendor multipliers are marketing until measured independently, which is why they are colorless on this page. And the productivity evidence is genuinely unsettled: high adoption is driven substantially by perception, and developers systematically misjudge their own AI speedup (METR documented a ~39-point gap between perceived and measured). Treat those three facts as governing constraints, and the rest of the snapshot reads correctly.