Darkening a factory
One of the five pipelines was rebuilt with its middle gates handed to agents. This page records which gates were darkened, what replaced each, and — the part worth having — what three runs actually measured.
What stayed lit, and the argument for each
The design note refuses the stronger claim in writing: this pipeline is not fully dark, and says so rather than selling itworkspace. Two gates stay on.
| Gate | Why it cannot go dark |
|---|---|
| The brief | It is the definition of done. Nothing downstream can check a site against a contract nobody agreed to; a machine-approved brief makes every later judgment self-referential.workspace |
| The publish | It is the point of no return and the point where authorship attaches. Approving is authorship; a URL is outward-facing and cannot be un-served.workspace |
The first of those is the more transferable insight. A pipeline that generates its own acceptance criteria and then grades itself against them is not verified; it is consistent. Someone outside the loop has to fix the target.
What replaced each darkened gate
The governing principle: deleting a gate hides its residue rather than removing it, so each darkened gate got a named replacement, and the replacement is auditableworkspace.
| Was | Became | What was lost, and how it was covered |
|---|---|---|
| Design-mock approval | judge agent | Pinned to a different model family from the maker. Still judges the mock, so rejection costs minutes rather than a loop. |
| Per-round feel check | deferred, not replaced | Feel is the residue by definition. Claiming a judge covers it would be the dishonest move, so it moves whole to a single handoff. |
| “User says stop” | convergence rule + cap | Without a human ending the loop it needs its own stop condition, and a non-converging loop must escalate rather than grind. |
The middle row is the honest one. The feel check was moved rather than automated: from after every round to once at the end, where it is a single interruption that has to be worth makingworkspace.
The trap this design is arranged against
A check is only as independent as its failure modes. When a judge’s errors correlate with the maker’s, every individual verdict looks considered and the aggregate is worthless on exactly the dimension the judge was built forworkspace.
This was not hypothetical. The upstream pipeline hit it in July 2026: a judge sharing the maker’s model family waved through that family’s own prose tells, silently. The fix was to pin the taste judges to a different model familyworkspace.
A dark pipeline multiplies the exposure, because no human downstream will notice a gate that has quietly stopped working. Hence four rules, each of which is a general lesson about machine verification:
- Taste judges run on a different distribution from the maker.
- A model floor: a dark run below a set capability is merely an unattended cheap run, and the orchestrator refuses it.
- Judges are dispatched fresh every round, never continued — a judge carrying its own prior verdict defends it; a new instance re-derives.
- Judges regenerate their own evidence. No judge grades screenshots or a build the maker left behindworkspace.
What three runs measured
The pipeline writes an append-only gate ledger: gate, who held it, model, round, verdict, what it caught. It costs a few lines per stage, and it is what made the retro mechanical rather than anecdotalworkspace.
The gates that earned their place
Across three runs the darkened gates caught more than thirty real defectsworkspace. The workhorse was the editorial judge, the one pinned to a different model family, and its catches are worth listing because they are the class of thing people assume machines cannot catch:
- fabricated quotation marks around text that was narration, not quotation;
- a fidelity regression introduced while fixing fidelity;
- an attestation gap: the site’s colophon promised a completeness the register did not deliver.
None of those were caught by the human at the end, and the retro’s own assessment is that none of them likely would have beenworkspace. The model pin was buying independence.
The gates that failed, and how
Two failures, both instructive, and neither of them about model capability.
A judge asserted numbers it had not computed. A verdict quoted contrast ratios to two decimal places that were never calculated, and a later cold pass overturned the reasoning entirelyworkspace. A wrong number quoted precisely reads as measurement. The fix was a rule: never assert a number you did not compute.
An evidence-free waiver survived every gate. An entire class of assets the brief had opted into was missing from a site, and every visual verdict, including the final conformance pass, was rendered against the site without them. The absence had been waived on a record that the generating tool was unavailable, and that record was false: the probe had aborted early for an unrelated reason, and the failure was read as an answerworkspace. The maker’s own re-check caught it, after the packet was assembled.
That second one generalises well beyond this pipeline. A capability probe that fails for an unrelated reason is not an answer to the question you asked — and a waiver without the failing output behind it is only a claim.
The residue that stayed residue
Three runs, consistently, and correctly routed to the human every time: long-scroll feel, sticky rail behaviour, the mobile drawer, focus states: “not decidable from static captures”workspace. No proposal was made to automate them. The packets named them and that is the design working.
The honest zero
One run reached a human review. What that human caught, in the ledger’s own words: nothing — the judges caught it all, with the caveat recorded beside it: found nothing, not verified everythingworkspace.
The retro’s treatment of this row is the most disciplined thing in it. It reports zero content escapes and then states, in the same breath, that this is a count and not a conclusion, from a sample of oneworkspace. A trustworthy zero is what makes every other row worth reading.
The finding: dark-after-design beat dark-after-brief
The sharpest empirical takeaway of the whole exercise is about which gates to darken first, and it is worth stating as a claim rather than leaving implicit. Dark-after-brief (automating from the intent onward) removes the cheapest, highest-leverage gate in the pipeline first. Dark-after-design captures nearly the same attention savings while preserving upstream judgment, because a rejected mock costs minutes and a rejected built site costs the whole loopworkspace. The three runs bore the choice out from the other side: the brief gate pinned binding corrections in every runworkspace. That is the exact gate the dark-after-brief variant would have deleted.
Build dark-after-design, terminate at a live private URL, keep fabrication-prone archetypes lit, and let retro evidence decide which gate goes dark next.workspace
What was re-lit afterwards
Nothingworkspace, at least on the evidence available, which the retro is careful to bound. But two things were confirmed as permanently lit, and one habit was corrected:
- Both lit gates paid. The brief gate pinned binding corrections in all three runs. The publish gate’s catches were human-shaped (spend decisions, a link sweep) even though its content catch was zeroworkspace.
- The loop stalls rather than passing. One run exited at its round cap without converging, and the design held: it stopped and said so instead of grinding to a cosmetic passworkspace.
- “Settle” was redefined. The human pushed back on being asked to adjudicate four unambiguous defects, in his own words: unambiguous defects should be fixed and reported, not turned into a decision round. The skill was rewritten to split findings into “yours to fix” and “genuinely the user’s”workspace.
That last one is a small process fix carrying a large principle. A gate that hands a human decisions that are not really decisions burns the resource it exists to protect.
It supports chapter 6's claim that marginal catch rate per gate is the only honest input to a darkening decision — the retro exists because somebody recorded that column while it was still boring. It supports chapter 7's claim that fresh instances re-derive where continued ones defend: a cold pass overturned a warm one, on the record. It tests nothing about scale, stakes, or brownfield code. These runs produce disposable websites. Nobody's billing system was on the line.
Next: how this site itself came out of a five-way benchmark.