← commits
c493fe0Aug 23, 2026rant(harness-engineering): part 2

The best harness is the one you can't see

Agent tools keep giving me panes to watch. I want a requirement in and a verified PR out.

invisible-harness.md · 114 lines · 8.0 KB

Agent tooling keeps trying to give me a better seat in front of the factory. I don't want a better seat. I want no seat.

Six tools keep showing up in my feed, in three shapes:

Different products, same tell: each explains what the human should look at while the agents run. The orchestrator gives you a pane to talk to. The multiplexer gives you better panes to watch. The factory gives you a ticket, then a run history or control center.

The interface keeps changing. The human attention budget does not.

Orchestration is a commodity

Part of my skepticism about orchestrators is how little machinery it takes to get one working. A coding agent already knows how to delegate. Add a stop condition and isolated worktrees and you have the interesting half:

sh
# After every turn, evaluate the goal against the transcript./goal every issue in docs/issues/ has a merged PR, or stop after 40 turns# Fan each issue into an isolated worker, then review the result separately.ultracode: for each file in docs/issues/, implement in a worktree, runmake verify, open a PR, and assign a separate reviewer agent

I've shipped MVPs with roughly this shape: an orchestrator grinding until the output was legible enough for me to inspect. People build cleverer versions: two sessions pointed at each other, one running Matt Pocock's grill-with-docs (https://github.com/mattpocock/skills) as the interviewer and the other playing founder until the ADRs are written.

Fun to watch. Also the problem.

Two things wrong with a model in the driver's seat

It is prompt-shaped. An LLM orchestrator's behavior depends on the initial prompt, the context it found, and whichever instructions happened to load. Hand one feature to two agents and you can get two codebases, not two implementations of one design. Adding a model that normalizes the requirement first moves the prompt one layer up. It does not turn the prompt into policy.

It has gravity. The orchestrator lives in a terminal, so the terminal stays open, so you look at it. Blue rings, unread badges, and blocked/idle tags make the looking more efficient. They do not make it comprehension.

In part 1 I called comprehension a budget: stop at a checkpoint, trace a critical path, and explain the change. Scrollback is the opposite. It spends attention continuously and builds no reliable model of the system. The factory that burned me had a green suite and a terminal I watched for 18 epics. The watching caught nothing.

My bet is that model capability has flattened enough that the harness is now the important variable. Once a model can run for hours, trust has to come from the structure around the run, not from your ability to supervise its transcript.

What the SDLC already knew

Strip the AI out and software teams already look like a harness for non-deterministic workers:

  1. A requirement is written and split into PR-sized issues.
  2. Dependencies decide what can start.
  3. Developers implement without anyone watching them type.
  4. CI, review rules, and a definition of done decide what can land.

The useful property is not that every step is deterministic. People are not deterministic either. The useful property is that authority is explicit: the ticket owns intent, the dependency graph owns order, the worktree isolates implementation, and the gates own admission.

As a function:

text
S : requirement → [issue]         split into PR-sized unitsG : [issue]     → dag(issue)      encode dependenciesI : issue       → diff            implement in isolationV : diff        → diff | reject   verify with independent evidenceH(x) = fold(merge, map(V ∘ I, G(S(x))))

x and S carry the highest-leverage human judgment. A vague requirement or bad split poisons the graph before a model writes a line. G, I, and the shape of V can be policy even when the workers inside them are probabilistic. V is where the oracle, mutation gate, and scheduled comprehension brake from part 1 belong.

The harness should have a contract, not a cockpit:

yaml
input:  requirement: docs/requirements/refunds.md  split: docs/issues/policy:  verify: make verify  oracle: testdata/refunds.json  mutation_threshold: 1.0output:  - verified_pr  - escalation_with_reason

Vanishing as the feature

CI is the obvious comparison. Nobody tails a runner because it might be lonely. You see a check, or a failure with a link. CI became infrastructure when it stopped asking for ambient attention.

Graded on that axis:

S, GIVwhat you look at
firstmatecoordinator, per promptagents in worktreescrew policytmux or another visible backend
HumanLayerhuman, at plan altitudesessions under a daemonworkflow-defined checkstasks, artifacts, session UI
herdrnonewhatever you runnonepanes with state tags
cmuxnonewhatever you runnonepanes with notification rings
OpenHandsexternal trigger, usually one taskone agent runworkflow-defined checksconversation or control surface
Factoryexternal trigger or missionDroid sessionsworkflow-defined checksticket, app, or run history

firstmate's watcher and herdr's detachable daemon reduce the babysitting, then both preserve the pane. HumanLayer detaches further, since agents keep running away from the laptop, while making it easy to jump back into sessions. Those are useful tools. They still treat the run as the product surface.

OpenHands and Factory get closer to the interface I want: work starts where the requirement already lives, and execution can be headless. But hiding I is only one third of the problem. If I still split every requirement by hand, encode no dependency graph, and let the implementation grade its own tests, then S, G, and V are still mine. The agent run vanished; the harness did not.

Invisible must not mean opaque. CI leaves logs, artifacts, exact SHAs, and failed checks. An invisible agent harness should leave the same kind of evidence: which requirement produced which issues, which SHA each gate examined, which oracle disagreed, which mutant survived, and why the run escalated. I should be able to audit it without having to watch it happen.

The harness I want has no pane in its steady state. Input is a requirement and an accepted split. Output is a PR that passed V, or an escalation with a reason and the evidence needed to act. Between those events, it should consume compute, not attention.