~/octoblog / harness

Model + Harness = Agent: The Gap Isn't Where You Think

[series:Into the Harness] [date:2026-07-06] [read:4min] [words:3.9k] #agents#harness#methodology

Over the past year I’ve been a heavy daily user of Claude Code, Codex, Cursor, and OpenCode, and I built my own 17-role multi-agent orchestration system. After enough scars, I converged on one judgment:

The model sets the ceiling of what an agent can do. The harness decides how many people can reliably reach that ceiling.

Take the same model, put it in Claude Code versus a bare ReAct loop, and usability differs by an order of magnitude — the entire difference is harness. I’ll go further: of today’s gap in agent product experience, roughly 80% is not the model. It’s the engineering layer around it.

What a harness actually is: six jobs

“Harness” is having a moment, but most takes stop at “it wraps tool calls.” I break it into six concrete jobs, each with its own deep end:

  1. Context engineering — putting the right things in a finite window at every step. Retrieval, compression, layered memory. This is information architecture, not prompt tricks.
  2. Tool use and safety — let the model act, but never on what it shouldn’t. Sandboxes, permission tiers, human-in-the-loop, fail-closed defaults.
  3. Human interaction — steering, approvals, interruptibility, reviewability. The human stays in the loop without exhaustion.
  4. Memory — knowledge across sessions and tasks, with sources, contestable, and allowed to expire.
  5. Multi-agent orchestration — decompose, dispatch, parallelize, verify, converge. Work like an actual team.
  6. The eval loop — quantify “did it actually help someone,” not “did it appear to run.”

Look at today’s products through these six lenses and a pattern jumps out: every product gets two or three right.

Claude Code pairs with its model beautifully (cache-friendly prompt structure, subagent context isolation) but ships no built-in evaluation — whether the agent did well is vibes. Codex has the most industrialized pipeline (sandbox + staged handoffs), but information bleeds between stages; each stage’s agent wakes up with amnesia. Cursor proved that IDE-native is the highest-frequency developer surface, but its harness has almost no governance — constraining the agent is the user’s job, eyeball by eyeball.

What everyone is missing, in one sentence

Everyone is using the model to patch the harness’s gaps, instead of using the harness to constrain the model’s uncertainty.

Concretely, four shared failures:

  • Everyone trusts the model to “do the right thing” — no systematic independent verification layer. The agent says “done, tests pass,” and the product believes it.
  • Tool permissions are all-or-nothing. Allow bash once, and the whole session has bash.
  • No quantified measure of agent performance — quality is a feeling.
  • Context management is passive (the model remembers what it remembers) instead of active, structured injection.

Those four gaps are exactly what the harness direction is worth working on.

A testable corollary

If the judgment above holds, it yields a falsifiable prediction: with comparable model capability, the product with the harder verification layer will show a significantly higher task completion rate.

I’ve verified this in my own multi-agent system. The coordinator does not trust the executor’s self-report — after every delivery it independently re-runs the tests, checks the deliverable against the Done Definition written at task intake, and decides from diff size and sensitive-operation signals whether to add human review. The most immediate effect of adding that gate: fake completions stopped getting through. Deliveries that claimed success but didn’t actually build get bounced back with failure evidence attached, instead of landing on my desk to waste my time.

None of this depends on the model. Swap in any model and the layer still holds the line. That’s the value of a harness: model progress is given to you by someone else; harness reliability is earned by you.

Why this series exists

I’m writing up a year of scars from the deep end of harnesses as a series: how an agent knows what “done” means (the spec triad), why you never trust self-reports (gate design), how context compression eats your constraints (context engineering in production), why memory needs skepticism (Skeptical Memory), and how to quantify “did the agent actually help” (a metric system).

Every post follows the same spec: real mechanisms, real cost of failure, designs you can steal. No listicles. Deep water only.


This is the first post of the Into the Harness series. Next: “Your Agent Doesn’t Know What ‘Done’ Means — Designing the Spec Triad.”