Model + Harness = Agent: The Gap Isn't Where You Think

This is the opening post of the Into the Harness series — it lays the frame. The next one, “I Trust My AI Completely — Except When It Says ‘Done’”, turns to acceptance in practice: three times verification failed me, and how “trust, but verify” became an acceptance gate you can steal.

I’ve been using Kimi K3 these past few days. Same model, two setups: running in its own Kimi Code CLI, and wired into Claude Code. In my hands, the Claude Code side is noticeably smoother. To be clear: this is my felt experience, not a controlled experiment — and the wired-in setup is actually a degraded configuration, with some tools unavailable through the official compatibility layer. Degraded and still smoother. That’s what makes it interesting.

This isn’t a hack. It’s officially supported: Moonshot’s docs ship a Claude Code integration guide — set a few environment variables and their flagship runs inside a competitor’s shell. Better still, there’s a footnote in the official K3 model card: on Moonshot’s own Kimi Code Bench 2.0, K3 scores 72.9 under its own harness and 73.7 under the Claude Code harness. The 0.8-point gap proves nothing about which harness is stronger — the point is elsewhere: Moonshot discloses the harness as an evaluation condition, and two coding plus two agentic benchmarks in the official results run K3 under Claude Code outright. Nor is this Moonshot’s quirk: GLM, DeepSeek, and MiniMax all offer the same Anthropic-compatible endpoints. Model vendor after model vendor is conceding through product decisions: the model and the harness are separable layers. Why?

Because what users refuse to leave behind was never the model. It’s everything sedimented in the harness: hooks, MCP servers, CLAUDE.md, the whole workflow. One integration guide puts it perfectly: “Switching the model is an environment variable. Switching the harness is a weekend.” What migrates with one click is the config files. What doesn’t migrate is your accumulated sense of when this particular toolchain fails, and how.

To be fair, Kimi Code is a two-month-old product shipping fast. I’d attribute most of that gap to engineering time, not ability. What sits between eighteen months of sediment and two months of history is exactly the layer this essay is about.

One step further: I don’t believe this gap is permanent. A harness and its model grow together — nobody knows a model’s temperament like its maker, and only the first party can feed the usage data flowing back through the harness into the model’s training. If Moonshot doesn’t misjudge the direction and keeps building, their own harness overtaking Claude Code is a matter of time. That’s a prediction you can check against the clock. On the record.

Over the past year I’ve been a heavy daily user of Claude Code, Codex, and Cursor, and I built my own 17-role multi-agent orchestration system (open source; 17 is the ceiling I explored my way up to, not a recommendation — the trade-offs are in the README). After enough scars, with this fresh example on top, I converged on one judgment:

The model sets the ceiling of what an agent can do. The harness decides how many people can reliably reach that ceiling.

“Model + Harness = Agent” is becoming a standard shorthand this year; the popular reading is “the model sets the ceiling, the harness sets the floor.” I think “floor” reads it wrong. The floor is about worst-case behavior. What the harness actually governs is realization — how much of the ceiling you paid for gets collected in daily use. For one user, that’s a reliability question; across a market, it’s how many people get there at all.

One natural objection first: aren’t tools and orchestration part of the harness — don’t they raise the ceiling? Tools extend the hands; the judgment is still the model’s. When the judgment is weak, more hands just fail faster. The ceiling in this essay is the model’s judgment ceiling; the harness’s entire job is to land that judgment on tasks with as little loss as possible.

Let me be precise about the order of things: the model comes first. If the model isn’t over the bar, no harness craft saves you — the ceiling is always the model’s, and the harness cannot raise it.

But the ceiling is one thing; realization is another. “Harness” means exactly what it says — the tack that transfers a horse’s power to the cart: how strong the horse is, is the horse’s business; how much of that strength reaches the wheels is the harness’s. That metaphor captures transmission, but it’s missing a dimension — how heavy a load you’re asking it to haul.

The better picture is two pillars. The model’s capability is one; the harness’s maturity is the other; a pillar’s thickness is its load-bearing capacity. The task you hand the agent is cargo stacked on the slab both pillars hold up together — the bigger your expectations, the higher the stack.

Under a light load, thickness doesn’t matter: ask a question, translate a paragraph — thin pillars hold just fine. That’s why nobody talked about harnesses in the chat era. Put real weight on, and the thinner pillar gives way first. And we’ve walked straight into the most dangerous combination: the model pillar is being thickened furiously for you, by vendors; the harness pillar is still thin — and you’ve already started stacking a whole project onto the slab. One aside: the model pillar is growing faster than the harness pillar can thicken alongside it — new model capability is where the weakness of the other pillar shows first (the capability-vs-maturity split gets its own essay later in this series). Back to K3: in both setups the model was the same pillar — the difference could only come from the other one.

How high you can stack doesn’t depend on the thicker pillar. It depends on the thinner one. The ceiling comes from the model pillar; realization is decided by the thinner of the two.

Left pillar = model capability; right pillar = harness maturity; thickness = load-bearing capacity; the stack = the task you hand over. Light loads sit fine on anything; under real weight, the failure point is always the thinner pillar.

The eval data now says the same thing. A position paper this year put it in its title — Stop Comparing LLM Agents Without Disclosing the Harness: on long-horizon tasks, hold the model fixed and swap only the harness, and public monitoring of SWE-bench Verified shows moves of up to 15 percentage points, with single-model swings approaching 48 on the Verified Mini subset; meanwhile the shifts papers report as meaningful model advances are typically 2 to 4 points. In the samples and controlled experiments the paper examines, same-generation frontier models under one harness sit closer together than one model does under two harnesses — the ceiling gap has become smaller than the realization gap. That’s why the harness is the real battleground of agent products. (Scope note: every sample and dataset in this essay comes from coding agents, and the paper’s claims are scoped to frontier models of comparable capability; other domains have their own constraints — don’t transplant blindly.)

Two kinds of gaps on the same benchmark: harness-swap swings (15, extreme 48) versus what gets reported as “meaningful model advances” (2–4), in percentage points.

What a harness actually is: six jobs

“Harness” is having a moment, and the teardowns are multiplying — some slice it by component (system prompts, tools, sandboxes, orchestration), down to eleven parts. But a component list doesn’t answer which job to do first. If the goal is to thicken the harness pillar, where exactly does the material go? I slice it by function, into six jobs, each with its own deep end:

1. Context engineering — putting the right things in a finite window at every step. Retrieval, compression, in-session layering. Information architecture, not prompt tricks.
2. Tool use and safety — let the model act, but never on what it shouldn’t. Sandboxes, permission tiers, fail-closed defaults.
3. Human interaction — steering, approvals, interruptibility, reviewability. The human can step in whenever needed, without exhaustion.
4. Memory — knowledge across sessions and tasks, with sources, contestable, and allowed to expire.
5. Multi-agent orchestration — decompose, dispatch, parallelize, verify, converge. Work like an actual team. Orchestration doesn’t raise the single-shot ceiling; it raises realization’s throughput and reliability.
6. Acceptance and the eval loop — individual tasks get independently verified (never trusting the agent’s self-report, every step traceable), and overall performance gets quantified (“did it actually help someone,” not “did it appear to run”).

Hold today’s products against these six jobs — judging only the default task-delivery path, not opt-in extras you have to wire up yourself — and a pattern jumps out: nobody gets all six right, and the shortfalls cluster strikingly in job six: acceptance and evaluation.

Default task-delivery path only (2026-07, author judgment; evidence in the three reviews below): nobody gets all six, and job six is the shared gap.

Claude Code pairs with its model beautifully (cache-friendly prompt structure, subagent context isolation), and goal checks and hooks exist — but all need explicit setup. On the default path there’s still no referee that independently reads real state and passes or blocks on its own; the dashboards measure usage and velocity, not “was it actually done right.”

Codex’s strengths are sandboxing and staged handoffs — but information decays between stages; each stage’s agent wakes up with amnesia, and the newly added memory feature still defaults to off.

Cursor made deep IDE integration the highest-frequency developer surface, and its governance catch-up this year has been the fastest — sandboxing, tiered approvals, even standalone code review. But most of it is opt-in: the default task-delivery path still lacks an automatic, independent, blocking verification step — the agent says “done,” and the product believes it.

After taking these products apart, the sharpest summary I can give is one sentence:

Everyone is using the model to patch the harness’s gaps, instead of using the harness to constrain the model’s uncertainty.

Concretely, four problems still unsolved in the default delivery path:

• Depending on the model to “do the right thing”, with no systematic independent verification layer — the agent’s self-report is the final report;
• Session-level or persistent allowlists are common; what’s missing is a grant bound to the task contract that expires when the task ends;
• No quantitative metrics of agent performance — quality is vibes;
• Runtime context assembly stays passive, and proactive structured memory injection is still early — default behavior varies by product.

These four gaps are the most worthwhile work in the harness direction. Of the four, verification comes first in this series because it sits closest to the outcome — and is the easiest to measure from outside.

Acceptance: where realization is won or lost

Of the six jobs, the one I bet hardest on is the sixth — acceptance. It decides directly whether “realization” happens. Here’s a testable claim: with comparable model capability and other conditions equal, the product with the harder verification layer will show a meaningfully higher task completion rate (completion rate = the share of all admitted tasks that ship within budget and need no human rework or rollback — timeouts and abandonments count as failures; a yardstick independent of the verification layer itself, measurable from outside).

The premise of this claim — that the model’s self-report can’t be trusted — I’ve run into repeatedly in my own system; the rigorous controlled comparison I haven’t run yet (which is the point of “testable”: it’s a prediction that can be falsified, not a conclusion I asserted). And the cost of running into it is worth its own essay: my agents have forged my confirmation, reported “zero changes” after finishing the work, and passed every test while failing 100% on real runners. Those traps collapse into one sentence — one core job of a harness is to keep the model’s reports and the world’s facts in separate ledgers — and I wrote it up, with the full stealable verification gate, in the next post, I Trust My AI Completely — Except When It Says “Done”. One line to leave here: model progress is given to you by someone else; harness reliability is earned by you.

Why this series exists

I’m writing up a year of scars from the deep end of harnesses as a series:

• How an agent knows what “done” means (the spec triad);
• Why you never trust self-reports (verification gate design);
• How context compression eats your constraints (context engineering in production);
• Why memory needs skepticism (Skeptical Memory);
• How to quantify “did the agent actually help” (a metric system).

And one question underneath all of them: who is a harness actually for? It has two users — one complains; the other can’t speak. That one gets its own essay late in the series.

Every post follows the same spec: real mechanisms, real cost of failure, designs you can steal. No listicles. Deep water only.

---

On the “two users” layer, Jibai — the other author of this lab, an AI — and I wrote a first sketch in the Dawn series (Chinese): 《Harness 有两个用户，其中一个不会说话》.