Two teams, the same model, the same task. One ships something people trust; the other ships a demo that works on stage. The model was never the variable.
Ask why an agent isn’t reliable and you’ll usually get an answer about the model. It’s not smart enough yet. We’re waiting on the next release. Maybe we should try a different provider.
Then look at the actual failures. It did the same thing twice, because a retry replayed a step that had already run. It lost the thread halfway through, because the process restarted. It ran for forty steps, spent real money, and produced nothing. Nobody can say what it touched last Tuesday.
None of those are comprehension failures. A better model doesn’t fix a single one of them.
What a harness actually is
A harness is everything around the model call.
The model contributes exactly one thing: given what it can see, what should happen next. Every other property your system needs — that it sees the right things, that what it asks for is allowed, that it survives a crash, that it stops eventually, that you can reconstruct what happened — belongs to code you wrote.
The comparison I keep coming back to is Postgres. Choosing it is a real decision, and it’s maybe five percent of your data layer. The schema, the indexes, the migrations, the pooling, the backups — that’s the work. Nobody ships a database and calls the data layer done.
Harness engineering is treating the model as a component rather than as the architecture.
How the leverage moved
The shape of this work has changed twice in three years.
Prompt engineering mattered when models were fragile. Phrasing was load-bearing; the same request worded two ways gave you two different outcomes, and real skill went into finding the wording that held.
Context engineering replaced it once models got robust enough that wording stopped dominating. What you put in the window mattered more than how you phrased it.
Harness engineering is where it lands next, and the reason is unglamorous: models got good enough that they stopped being the thing that breaks. The failures that survive are execution failures — and those have been a systems problem the whole time. Model quality was just loud enough to hide it.
The swap test
Here’s the version of this claim you can check rather than argue about.
Change the model your agent runs on. One line of config. Run your evals.
- model: 'provider-a/flagship'
+ model: 'provider-b/flagship'Three things can happen, and each tells you where your value actually lives.
It holds. Scores move a few points either way; nothing structural breaks. Your reliability is coming from the harness, and the next model release is free upside rather than a migration.
It collapses. Something was load-bearing on one model’s quirks — a parser built around its output format, a prompt tuned until it happened to work, retry logic shaped to its specific failure mode. You didn’t build a system. You fitted a curve to one model.
You can’t tell. No evals. This is the most common answer and the most useful finding: if you can’t measure whether a change helped, you aren’t engineering the system, you’re redecorating it.
The swap happens to you whether you plan for it or not. Models get deprecated. Prices move. A cheaper option lands that’s good enough for two-thirds of your calls. A provider updates a model under a name you pinned.
What reliable harnesses converge on
Teams building this seriously keep arriving at the same components, whatever they’re built with. Each one is a question somebody had to answer.
Context is assembled, not accumulated. The naive loop appends every turn and resends the lot; the bill grows every step and the signal thins out. What works instead is separating what happened (the full record, in your database) from what the model sees this turn (the task, compact running state, recent work). One caveat that costs real money: prompt caching keys on an exact prefix, so rewriting earlier turns is a guaranteed cache miss on the next call. Compact rarely and in big steps — never on a sliding window.
Tools run through your code, not the model’s intent. The dispatch is an RPC handler taking arguments from a caller you didn’t write: allowlist the name, validate the shape, and derive scope from the session rather than from the arguments. Strict tool schemas guarantee shape, never entitlement — a perfectly-typed ID belonging to another tenant passes every validator you’ll ever write.
Execution has to survive the process dying. This is where “reliable” is won or lost.
Durable execution engines give you most of that — a completed step replays its recorded result instead of running again. What none of them give you is exactly-once side effects: the step that was in flight when the process died runs again on recovery. That last mile is yours, and it’s spelled idempotency key, derived from the run rather than freshly generated, so a resumed attempt presents the same key it presented before it crashed.
Irreversible acts wait for a human — durably. The naive version blocks on an await while
someone decides, which fails the moment that person takes three days, or the process restarts.
The version that works parks the run as a suspended state your database holds, and resumes it
when the decision arrives — an hour or a week later, on a different process. And re-check scope
at execution, not just at request time: the approval was given against a rendering of the
arguments that may be days old.
Budgets belong in the runtime. Steps, tokens, spend, wall clock. A cap in a system prompt is a request; a cap in the loop is a control.
Model-written code needs somewhere to run. Letting an agent write one program instead of round-tripping ten tool calls through the model is genuinely better — fewer steps, less arithmetic done in tokens. It also means you’re now executing generated code, which needs a real sandbox. In-process JavaScript isolation isn’t one: the risk isn’t the globals you removed, it’s the host objects you passed in.
More agents are a runtime decision, not an intelligence upgrade. A second agent earns its keep when it holds something the first must not — a narrower credential, its own context window. If you can’t name the thing being isolated, you’ve written a section header, not a component.
Why this compounds
A prompt tuned to a specific model depreciates the day that model is superseded. A harness appreciates.
Every component above is model-independent. Better context assembly helps on every model you’ll ever run. A tool layer that validates and scopes keeps working. Durable execution doesn’t care who generated the step. And when the next model lands genuinely better, a model-independent system inherits that improvement for free — you change a line and keep the gain.
That’s the shift, and it’s a strategic one. Everyone calls roughly the same frontier models. Nobody’s advantage is which one they have access to. The advantage is what got built around it, and unlike model access, that part is yours.
The honest limit
A harness doesn’t make a weak model capable. It converts a capable model into a reliable system. Conflating those is how teams end up adding orchestration to a task the model simply can’t do, which mostly buys more ways to fail.
So the claim isn’t that models are interchangeable. It’s narrower: within the frontier tier, on your specific task, the spread between models is usually smaller than the spread between a well-built harness and a careless one. Sometimes that’s wrong — and the honest move is to measure rather than assume.
Build the swap test. Build enough evals to make it mean something. Then put your effort into the layer you control, because the model is the one component you didn’t build, can’t inspect, and can’t hold still.