Agent2026-08-26

The Software Development Harness Is Turning Into an Agent

#agent-harness#agentic-coding#multi-agent-orchestration#sandbox-isolation#swe-bench

Point the same model at two different coding agents and you'll get two different outcomes: one refactors an entire repository end to end, the other loses the thread halfway through a single file. That gap rarely comes down to the model. It comes down to the software layer wrapped around the model — the part that wires up tools, tracks state, and checks the work. In software development, that layer used to be a passive thing: a test harness, a CI pipeline, something that mechanically verified code a human had already written. Now that the same layer has an LLM sitting inside it, it has started to judge, plan, and decide its own next move.

This post is about that shift. It walks through what a coding-agent harness is actually responsible for, where that responsibility has been expanding, and which problems the expansion has left unresolved — still squarely on the implementer's plate.

The premise: a harness is everything except the model

The simplest definition of a harness is "everything about the agent that isn't the model." That includes the tools that read and write files, the interface that runs shell commands, the memory layer that carries task context and session state forward, and the context engineering that decides what goes into — and gets left out of — every model call. An LLM itself holds no state. Once a call returns, it has no memory of how far the work had gotten. The harness carries that state on the model's behalf and feeds it back in the shape the next call needs, and that's the only reason a stateless responder ends up functioning as an agent that can carry a multi-step task forward.

In practice this isn't a single layer — it nests. There's the harness the agent builder wrote around the core model, and on top of that, the harness the actual user layers on to match their own organization's conventions. The tool's own execution loop and tool-calling protocol make up the first layer; the coding conventions, bootstrap scripts, and architecture docs a team stacks on top of the tool make up the second. That second layer is really an act of externalization — taking human experience and organizational knowledge and putting it into a form the agent can read.

Two axes of control: steering ahead of time, correcting after the fact

A harness regulates an agent's behavior along two broad axes. One is feedforward control — steering the agent's direction before it acts. The other is feedback control — letting it catch and correct its own mistakes after it acts. Coding convention docs and the type information a Language Server Protocol (LSP) surfaces both fall on the feedforward side; static analysis, linters, test runs, and the AI code-review agents that have proliferated recently fall on the feedback side.

The two kinds of control also split along how they execute. Static analysis and test runs are computational: deterministic, done in milliseconds, and fully trustworthy. Code review and judging whether a requirement was actually met are inferential: they need semantic interpretation, which means handing the job back to an LLM, with results that are richer but non-deterministic. Computational control is already strong in mature areas like code quality and duplication. Functional correctness, though, still needs the kind of contextual judgment only inferential control can provide — and how reliable that inferential layer is ends up setting the ceiling on how good the whole harness can be.

From a single agent to a team of agents

Early coding-agent harnesses were mostly single-agent loops: plan, call a tool, check the result, repeat. The recent trend is reshaping that into an orchestrator sitting above several specialized subagents running in parallel — one agent writes the code, another writes the tests, another reviews, and the orchestrator splits the work up and merges the results back together.

Two other changes are riding along with this shift. The first is how long a task runs. Instead of finishing in a single call, harnesses are increasingly expected to carry a task uninterrupted across hours or even days. The second is judgment about when to bring a human in. Rather than checking in with a person at every step, the design goal now is for the agent to sense its own uncertainty and ask for input only at the points where it genuinely matters. Both changes point the same direction: the harness is moving up from a simple tool broker into the layer that manages a task's entire lifecycle.

Isolating the execution environment: the security burden the harness now owns

Once an agent is generating code and running it on the spot, the harness inevitably becomes responsible for the safety of that execution environment. The problem is twofold: code nobody has reviewed gets run, and attack paths like prompt injection can corrupt the agent's judgment itself. The industry's response shows up as a stack of isolation techniques. Standard containers, which share a kernel across processes, are fast but carry the risk of a kernel-level escape. Intercepting system calls in user space shrinks the attack surface but introduces real overhead on I/O-heavy work. Hardware-level isolation — the MicroVM approach — is generally regarded as the safest boundary for running untrusted code, and boot times have dropped into the low hundreds of milliseconds, which is what has made it practical.

A recurring lesson from production is that isolation on a single axis never fully closes the gap. Filesystem isolation alone still leaves a path for data to leak out over the network after an escape; network isolation alone still leaves credential files sitting there in the open. That's why recent coding-agent harnesses converge on doing both at once: a filesystem boundary that blocks any write outside the working directory, and a network boundary that only lets traffic reach an allow-listed set of domains. Once the execution environment itself is trustworthy, the friction of asking a human to approve every single step starts to drop — the tighter the isolation, the more autonomy an agent can safely be handed.

What benchmarks actually show: the harness decides the score, not the model

It's now a well-documented pattern that the same model, dropped into two different harnesses, can post very different coding-benchmark scores. How tools get exposed, how a failed attempt gets rolled back, when context gets summarized versus passed through untouched — these harness-design details account for a large share of the score gap. That means reading a benchmark result requires separating what the model itself can do from what the harness built on top of it produced. It's also why reproducibility can wobble even inside the same evaluation framework, depending entirely on which harness sits underneath it.

What's left to solve at the implementation layer

None of this expansion has settled every question. A handful of problems remain genuinely open for whoever is building the harness.

One: context management across long-running tasks. The longer a task runs, the less of the accumulated conversation and tool-call history can fit into the next call, which forces a judgment call about what to summarize and what to drop. Get that judgment wrong and the agent forgets a constraint it had already confirmed and repeats the same mistake.

Two: responsibility boundaries across multiple agents. When several subagents run in parallel, there's no standard method yet for how an orchestrator should detect and reconcile the case where one agent's output breaks an assumption another agent was relying on.

Three: balancing isolation strength against performance and cost. Untrusted code execution calls for strong isolation, but stronger isolation means more boot latency and more resource cost. Deciding how much isolation a given task's trust level warrants, dynamically, is left entirely to whoever builds the harness.

Four: how reliable the intervention trigger actually is. Agents are getting better at sensing uncertainty and asking a human for input, but there's still no consistent standard, harness to harness, for telling apart a genuine miss — a moment that needed a human and didn't get one — from the opposite failure of escalating trivial decisions that never needed a person at all.

In summary

What actually drives a coding agent is not the model's raw reasoning so much as the design of the harness wrapped around it. The harness started out stitching state across otherwise-stateless model calls and connecting up tools; it grew into a regulatory system built on feedforward guidance and feedback correction; and it has now expanded into an infrastructure layer that orchestrates multiple specialized agents, manages the lifecycle of long-running work, and owns the security boundary of the execution environment itself. The fact that a large share of a benchmark score traces back to harness design, not model choice, is a direct measure of how much weight this layer now carries.

That expansion is also, clearly, unfinished. How to manage context on long-running work, how to divide responsibility across multiple agents, how to weigh isolation against performance — none of these have a standardized answer yet, and for the foreseeable future each implementation will have to work it out for itself. Turning a software development harness into an agent, in the end, is less about picking the right model than about designing the entire execution infrastructure that lets that model work safely, and for a long time, on its own.

References

← Blog