Build to Delete §

Pattern: A named solution to a recurring problem.

“Every component in a harness encodes an assumption about what the model can’t do on its own.” — Prithvi Rajasekaran

Build each piece of harness scaffolding that makes up for a model weakness as a recorded, switchable bet, so that when a stronger model arrives you can turn the piece off, measure, and delete it if it no longer pays.

Also known as: Harness Impermanence, Architecting for Model Impermanence

Open the harness around any agent that has been in production for a year and you’ll find layers of old fixes. One step splits every task into small chunks because last spring’s model lost the thread on long ones. A wrapper retries malformed JSON three times, written before structured output was dependable. A line in the system prompt tells the model, in capital letters, not to stop early. Each was the right call when it went in. Nobody remembers which of them still are.

Philipp Schmid’s name for the remedy is Build to Delete. It sounds like a contradiction until you read it as an instruction: write each fix expecting to remove it, and leave yourself a way to tell when that day has come.

Understand This First §

Context §

This is a heuristic for anyone who owns an agent harness across more than one model generation. Harness Engineering teaches you to add machinery around a model: instructions, tools, sub-agents, hooks, retries, checkpoints. Most of that machinery is added for a reason that sounds like “the model can’t yet do X reliably.” The reason is true on the day you write it. Frontier models then change every few months, and the harness doesn’t change with them unless someone makes it.

The idea has an old ancestor. Rich Sutton’s “The Bitter Lesson” (2019) observed that across 70 years of AI research, methods that scale with computation have repeatedly overtaken methods built on hand-coded human knowledge. Harness scaffolding is hand-coded knowledge about a model’s limits. The more of it a harness carries, the more of it the next model can make obsolete.

Problem §

How do you add the scaffolding today’s model needs without carrying it forever after the model stops needing it?

Forces §

Solution §

Sort each harness component into structural or compensating, write down the bet behind every compensating one, build it so it can be switched off cleanly, and re-test the bets each time you change models.

Structural components would still be needed if the model were perfect. The Sandbox, permissions and the Approval Policy, the tools that reach your own systems and data, logging and tracing, and the human checkpoints of Bounded Autonomy all belong here. A stronger model needs them at least as much as a weaker one. They aren’t candidates for this pattern.

Compensating components exist because the current model falls short. Forced task decomposition, a separate planning step, context resets, and repair loops for malformed output are all compensations. So are narrow tools that pre-digest work a general tool could handle, hand-written control flow that stands in for the model’s own judgment, and prompt lines that push against a known bad habit. Each one is a bet that the model can’t do something on its own.

Treat those bets the way you’d treat any assumption you expect to expire:

The same runs can also turn up a weakness the old model didn’t have. A compensation added for it goes in the same way: labeled, switchable, and dated. Harnesses don’t shrink to nothing; their mix of temporary parts turns over with each generation.

Delete on observed capability, not on expectation. The pattern doesn’t say harnesses are going away or that you should build thin ones in anticipation of a smarter model. It says you should be able to tell, within a day of an upgrade, which parts of your harness the new model has made unnecessary.

How It Plays Out §

Prithvi Rajasekaran, on Anthropic’s Labs team, built a harness for multi-hour application builds and published what happened to it over two model releases. Claude Sonnet 4.5 tended to wrap up early as it sensed its context filling, so the harness periodically wiped the context and started a fresh agent from a handoff note. Opus 4.5 largely stopped doing that, and the resets came out in favor of the SDK’s ordinary Compaction.

The same harness also broke each build into feature-sized sprints, each graded by a separate evaluator. With Opus 4.6 the builder stayed coherent for more than two hours without them, so the sprints went too, and the evaluator moved to a single pass at the end. The evaluator itself stayed, because the task still sat beyond what the model did reliably alone, and so did the planner, without which the builder scoped the work too small. His first attempt was to cut the harness back hard, all at once. It couldn’t match the original results, and he couldn’t tell which of the missing pieces had been load-bearing. Removing one component at a time, and checking the result after each, is what worked.

Vercel’s d0, an internal agent that turns plain-English questions into SQL against the company’s analytics data, started with 17 specialized tools. Its team replaced them with two: a sandboxed shell where the agent reads the data definitions with grep, cat, and ls, and a tool that runs SQL. On five representative queries, success went from four of five to five of five, and average run time fell from about 275 seconds to 77. Token use dropped 37% and step count 42%. Five queries is a small test, so treat the numbers as a direction rather than a measurement.

The team concluded that the old tools were solving problems the model could already handle, and that every model update had meant recalibrating them. They’re also clear that the change depended on data definitions that were already well documented: a shell is only as useful as the files it can read. That documentation is structural, and it stayed.

A platform team maintains a code-change agent whose harness has grown over three model generations. When they move to a new model, they run their 300-task eval suite with everything switched on, then with each of six compensating components off in turn. The JSON repair wrapper never fires in any run, so they delete it. Removing the forced planning step makes small fixes faster with no loss in results. Multi-file refactors, though, fail 14 percentage points more often, so the planner stays for tasks over a size threshold and its bet gets a new date.

The capitalized “do not stop early” line turns out to make the new model pad its final reports, and the scores improve without it. The sandbox, the approval hook on git push, and the tracing layer never enter the experiment, because they were never there to make up for the model.

Warning: A component that started as a compensation can become load-bearing for something else. A retry wrapper that also writes the failure counts your on-call dashboard reads will pass its eval when you switch it off and break the dashboard when you delete it. Before deleting, search for every reader of the component’s output, not only the model behavior it was written to fix.

Consequences §

Benefits. The harness gets smaller as models get better instead of only growing. Runs get cheaper and faster when retired steps stop spending tokens and adding round trips. A newer model’s capability actually reaches your users instead of being capped by instructions written for its predecessor. And the reasons behind the harness are written down, so a new maintainer can tell a safety control from a workaround at a glance.

Liabilities. The pattern only works if you have an eval suite good enough to detect what a removal costs. Without one, you’re deleting on vibes, and the cost shows up in production. Testing components one at a time multiplies eval runs on every upgrade, which takes time and money. Components interact, so a piece that tests as removable alone can matter in combination with another removal.

The bet labels rot like any comment unless the upgrade checklist forces someone to update them. Capability doesn’t improve evenly, so expect some upgrades to retire nothing, and expect a few compensations to survive several model generations.

The payoff also depends on how often you change models. A deployment pinned to one model version for regulatory or stability reasons gets little from this pattern until the pin moves. And don’t let a vendor’s launch benchmark stand in for your own measurement. A model that tops a leaderboard can still need the scaffolding your tasks depend on, which is the trap Benchmark Mirage describes.

Related Articles §

Sources §