Anthropic Calls for Leaner Harnesses for Agentic AI Development

Anthropic is advising DevOps teams to rely on lighter-weight harnesses to drive agentic engineering workflows to de-duplicate instructions in a way that ultimately reduces costs.

Speaking today at an AI DevCon event hosted by Tessl, Nicolas Carmont Zaragoza, a member of the technical staff for Anthropic, told conference attendees that in many cases the harnesses that application development teams have adopted are now issuing duplicate instructions to AI models that result in increased consumption of tokens.

As AI models become more advanced, many of the instructions that application developers once needed to issue are now automatically created by the AI model itself. Instructions embedded in the AI model are always going to run, so application development teams should instead focus on lean harnesses that issue high-level prompts to reduce the chance instructions will clash with one another, said Carmont Zaragoza.

In fact, DevOps teams should regularly revisit how their harnesses are constructed because they have a limited shelf life as AI models continue to advance, he added. For example, verification is now built into the AI model so there is no need to rely on a harness to provide explicit verification instructions, noted Carmont Zaragoza. In effect, too many DevOps teams are using harnesses to micromanage AI models in a way that is no longer required, he said.

DevOps teams should, in effect, “unharness” instructions by cutting stale and outdated prompts to create a new baseline to grade outcomes, added Carmont Zaragoza. The objective is no longer to build the biggest harness possible, he added.

Additionally, DevOps teams should spend more time evaluating how outcomes were generated, he added. It’s not uncommon to, for example, discover in the transcripts provided that AI agents are wasting tokens reasoning across conflicting sets of instructions, said Carmont Zaragoza. Beyond tracking how AI models are “venting,” DevOps teams can ask the model how to improve evaluations, he noted. Software engineers should continuously review those evaluations rather than relying solely on traditional approaches to managing test coverage.

Anthropic previously disclosed that it has cut by 80% the size of the system prompts it internally uses to instruct AI models and is now making a stronger case for DevOps teams to follow suit.

Exactly how DevOps workflows in the AI era will evolve isn’t entirely clear, but it’s apparent that there is a general move toward creating software factories through which those workflows are increasingly automated. The challenge is that a debate is emerging over to what degree to rely on an AI model or some type of complementary platform through which AI skills can be defined and reused. The providers of AI models are, in effect, signaling that more AI skills and related instructions are in time going to be embedded into the AI model.

Eventually, the level of expertise needed to successfully build and deploy software using AI models is going to continue to decline as it becomes more feasible to turn intent into code that runs with minimum additional effort required.