Your Agent Architecture Might Matter More Than the Model

The harness—tools, safety checks, feedback loops, and sub-agent orchestration—can cause a 20-point performance swing on the same model, according to the Harness Bench paper. That suggests harness engineering is a primary lever for agent quality, not just model selection.

If harness improvements can make a weaker model perform like a cutting-edge one, teams could switch to local or open-source models, slashing API costs and reducing vendor lock-in. The strategic shift would move investment from model access to internal platform expertise.

However, the speaker argues existing frameworks are insufficient and proposes a new language, Agency, still only 6 months old. The demo tasks are trivial—fixing a median function—and the leap from that to production systems is enormous. The language-level claim remains unproven at scale.

Engineering leaders should treat harness design as a core competency but not rush to adopt a nascent language. The rising importance of agent architecture will affect hiring, tool selection, and safety governance. The talk is a useful nudge, not a blueprint.