The Inference Moat: How Cosine Builds Frontier AI on Millions

Cosine, a UK coding-agent startup, won government backing to build a sovereign LLM using free Isambard supercomputer compute. They avoid the inference cost trap by licensing model weights, not hosting, sidestepping massive inference clusters and focusing resources on a narrow, enterprise-aligned model.

Pullen argues that architectural choices like active parameter count and dense models are critical; open-source labs lag because they optimize for inference efficiency over capability. Cosine plans a model above 100B active parameters, funded by tight feedback loops with large UK enterprises that shape training data—a data flywheel unavailable to generalized models.

To improve with less, Cosine experiments with credit attribution in RL: rewarding process, not just outcome. They pinpoint decision points in trajectories and adjust rewards to reduce “slop” and teach reusable abstractions, contrasting with correctness-only RL that produces spaghetti code.

The team’s agentic swarm automatically decomposes complex tasks, though Pullen believes harnesses will matter less over time. The interview surfaces a tension: sovereign AI is possible with narrow scope and clever cost avoidance, but scaling and reliability remain open questions.