Why your coding agent costs explode and how to fix it

The core problem. Training a large coding model at scale requires optimizing a tradeoff between solving rate and inference cost. A model that solves every problem but takes an unbounded number of tokens is not useful. The talk, from a Cognition Labs engineer, walks through the infrastructure, stability, and reward design choices behind SWE-1.7 and SWE-2, arguing that the real frontier is not raw capability but how efficiently a model uses inference tokens.

Asynchronous infrastructure. The most basic RL loop is synchronous: sample rollouts from one model version, grade them, update weights. This forces idle GPU time when solving problems of very different difficulty. Cognition's solution is asynchronous RL: continuously submit new problems while updating the model mid-rollout. Each trajectory's tokens may come from different model versions. This keeps inference GPUs saturated, but introduces distribution mismatch between the policy that generated a token and the weight state used for the gradient update.

Weight synchronization and fault tolerance. Distributing inference across continents (SWE-1.7 trained across three) requires shipping weight updates efficiently. The key property: RL weight deltas are extremely sparse — only a tiny fraction of bits change per step. By compressing deltas by ~99% and uploading to cloud storage, each inference engine can pull updates asynchronously with only 3-4 seconds of pause. On the training side, Grace Blackwell racks of 16+ nodes require auto-swapping faulty nodes from a spare pool, a mechanism provided by CoreWeave.

Training stability through replay. The distribution mismatch from asynchronous updates introduces instability that can cause training runs to diverge. The fix: record and replay every inference-side decision during training. This includes quantization precision (FP8 for MLA attention layers, NBF4 for most weights), expert routing choices in MoE models, and sampling parameters like top-P. If inference discards low-probability tokens via top-P sampling, training must use the same truncated distribution. Without this replay, runs explode.

Preserving entropy for long runs. Long RL runs suffer from entropy collapse: the model becomes overconfident on a small set of tokens and stops exploring. The talk shows that penalizing a low-probability token actually reduces entropy further, because the highest-probability token gets upweighted more. The simple fix is changing the inference sampling distribution itself — using top-P to discard the long tail of improbable tokens. This alone lets training runs continue far longer and achieve higher performance.

The linear length penalty. The most important design choice is the reward function. A model that only optimizes solve rate will explode in inference cost, spending more tokens per problem without becoming more intelligent per token. The solution is a linear penalty on token length. The coefficient must be tuned so the reward function's iso-reward line is tangent to the model's test-time scaling curve.