Your RL rollout fleet doesn't need to live in the same cluster

Modal's Nan Jiang proposes decoupling the rollout fleet from the training cluster in RL post-training. The default architecture forces both into one tightly coupled, high-bandwidth cluster—exactly the scarce resource. The insight: rollout only needs the served version of weights, not the optimizer states. Full parameter updates are dense but tiny; 99% of BF16

visible weights don't change per step due to Adam's small step size falling below the rounding floor. This creates a sparse, lossless delta of ~500MB instead of a 500GB full checkpoint. The implication is architectural. Rollout can run on any GPU globally, connected via commodity internet. A sidecar makes any serving engine version-aware,

applying deltas to catch up. This turns scattered inference capacity into an elastic rollout fleet, breaking the tight coupling that limits scaling. The tradeoff is increased system complexity: version management, async synchronization across regions, and the need for a shared bulletin board. Hype alert: the claim relies on Adam's small updates holding at

scale. The evidence is strong for BF16, but the speaker openly flags open questions for Muon optimizers and for async RL stability across global latency. Teams should watch their own weight-change sparsity before adopting. The core move—treating weight deltas as the sync unit—is proven and practical for anyone bottlenecked by GPU cluster availability.