Why Frontier LLMs Still Can't Write Fast Multi-GPU Kernels

Simran Arora argues the AI efficiency bottleneck has shifted from single-GPU compute to multi-GPU communication, yet current LLMs cannot navigate the nuanced tradeoffs required to write fast distributed kernels. She introduces Parallel Kittens, a minimal set of primitives that achieve state-of-the-art performance by carefully choosing transfer mechanisms and overlapping schedules.

Despite models like GPT-5.5 and Gemini 3 Pro excelling on single-GPU benchmarks, they solve only 28 of 87 multi-GPU tasks correctly in zero-shot settings, with performance plateauing quickly. The models fail to reason about collective ordering, intra vs. inter-SM scheduling, or selecting between copy engine, TMA, and register-level transfers.

Even with iterative coding agent approaches, capabilities plateau, suggesting that the challenge is not CUDA syntax but understanding fundamental distributed system tradeoffs. This is a sobering finding for teams betting on LLM-driven kernel generation for critical AI infrastructure.

The implication for engineering leaders is that current LLM reasoning is brittle for novel multi-GPU patterns, and hand-tuned primitives remain necessary for peak performance in production distributed training and inference.