Inference, Diffusion, World Models, and More | YC Paper Club

This YC Paper Club session distills four papers into actionable signals for engineering leaders. Tanishk on “Speculative Speculative Decoding” reframes inference not as a cost problem but as a capability lever. The core idea: if model intelligence scales with test-time compute, then tokens-per-second directly caps peak intelligence. SSD parallelizes drafting and verification (previously a sequential bottleneck) by predicting verification outcomes on the draft model, hiding latency and boosting throughput. The practical upshot is that serving large models efficiently is now a strategic advantage, not just an infrastructure cost.

Stannis on “Diffusion Model Predictive Control” decomposes robotic control into a learned action proposal and a multi-step dynamics model, then uses a simple sampling-based planner. The key win is runtime adaptability—agents can handle novel rewards or broken actuators without retraining the whole policy. The factorization means the dynamics model can be updated separately with small amounts of play data, a pattern relevant for any team managing models in non-stationary environments.

Isaac on “Lay World Model” addresses the collapse problem that plagues latent-space world models. Instead of a bag of tricks, they enforce a healthy latent distribution with a single regularization term (SIGG). This yields a 50x speedup and runs on a single GPU. The crucial capability: world models can quantify their own prediction error, detecting out-of-distribution inputs at runtime—a safety feature absent in model-free policies.

Ash on “Deep Learning is Not So Mysterious” uses PAC-Bayes bounds to dissolve the mythology around overparameterization and double descent. As models grow, both the training loss and the compressibility of solutions improve. The implication: generalization is not an empirical mystery; it can be bounded and potentially optimized for, offering a path beyond blind scaling.