DoorDash shows why LLM distillation beats real-time serving

The core claim. Raghav Saboo argues that marketplace discovery is fundamentally a semantic understanding problem, not an engagement optimization one. At DoorDash's scale—spanning restaurants, grocery, retail, and pets—users arrive with broad missions like "I just adopted a puppy." Historically, retrieval and ranking systems optimized for click-through and conversion, which rewards popular items even when they fail user constraints. LLMs offer a new lever: the ability to reason about item meaning and shopper intent, but only if integrated thoughtfully into existing infrastructure.

The supervision primitive. DoorDash uses LLMs to generate graded relevance labels at scale. Human annotators create a high-quality seed set on a three-level scale. A stronger LLM audits suspicious cases—for example, items humans marked irrelevant but that convert well. After reconciliation with taxonomy models, the cleaned dataset fine-tunes a lightweight model like GPT-4o mini. This cheap, fast labeler then scores the full catalog offline. The key insight: spend expensive reasoning once, distill it into a model you can serve at production latency.

The catalog semantics primitive. Standard embeddings at e-commerce scale collapse relevance distinctions—related items sit near exact matches. DoorDash's fix is a two-stage contrastive method trained on the same graded labels. Stage one shapes global geometry with multi-level supervised contrastive loss. Stage two mines hard negatives (items the model confuses) and runs curriculum training. The result improved retrieval NDCG by 2.3% and translated to business metrics.

The memory primitive. DoorDash represents consumers at multiple time scales: long-term durable preferences from orders and searches, real-time in-session context, and stated preferences from agentic interactions like "Ask DoorDash." Each consumer's memory is materialized in three forms—human-readable text, latent embedding vectors for ML models, and graph structures linking consumers to brands, taxonomies, and revealed preferences. Graph-based embeddings outperformed taxonomy-based ones for retrieval at fine-grained levels. This memory framework powers personalized collections, agentic personalization, and feeds into downstream retrieval and ranking models as features.

The steerable content primitive. All four primitives feed into an offline batch process that generates personalized collections on store pages. Using consumer memory and semantic IDs, an LLM synthesizes collection titles, subtitles, and item lists. Because this runs offline, DoorDash still uses its existing retrieval and ranking stack for item hydration and collection ranking at serving time. The result is steerable by occasion and consumer context—plant-based pantry rows for vegan shoppers, cat dry food rows for cat-only households.