Edge AI's dirty secret: DRAM cost, not model quality, is the bottleneck

Cormac Brick argues that the industry's focus on billion-parameter models overlooks a critical bottleneck: DRAM cost. Deploying a 2B parameter model requires roughly 4GB of RAM, pricing out many consumer IoT, robotics, and lower-end devices. The real opportunity, he suggests, lies in models under 500M parameters that can run on hardware with under 2GB of

memory. The playbook for these 'tiny models' is fundamentally different. Instead of zero-shot prompting, teams must invest in synthetic data generation and fine-tuning for specific tasks. Brick shows a voice-to-function-calling model, fine-tuned from a 270M parameter model, achieving over 86% reliability. This tradeoff—losing generality for speed, reach, and cost-efficiency—is the core operational decision. This shift

has direct organizational consequences. It moves AI deployment from a cloud-cost problem to a hardware-BOM and data-engineering problem. Teams building edge or IoT products can no longer rely solely on prompt engineering; they need capability in synthetic data pipelines and model distillation. The Chrome summarization API example underscores that even browser-based features depend on this

approach. Brick's evidence is strong, grounded in shipping production code at Google. The hype lies in assuming all problems need large models. Most teams will miss that the constraint is not model performance but hardware economics. The actionable insight: for specific, high-reliability tasks on constrained devices, fine-tuning a tiny model on a synthetic dataset (10K-10M