Inference costs are crashing — don't let bad math scare you off

The viral claim. A tweet claimed Lauren (Cursor/SpaceX AI) wastes $132M/year on AI tokens, based on her public Cursor profile showing 1.5 trillion tokens in 30 days and an assumed $8 per million tokens average. Many shared it as proof that agentic coding is absurdly expensive. Theo tears this apart by showing the math is off by over 20x.

How token pricing works. The $8 average is wrong because it ignores cache tokens. For models like Opus 5.5, cached reads cost $0.20 per million tokens, not $5. Cache hits often make up half or more of total usage. For Lauren, who likely uses Grok via SpaceX AI, the blended rate is probably under $0.20 per million, not $8. That alone drops the annual figure from $132M to under $3M, and likely under $1M.

What the spend buys. That $1M or less covers not just inference but product R&D, marketing, and engineering output. Lauren files thousands of PRs monthly, improves Cursor and Grokbot, and generates massive Twitter attention. Theo estimates $25k–$100k/month, which is less than a single senior engineer’s salary and far cheaper than equivalent marketing spend. From a product and engineering ROI lens, it’s easily justifiable.

Inference cost trends. Theo presents data from Artificial Analysis showing that for a given intelligence level, cost has dropped ~10x in a year and recently ~100x in 6–8 months. For example, O1 (Dec 2024) cost $0.12 per skate-bench task; Gemini 3 Flash (Dec 2025) cost $0.002 per task — a 58x decrease. GLM-53 Flash is 680x cheaper than O1 for comparable accuracy. The trend is accelerating, not slowing.

The counterargument. Detractors argue benchmarks oversell real-world gains and that old models get retired. Theo responds that models like GPT-4o cannot even call tools properly today. The real-world experience gap is even larger than benchmarks suggest. The only reason a task costs the same as a year ago is that we now ask agents to do far more work, replacing steps we used to do manually.

The organizational lesson. The takeaway for engineering leaders is twofold. First, token pricing is opaque; never use list price or average cost — demand cache hit rates and model-specific deals from your vendor. Second, budget for inference as a rapidly declining cost; what seems expensive today will be trivial in 12 months. The real risk is underinvesting in experimentation while competitors who ignore short-term cost barriers build the next generation of AI workflows.