Prompt Caching: The Hidden Cost Lever in AI Agents

Prompt caching is misunderstood. The common mental model—caching LLM outputs like database results—is wrong. What actually happens is input caching: the model's attention mechanism reuses previously computed states. When an agent resends 50k tokens of conversation history with each turn, those first 50k tokens are processed at full price only once. Subsequent reads cost roughly

10% of the original price. Without caching, a 200k-token session on Claude Opus 5 or GPT-5.6 Sol can cost $41. With caching, that drops dramatically—to near zero on providers like DeepSeek. The practical implication for teams running agents is obvious: caching is the difference between viable and bankrupt. But the operational details matter. Cache expiry

varies widely—OpenAI keeps it for an hour, Anthropic for five minutes on the API but an hour with Claude Code, and Hugging Face routing depends on the provider. Every cache miss resets you to full price. This creates an invisible budget risk: a lunch break can cost your team hundreds of dollars if the cache

expires mid-session. Most teams skip the subtle failure mode: dynamic system prompts. Adding a timestamp, current working directory, or dynamically updated tool list to the system prompt invalidates the entire cache on every change. That one small line of metadata can erase 90% of your cost savings. The fix is trivial but rarely taught—keep system