The cost of running agents now matters more than the model
Inference is getting cheap enough that the VM running your agents is the real line item.
Policy documents don't stop a misconfigured agent; gateways and lightweight runtimes do.
Compute costs. Inference costs drop 10x per year, but the VMs running agents aren't getting cheaper.
Teams should experiment with lightweight runtimes like V8 isolates for agent execution before the compute tax dominates budgets.
Governance gap. 88% of orgs have an agent in production, but only 14% have governance.
Agents follow tool permissions, not written policies. Gateway enforcement with rate limiting and identity scoping is the real control point.
Avoid lock-in. Vertical integration like Copilot locks teams into one vendor.
Decoupling LLM, data, interface, and sandbox layers preserves flexibility, but an AI gateway adds its own dependency.
My take
I think the cost trajectory Theo describes is the most actionable signal today, even if the V8 isolate demo is a stretch.
The governance debate matters, but it's secondary: you can't gatekeep an agent you can't afford to run.
Start by understanding your agent compute costs per task, then layer permission scoping on the existing API gateway before buying a dedicated AI gateway.
The decoupling principle from video 1184 is sound, but don't let perfect flexibility block shipping your first useful agent.
The videos behind this brief
We watched these so you don't have to.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.