Your AI Governance Policy Is Useless. Here's What Works.

The claim. Sam, CTO of Gravitee, argues that AI agents will ignore written policies and must be governed at the infrastructure layer, specifically through an API gateway. He demonstrates two agents side by side: one with prompt-based controls, one governed by Gravitee's gateway. The ungoverned agent deletes a database on request; the governed one returns a 403.

Why it matters. 88% of organizations have an agent in production, but only 14% have any governance. Most teams rely on policies written in markdown or system prompts. Sam shows that agents are inherently compliant—they want to fulfill requests—so they will execute dangerous actions if tools are available. The real control point is not the prompt but identity and permission scoping at the gateway.

The cost angle. Beyond security, the gateway enforces rate limiting, token capping, and semantic caching. Sam demonstrates a buggy agent sending the same request eight times: the ungoverned agent spends tokens on all eight, while the gateway blocks after three. Semantic caching reduces costs by serving similar prompts from a Redis store. These operational concerns become significant at scale.

Context management. Large prompts (10 minutes of transcript) are blocked at the gateway before reaching the LLM, preventing wasted compute. Sam also shows content filtering to reject irrelevant queries like weather checks. This suggests that teams should design agents with just enough context, enforced by gateway policies.

The vendor context. This is a live demo of Gravitee's platform, which includes MCP studio for tool scoping, LLM routing across providers, and edge management for shadow AI traffic. The speaker is the CTO selling the product. The claims are plausible but unvalidated by independent production data.

The tradeoff. Shifting governance to a gateway adds a dependency on an API management layer and may increase latency for simple queries. Teams with few agents might over-engineer. But for organizations scaling to hundreds of agents, the gateway approach beats hardcoding controls into each agent's prompt. Start with identity and permission scoping, not caching or routing.