AssemblyAI's Joey: Building vs buying an AI support agent

The signal. AssemblyAI's forward deployed engineer Matt Lawler describes building an AI support agent named Joey that now handles 80% of inbound support tickets end-to-end, up from a 10% resolution rate with an off-the-shelf bot. The team operates at a thousand API signups per day with a single onboarding engineer, a constraint that forced automation. The result is not theoretical: Joey runs in production, costs $700 a month, and has replaced the first line of human support entirely.

What changed. Instead of buying a vendor bot that offered no control over prompts, tools, or retrieval, AssemblyAI built its own using the Cloud Agent SDK. Joey has local access to all documentation as markdown files, uses Voyage embeddings for retrieval, and lives on Railway for instant deploys. The team can fix bugs mid-conversation without the customer noticing. The shift from vendor dependency to in-house control is the operational lesson that matters here, not the 80% headline.

The tradeoff. Building your own agent trades upfront engineering time for flexibility and iteration speed. AssemblyAI spent the time because the vendor could not meet their needs and the volume justified the investment. For most teams, the decision hinges on how much control you need over system prompts, retrieval pipelines, and escalation logic. If your support volume or compliance requirements are modest, an off-the-shelf bot might still be the right call.

What's missing. The talk does not address how Joey handles complex multi-turn conversations, escalations, or edge cases where the documentation is ambiguous. The 80% resolution rate is impressive, but it masks the 20% that still require human judgment. The speaker acknowledges that some of those escalations will eventually be automated, which suggests the boundary between autonomous agent and human is still shifting.

Why it matters for teams. For engineering leaders, the real signal is that an AI support agent can be built with modest infrastructure cost and maintained by a small team. The $700 monthly figure includes both inference and infrastructure, which changes the cost calculus for teams considering automation. The speaker's advice to 'automate yourself out of a job' is a concrete staffing philosophy: FDEs should build systems that scale beyond their own availability.

The bottom line. AssemblyAI's Joey is a production example of building rather than buying an AI support agent, with clear metrics and a replicable architecture. The hype risk is that 80% resolution becomes a generic claim without context on ticket complexity. Engineering teams should evaluate whether their support volume and compliance needs justify building their own agent, or whether a vendor solution with better control than the one AssemblyAI tried first could suffice.