Voice agent evals: from 33% agreement to 92% in regulated fintech

The problem. Upstart handles thousands of regulated lending calls daily. A voice agent automates verification calls for the 10% of applicants not instantly approved. The agent asks three questions about income, fund use, and obligations. But regulatory constraints make even this simple flow complex: if a borrower mentions medical use, the agent cannot ask for an amount. If they mention travel, the agent must clarify whose graduation it is. Compliance violations are not just poor UX but potential regulatory issues.

Manual chaos. Upstart started with a Google Sheet to track human reviews. Within weeks, five columns became ten, then fifteen people added columns randomly. The process became unmanageable. Teams needed structured data to improve both the agent and the review process itself. This drove them to adopt Coval for structured human review and evaluation infrastructure.

Metric evolution. Shiv's team initially tracked many metrics: call time, accuracy, clarity, completeness, policy alignment, escalation correctness, CSAT. Over time they whittled down to what mattered most for their use case. His advice: don't get attached to metrics. Start with what seems logical, then discard what doesn't drive real outcomes.

The agreement gap. Upstart ran an LLM judge on every call to score compliance adherence. Human reviewers then evaluated the judge's output. Initially, human and LLM agreement was only 33%. Three causes emerged: tribal knowledge not encoded in the agent, disagreement among human reviewers themselves, and judge quality issues. Each was addressed systematically, bringing agreement to 92%.

The continuous loop. Brooke described a complete evaluation cycle: define metrics aligned to product definition, monitor live calls at scale, detect known failures and unknown unknowns via clustering, route failures to the right teams (compliance, engineering, agent), re-simulate fixes before production, and ultimately enable a self-healing loop gated by human approval. Every change is behind a feature flag; compliance changes require compliance team sign-off.

Why this matters for regulated AI. The core insight is that evaluation infrastructure is not a nice-to-have but a regulatory requirement when deploying autonomous voice agents at scale. The loop is not about automation replacing humans but about humans focusing on the calls that matter most. Upstart's blueprint shows how to launch AI safely in regulated environments while continuously improving performance.