80% agent reliability means zero customer trust — here's proof

The benchmark illusion. Most AI teams optimize against static benchmarks, then ship and hope. Felipe Blanes from the Amazon AGI lab calls this the "benchmark illusion" — the gap between what you measure internally and what customers actually experience in production. His team built Nova, a browser-agent service, and learned the hard way that synthetic evals and public benchmarks do not capture the long tail of real user behavior. The only fix is closing the customer feedback loop.

The eval flywheel. The core proposal is a continuous loop with four steps: define success (from the customer's perspective, not your own), capture signals (both instrumentation and direct conversations), diagnose gaps (categorizing them into model issues, engineering bugs, or product positioning problems), and feed those into decisions. Blanes emphasizes that data alone is misleading — you must talk to customers. His team repeated this flywheel many times over the year, and each iteration improved the product in ways benchmarks never would have caught.

The trust cliff. Blanes's most striking operational finding: 80% reliability is functionally zero trust. When you tell a customer the agent is 80% reliable, they hear "I still have to monitor it and handle failures manually, which means more work, not less." The threshold for earning trust is around 90% reliability — below that, the agent creates overhead rather than removing it. Reliability is binary from the customer's perspective: either they hand over the workflow, or they don't.

Transparency beats benchmarks. Counterintuitively, being transparent about product limitations increases trust more than publishing higher benchmark scores. Blanes advises three things to share with every customer: what works and at what reliability level, what is being improved, and — hardest but most important — what is explicitly out of scope. Customers have many options; if you let them discover your agent's limits through failure, they switch products. Honest scoping earns the right to iterate.

Real examples from production. Working with Amazon's Project Kuiper (satellite internet), the team learned to cache agent trajectories instead of running inference every time, reducing cost. With Hertz, they discovered that QA teams included both technical engineers who could write Python and non-technical testers who needed a no-code interface — leading to a separate product experience. With an RPA startup called Sol, they added flexibility to the actuation stack. All three insights came from direct customer engagement, not from synthetic data.