Engineering brief

Verification, not code review, unlocks trusted agent scaling.

AI Engineer3 min read · saves 58 min

At a glance

Worth watching

Relevance
Practical value
Radar
Time horizon
Now
Warnings
None

Agents can claim done, but without proof they're just confabulating. WorkOS engineers show how deterministic CLIs and system-one judges give agents eyes, enabling 900 PRs/month with no incidents.

Without agent self-verification, scaling AI coding is unsafe and trust remains low; this is the missing layer.

Watch if you're leading a team that heavily uses coding agents and wants to move from prompt-and-pray to verifiable delivery.

Summary

The core problem. Nick Nisi and Zack Proser from WorkOS open with a worst-case scenario: an agent that deleted production databases in nine seconds. Their point is not that agents are dangerous, but that teams cannot trust agents until they can prove their own work. The traditional signal, reading code and reviewing PRs, no longer scales. Nisi admits he hasn't written code since July 30th, 2025, and relies on agents for everything.

The verification gap. The pair contrast how a human developer verifies a web app: refresh, check network, look for console errors, test persistence. When an agent is asked to "execute these instructions," it has no inherent way to confirm the result actually works. They demonstrate a simple waitlist app where the agent reports success, but after reload the email is gone because the persistence step was missed.

The solution. The answer is to give agents deterministic tools that can prove correctness. They built "snack," a CLI that brings the app up, runs checks like `doctor` for health, `drive` for interaction, `snapshot` for state, and `prove` for end-to-end assertions. A "judge" mode uses a system-one model to fuzzily evaluate whether the outcome matches expectations when exact text matching is too brittle. This gives agents "eyes" and closes the feedback loop: the agent runs the verification, sees the failure, fixes, and re-runs.

Organizational payoff. The real value is that agents can self-heal and work asynchronously. When a failure occurs, the agent doesn't need a human to nudge it; it reads the error, adjusts, and retries. At WorkOS, they cite colleagues shipping 900 PRs a month without incidents. The key is that each PR is backed by a verifiable receipt: screenshots, logs, and test results. This builds trust over time and lets humans focus on outcomes, not line-by-line review.

What's missing. The workshop is an opinionated playbook, not a proven framework. The demos are contrived, and the evidence is anecdotal. The claim that self-evolving codebases are around the corner is speculative. Building a verification CLI for every application is a significant investment, and the talk doesn't quantify the cost. It also assumes agents will reliably invoke these tools, which is not guaranteed. Teams will need to enforce verification through CI or agent instructions, but that's still an open area.

Takeaway. The message is clear: verification is the foundation for agent trust. Start small, build a simple CLI that proves the most important user journeys, and integrate it into your agent workflow. That is the highest-leverage move for any team scaling agent usage. The code itself matters less than the ability to prove it works.

My take

The core argument is correct: verification is what makes agents trustworthy. But building a bespoke CLI for every app is heavy. I'd rather standardize a few generic verifiers and let agents run them. The 'receipts' idea is good, but it can become ceremony. The real win is giving agents a tight feedback loop. I'd start with one critical user journey and get it deterministic. The rest can follow. Also, I'm skeptical of the 900 PRs claim without more data. Still, the principle is sound and worth adopting early.

What to do with this

  1. Identify your top user journey and build a CLI that deterministically verifies it end-to-end.
  2. Add an AGENTS.md instructing agents to run the verification CLI before opening PRs.
  3. Capture screenshots, console logs, and network traces as receipts for each PR review.

Watch the video

This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.

Related breakdowns

Get TL;DW

Too Long; Didn't Watch.

A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.

Free. Weekly. No hype.

Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.