Agent trust starts with tracing, not with better prompts

The product. MasterClass has moved from video with a side chatbot (Class TA) to an AI-native product, MasterClass Executive, built with Chicago Booth and OpenAI. It centers on an AI teacher that personalizes a business curriculum using user background, strengths, and engagement signals. The pitch is direct: personal tutoring has decades of proven effect, and AI is the first way to scale it.

The architecture. The system is a four-agent pipeline: planner designs curriculum, teacher delivers lessons with pedagogical rules, orchestrator decides when to move on, and memory stores learner background and preferences. Two learners taking the same course get different journeys. That is the intended design, and it creates the core operational problem: when something goes wrong, no two failures look alike.

The tracing layer. Priya's team built the answer on W&B Weave tracing. They record inputs, outputs, prompts, tool calls, and inter-agent reasoning for every session. On top of that they built a trace-search skill with an MCP server, a search agent, and a memory book. The agent forms hypotheses, tests them against trace counts, scans shallow, then deep, and returns a pattern. It is essentially agentic debugging over agent behavior.

The leverage. Once traces are searchable, one complaint becomes a pattern. The team can find other learners who hit the same failure, assemble issue datasets with all the relevant prompts, and replay fixes. They are extending this to autonomous monitoring: a skill connects traces, GitHub, Jira, and community feedback, flags unresolved issues, and creates datasets for humans. That is a credible ops loop for multi-agent systems.

The trade-off. The cost is complexity and tooling maturity. A 20-100 tool-call debugging session is slow and expensive, and the system depends on storing every trace and trusting a meta-agent to find patterns. MasterClass presents no metrics: no precision rates, no time saved, no failure reduction. The evidence is architectural and anecdotal, not measured.

What to ignore. The heroic framing, the Alexander the Great trailer, and the claim that the "holy grail" is near. The useful signal is narrower: instrument the whole agent pipeline before you need to debug it, because the failure modes are emergent. Teams adopting multi-agent systems should copy the tracing layer before they copy the teaching vision.