Stop Trusting Chat; Engineer Your AI Financial Analyst

The claim. The speakers argue that the bottleneck in AI-assisted financial analysis is not speed but trust. They propose five design principles for building a harness that earns user confidence: shared domain, consistency, evidence, accountability, and verification. The principles are demonstrated through a wedding budgeting example.

The problem. Chat-based agents produce non-deterministic results, especially for arithmetic. Asking an LLM to calculate costs repeatedly yields different answers. The speakers contrast this with a harness that enforces deterministic code for calculations, versioned and tested, so every run returns the same result.

The five principles. Shared domain means the agent and user align on definitions and context before executing. Consistency means any repeatable calculation runs as code, not LLM tokens. Evidence means every number is traceable to its source (fact, assumption, or interpretation). Accountability means the human remains responsible, so the harness should have high-signal gates rather than per-action approvals. Verification means a second agent or external reference challenges the plan.

The trade-off. Building such a harness takes upfront effort. The speakers acknowledge that for ad-hoc or low-stakes tasks, simply using a coding agent with good prompts is sufficient. The harness pays off when reproducibility and trust are critical—like financial decisions where a 1% error has large consequences.

What's missing. The talk focuses on principles, not production tooling. The demo harness is deliberately rough. Questions about how this scales across many use cases or how to manage expectation with leadership are only briefly addressed, with a teaser for a later session.

The bottom line. Trust in AI financial analysis must be designed in, not assumed. Teams should evaluate whether their use case warrants the investment in a purpose-built harness or if existing agent tooling suffices.