Why Can't Anyone Answer Questions About the Business? — Garrett Galow, WorkOS

WorkOS built an internal agentic workspace (“Studio”) that lets non‑engineers ask business questions and have the system generate reusable widgets (code-backed UIs) that query Snowflake, Linear, Notion, etc. Crucially, the LLM is used once to design the widget; ongoing usage directly hits data sources without invoking the model. This shifts many BI/support tickets to self‑serve tools.

The technical pattern is notable: no RAG. Instead, constrained agents, strict sequencing (preflight checks), and contextual tool hints at call time (e.g., how to join messy Snowflake tables) drive reliability. They layer prompts (base + org + tool-specific rules), explicitly distrust the model’s product knowledge, and validate queries return data before codifying them.

Impact: GTM and support teams reduce Slack/Snowflake dependency on engineers; data/platform teams stop building endless one‑off dashboards. The “widgetization” makes outputs governable artifacts, not ephemeral chats. It also controls LLM spend by front‑loading generation cost and eliminating per‑use inference.

Tradeoffs: Governance is immature. Permissioning is currently user-based with org connectors “in progress.” There’s a real “query-as-truth” risk unless you add review, status filters, and lineage. Data semantics live in prompt context, not a canonical model—maintain these or drift will bite. Costs are hand‑waved as acceptable; Snowflake compute and Opus prompts still add up.

What most will miss: the winning move isn’t cleaner data or bigger dashboards; it’s precise runtime context injection and validation to make agents trustworthy, plus converting agent output into code you can review, version, and reuse.