Digital twins are 85% accurate—but only if you collect the right data

Joon Sung Park argues that the killer application for foundation models is human simulation, not personal assistants. His team achieved 85% accuracy in predicting how individuals behave in surveys and experiments, by combining in-depth interviews with behavioral data instead of relying only on web text. Frontier models alone scored 20-60% on niche populations.

The key claim is that current LLMs are super-rational and lack the messy, biased, and inefficient traits that make humans predictable. Simile AI collects data through consent-based interviews, transaction logs, and randomized control trials—what Park calls "dark knowledge"—to capture causal mechanisms rather than surface attitudes.

The tradeoff is significant: this approach requires bespoke data collection for each population, unlike combinatorial persona generation that scales cheaply but poorly. The vision is multi-agent simulations tackling wicked problems like climate change or democracy collapse, but current deployments focus on replacing human panels for market research and concept testing.

What's missing is independent validation of the 85% claim outside their own study, and clarity on how behavioral accuracy degrades when simulations interact in multi-agent settings. The cost argument hinges on saving expensive real-world experiments, not competing with cheap inference.