Engineering brief

AI adoption is blocked by risk, not capability. Standards and insurance are

Latent Space3 min read · saves 84 min

At a glance

Niche watch

Relevance
Practical value
Radar
Time horizon
This year
Warnings
None

Model capability is no longer the bottleneck. Enterprises can't deploy AI because they can't make promises about what it will do.

AI adoption is bottlenecked by trust, not technology. Standards and insurance are becoming the critical infrastructure.

Watch if you are a security leader or platform engineer responsible for deploying AI agents in regulated enterprises.

Summary

The thesis. AI adoption is not held back by model capability. It is held back by risk. As Rune Kvist, founder of AI Underwriting Company (AIU), explains, the binding constraint on deploying frontier AI in enterprises, banks, hospitals, and governments is the inability to make credible promises about what these systems will and will not do. Waymo illustrated this years ago: superhuman driving skill meant nothing without a trust framework that allowed regulators and insurers to say yes.

The blueprint. Kvist draws a direct line from electricity, cars, and nuclear energy to AI. Every transformative technology needed standards and insurance before mass adoption. Standards define the rules of the road and the tests that quantify risk. Insurers then price that risk and put capital behind it, creating a profit incentive to measure truthfully and reduce harm. AIU's AU1 standard for agent security, safety, and reliability is the first commercial attempt to build this layer for AI agents.

The standard. AU1 covers six risk categories, from hallucinations to jailbreaks to data leakage. It is updated quarterly through a consortium of Fortune 1000 risk leaders. Companies like Cursor, Harvey, Lovable, and 11 Labs have gone through the certification process, which includes third-party technical testing, policy controls, and independent audits by firms like KPMG. The output is a 100-page report that enterprises can trust because it comes from a neutral party, not from the vendor themselves.

The insurance link. AIU does not just certify; it underwrites. By partnering with Lloyd's of London, it turns a passing certification into an actual insurance policy, as it did for 11 Labs. This is the mechanism that creates trust: a conservative, centuries-old institution has looked at the data and is willing to put its balance sheet behind it.

The model layer. Beyond agents, Kvist sees the same gap opening at the model level, especially after the Fable and Mythos incidents. Governments need a neutral third party to run frontier technical evals and produce consistent, legible reports, just as Moody's does for bonds. AIU plans to expand from agent certification into model certification, and eventually into physical AI and robotics, where the liability stakes are even higher.

The tradeoff. A for-profit standard setter creates obvious tension: the incentive to hollow out standards for competitive advantage. Kvist's answer is that insurers act as a counterweight because they bear the financial consequences of weak standards. Whether that holds in practice remains unproven. There have been no claims yet, and the first major incident will test whether the framework is as robust as advertised. The skepticism is warranted, but the problem AIU is solving is real and growing fast.

My take

I buy the problem: enterprises are blocked by risk, not capability. AIU's approach is historically sound, standards and insurance have unlocked every major technology wave. The execution is impressive, with real customers like Cursor and Harvey, and a credible insurer partner in Lloyd's. What I am not yet convinced of is the for-profit model. The Moody's comparison is apt, but Moody's also failed spectacularly during the financial crisis. Credit ratings lost trust precisely because the rater was paid by the rated. AIU's hedge is the insurer incentive, but it is untested. No claims have been filed yet. The first lawsuit will be the real exam. Until then, I would treat AU1 certification as a useful signal, not a guarantee.

What to do with this

  1. Map your AI agent deployment pipeline against AU1's six risk categories to identify gaps.
  2. Ask your security team: would we pass a third-party jailbreak and hallucination audit right now?
  3. Evaluate whether a standard like AU1 could replace the ad-hoc security questionnaires your enterprise customers send.

Watch the video

This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.

Related breakdowns

Get TL;DW

Too Long; Didn't Watch.

A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.

Free. Weekly. No hype.

Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.