Engineering brief

AI's messiest test: insurance claims buried in faxes and 8GB PDFs

AI Engineer3 min read · saves 42 min

At a glance

The brief is enough

Relevance
Practical value
Radar
Time horizon
This year
Warnings
  • Vendor pitch

Claims files arrive as faxes, PDFs, and duplicate records, not clean data. AI can extract insights only after real digitization and human-in-loop verification.

Messy enterprise data, not model capability, decides whether AI document processing delivers value.

Watch if you're evaluating AI document processing for regulated, document-heavy workloads like claims.

Summary

The core argument. Denys Linkov, who leads the AI program at Wisedocs, makes a familiar point with an unusually concrete backdrop: AI in insurance is not a model problem. Claims arrive as paper, faxes, and enormous PDFs, often before any data pipeline touches them. He cites a fax market above 7 billion dollars and EHR adoption that only reached 80-90% during COVID. The consequence is that claims processing still depends on manual extraction and human review.

The mess. A single workers' compensation claim can include intake forms from Texas, New York, and California that ask for different fields, doctor notes that contradict each other, missing X-ray reports, and duplicate records that run as high as 50% in some jurisdictions. Linkov describes multi-gigabyte PDFs, one 8GB file in production, and adjusters juggling 200 claims. Each handoff adds delay; the best claims close in weeks, the worst drag for years. That is the loss-ratio problem.

The benchmark. Wisedocs built MLCR with Artificial Analysis to test models on medical claims. Initial accuracy was around 20%; months later frontier models reached about 75% on harder reasoning tiers. The benchmark also measures conciseness and completeness separately, which matters because Linkov reports GPT models score well on accuracy but omit information, while Anthropic models are more complete with slightly lower accuracy. The lesson for teams: do not evaluate on one metric, and do not trust a vendor's single number.

The workflow. Linkov's proposed architecture is layered: OCR and preprocessing, classifiers and extraction models, summarization, human verification, then cross-claim analysis. Human-in-the-loop review is non-negotiable, both for regulatory comfort and because bad upstream data invalidates downstream insights. The more interesting claims are portfolio-level: provider networks, co-occurring diagnoses, treatment cost variation, and year-over-year price increases. None of that is possible until one claim is cleanly extracted.

The trade-offs. The vendor position is visible. Wisedocs sells exactly this pipeline, and the benchmark is self-reported. Linkov is honest about limitations: no industry-wide proof of reduced claim time, and case studies instead of hard numbers. The counterintuitive signal is that plaintiff lawyers adopted AI faster than insurers because their incentive is speed and settlement size. That asymmetry, combined with fraud and duplicate submissions, means slow digital transformation makes insurers more vulnerable, not just less efficient.

My take

The most useful part of this talk is the reminder that clean data and evals determine whether AI adds value. I'd be careful treating the benchmark as neutral: it is a vendor's own test, and the jump from 20% to 75% accuracy may say as much about question design as model progress. Still, the operational picture matches what I see in enterprise AI: most failures come from duplicate records, missing attachments, and undefined answers, not from model capability. The practical move is to start with data quality and human verification, then ask portfolio questions once extraction is trustworthy.

What to do with this

  1. Audit one claim pipeline for duplicate records, missing attachments, and nonstandard forms before choosing an AI vendor.
  2. Run a small benchmark on your own claims with human-verified answers; measure accuracy and completeness separately.
  3. Ask vendors how they handle 8GB PDFs and conflicting diagnoses, and confirm human-in-loop review sits before decisions.

Watch the video

This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.

Related breakdowns

Get TL;DW

Too Long; Didn't Watch.

A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.

Free. Weekly. No hype.

Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.