Engineering brief
Universal data schemas are dead. Build ontologies at decision time.
At a glance
The brief is enough
- Relevance
- Practical value
- Radar
- Time horizon
- This year
- Warnings
- Vendor pitch
A Southbridge pipeline resolved Ohio campaign finance records for $7.95 after a $10,000 projection, by splitting work into decision-specific questions. The catch: 10 days of agent runs, 6 billion tokens, and heavy verification.
Data teams may stop building universal schemas and start investing in decision-time transformation and model-role design.
Watch if you're designing LLM-based data pipelines or entity resolution and want a model-role framework.
Summary
The argument. Data infrastructure was built on the assumption that transformation is expensive. Olickel argues cheap intelligence inverts that cost, so teams should stop chasing a single universal schema. He describes four schemas, source, storage, decision, and model prior, that only align at the moment a decision is known. A customer means something different to billing, support, and compliance, so a schema divorced from a question is fiction.
The case study. Southbridge built an entity resolution pipeline for Ohio campaign finance records. Before a new model class called Jev launched, the work was expected to cost about $10,000. The team split 800,000 rows into 200,000 specialized cases, ran 90,000 calls to Jev, processed roughly 6 billion tokens, and produced ground truth for every donor case in Ohio for $7.95 on an $800 budget. The number is impressive, but it is one vendor example.
The role split. Models play three distinct roles: designing pipelines, working inside them, and hill climbing. Pipeline design wants slower, heavier models that can reason about the problem. In-pipeline workers should be cheap, fast, and narrow. The third category, called burrito surfing, self-optimizes a workload across cost, speed, and accuracy. The useful point is that organizations often compress these roles into one context window, which pushes cost and error in the wrong direction.
The hidden risk. Model priors, the fourth schema, are a growing error source. One pipeline looked for "in-kind" with an underscore while the file used a capital I and hyphen, so 15,658 contributions were miscounted. Another run turned $784,000 of debt into $5.3 million by storing absolute values of negative loan amounts. Larger models defend priors more strongly. Cross-model checks help: Jev and Luna agreed on 194 of 200 families, and every error sat in the disagreements.
The caveats. The work depends on a stable harness, bounded experiments, methodology, and a defined failure surface. The speaker acknowledges models cannot self-optimize natively yet. What is missing from the talk is engineering effort: 10 days of agent runs, more than 4,000 harness executions, and a team that can build and debug these systems. The price tag excludes that labor. Teams should take the mental model and treat the cost number as a bound, not a promise.
The implication. Ontologies are not universal; they are intermediates between intelligence and result, built when needed and reused with receipts. That reframes data platform work: schema migration projects become less valuable than decision-time transformations, verification rigs, and model-role design. Leaders should watch for agent cost, error budgets, and prior failures, not just model capability.
My take
The three-role split is the most useful idea here. Teams often run one expensive model for everything because it is simpler, and that conflates design judgment with execution discipline. The Ohio case is impressive, but I am not convinced the $7.95 outcome is reproducible in a normal data team. The talk hides ten days of agent runs, thousands of harness executions, and engineers who know how to debug model priors. That is real work and real budget. I also think the failure-surface concept deserves more attention than the burrito surfing metaphor. Define what failure looks like before you let an agent optimize anything, and keep a human check on the classes of error models share. The schema argument is strong; the self-optimizing future is not here yet.
What to do with this
- Review one existing data pipeline and classify models by role: design, worker, or self-optimizer.
- Add a failure surface or held-out verification set before expanding any LLM pipeline to production.
- Ask your data team what schema changes a high-stakes decision would require today, and test the cost.
Watch the video
This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.
Related breakdowns
Flywire's Orbit: Workflows, Not Autopilot, Is the Real Agent Pattern
Flywire built Orbit: deterministic AI workflows, LLM steps, guardrails, human review. 11,000 runs, but success isn't quality.
Bounded AI Judgment Is the Compliant Workflow Pattern
Why compliant AI workflows need deterministic control flow and gated model judgment. The open question: can labeled human confirmations really shrink…
Stop Fine-Tuning Your Way Out of Bad Retrieval
Stop fine-tuning to fix retrieval problems. Use prompt for behavior, memory for facts, weights for stable patterns. Here's how to avoid the accidental…
Get TL;DW
Too Long; Didn't Watch.
A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.
Free. Weekly. No hype.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.