Inference cost and agent regression are the new constraints
Inference costs 12x training and agent changes silently degrade performance.
Teams need eval loops and cost-per-insight measurement, not just better models.
Inference as COGS. Two teams report inference spending dominating total AI costs.
One cites 12x training flops, the other tracks cost per insight and maintenance overhead.
Budget for inference optimization and eval infrastructure alongside model access.
Beyond coding. Three teams independently identify problem discovery and workflow measurement as the key constraint.
Engineers now spend more time on specification and testing than on coding.
Hire for domain understanding, not just typing speed.
Invisible regressions. A one-line prompt change can drop task completion 17 points while safety stays flat.
Production traces and automated evals catch what code review misses.
Treat every production trace as a potential regression test.
My take
I think the central lesson is that AI adoption is now an operations problem.
Inference cost and agent regression will dominate your time and budget.
The hedge fund's AB testing framework and the regression loop are the right instincts.
But start with one workflow, not an entire platform.
The real value is in the discipline of asking 'what changed'.
The videos behind this brief
We watched these so you don't have to.
Inference costs 12x training: the new COGS problem for AI teamsY Combinator
How a Hedge Fund Measures AI ROI Without Waiting for P&LAI Engineer
The bottleneck is not code. It never was.AI Engineer
Bridgewater's data platform: humans need precision, agents need scale.AI Engineer
Your Agent Fix Just Broke Something Else – Agent Regression TestingAI EngineerVideo and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.