Model choice barely moves output; context, evals, and security do
The model you pick explains almost none of your output.
If you're budgeting for a model upgrade before context, evals, and dependency hygiene, you're paying the wrong bill.
Context beats model. Model choice accounts for under 1% of the variance in merged PRs across 50,000 engineers.
LinkedIn reports 8,000 daily users from MCP playbooks that encode internal knowledge.
The numbers are vendor-tinged, but the direction is consistent: spend on context and workflow before another model.
Evaluation speed. Evaluation speed isn't a nice-to-have.
Air Canada went from one expert grading session a week to dozens a day by planting known defects and grading them in code.
An analysis of 1,368 incidents puts deployment controls and human review closer to harm than model accuracy.
Start with synthetic tests, then keep human checkpoints.
Dependency debt. AI makes finding bugs cheap, but the fixing side can't keep up.
AI-generated apps pull roughly three times more dependencies, and stolen API keys fueled real charges up to $3,000 in the OpenCode and Railway incidents.
Audit dependency count, rotate keys, and decide which packages you can defend before the CRA or an attacker forces the issue.
My take
The model you choose is the least interesting decision you'll make this quarter.
The data behind that is proprietary and squishy, but Air Canada, LinkedIn, and the incident database all point the same way: context, evals, and workflow are where output and risk actually live.
I'd put next quarter's budget into encoded internal knowledge and synthetic test loops, not another frontier model.
The security half is the part I'd act on fastest, because cheap discovery means your dependency debt gets priced by someone else if you don't price it first.
The videos behind this brief
We watched these so you don't have to.
Model Choice Barely Matters: Fix Engineering Culture, Not AI ToolsAI Engineer
Context engineering is the real bottleneck for coding agents at scaleInfoQ
Testing AI Agents: Plant the Defect, Then Find ItAI Engineer
Stop evaluating models. Start evaluating the risk around them.AI Engineer
AI Finds Vulnerabilities Faster Than Open Source Can Fix ThemGOTO Conferences
LLM distillation demand turns API keys into prime targets for attackers.Theo - t3․ggVideo and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.