Engineering brief
CoreWeave's Forge signals agent production is now an operations problem
At a glance
The brief is enough
- Relevance
- Practical value
- Radar
- Time horizon
- This year
- Warnings
- High hype
- Vendor pitch
The real signal from CoreWeave's keynote: agents are an operations problem, not a model problem. The useful content is Capital One's evaluation-first playbook and the observability demo, not the partner benchmarks.
GPU clouds are moving up the stack; agent production now depends on observability, evaluation, and governance, not raw compute.
Watch if you are choosing an AI cloud or platform for agent workloads and care about evaluation, observability, and post-training.
Summary
The proposal. CoreWeave used its Fully Connected keynote to launch CoreWeave Forge, a platform built around a five-step AI loop: run, observe, curate, improve, evaluate. Forge combines inference, observability, distillation, RL rollouts, and evaluation in one environment. The bigger story is strategic: CoreWeave is no longer just selling GPU access. It is moving up the stack to own the workflow around agents, the same way cloud providers moved from raw compute to managed services.
The loop. The framing is useful even if the product is not. Agent workloads are not chat workloads. CoreWeave cites an industry benchmark where agents used 142 times as many input tokens and over 20 times as many turns as a chat interaction. That changes cost, latency, and failure modes. Teams racing to ship agents need a feedback loop that catches bad behavior in production and turns it into tests, evals, and better models. That is the signal worth keeping.
The evidence. The strongest demo was Mission Control, CoreWeave's operational intelligence layer. It detected KV cache thrashing, applied a model-level fix, then found that two replicas could share one GPU domain. Throughput went from 3,000 to 5,000 to 20,000 tokens per minute per GPU, with spend flat. That is an impressive result, but it is a vendor demo. No independent methodology, and the 2x price-per-token efficiency claim should be treated as directional, not contractual.
The practice. Capital One gave the most transferable content. Its evaluation-first architecture combines deterministic code checks for schema and policy with LLM-as-a-judge for open-ended reasoning, calibrated by humans. It described concrete agent failure modes: context and memory drift, agents falsely claiming they transferred funds without calling APIs, and early errors compounding into cost explosions. This is the real playbook for production agents: layered grading, golden datasets, and governance embedded in CI/CD.
The trade-off. CoreWeave insists connected does not mean closed. You can still bring your own models, frameworks, and clouds, and it launched a partner network with search providers Exa, Parallel, and you.com. But every integration runs through CoreWeave's platform, billing, and telemetry. The more of your workflow lives in Forge, the harder it is to leave. That is normal for a cloud platform, but engineering leaders should treat the platform as a dependency, not a neutral tool.
What to watch. The keynote leaned on frontier customers and benchmarks. Cognition, the first Vera Rubin NVL72 customer, claims 4.8x throughput on inference and 3.8x on RL rollouts versus GB200. Caterpillar says physical AI will reach unstructured job sites soon. Useful, but all are partner claims with no reproducible data. What matters for most teams is the loop itself: run, observe, curate, improve, evaluate. Build that discipline before adopting a vendor's version.
My take
I think the CoreWeave keynote is worth mining for one idea: the agent lifecycle is a continuous loop. The vendor framing is predictable, and the benchmarks should be ignored until someone reproduces them. But the Capital One segment describes something real. Production agents fail in ways chat apps never did, and evaluation has to be layered, continuous, and embedded in the developer workflow. My pushback is on where the loop lives. CoreWeave wants you to run it on its platform, with its telemetry and billing. I would keep the loop in your own stack first: your own traces, your own golden datasets, your own eval gates. A vendor can help, but it should not become the memory of your production system. Switching later gets painful.
What to do with this
- Map your agent workflow as a run-observe-curate-improve-evaluate loop and identify which stage lacks tooling.
- Create a golden evaluation dataset and two-layer grading (deterministic checks plus an LLM judge) before scaling agents.
- Ask your AI platform provider for failure clustering, lineage, and cost-per-request metrics, not just throughput benchmarks.
Watch the video
This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.
Related breakdowns
CoreWeave Forge aims to turn agent experiments into governed production loops
CoreWeave Forge bundles agent observability, distillation, RL rollouts. Vera Rubin claims need proof. Agent operations are the new cloud battleground.
Documents, Not Models, Are the Real Agent Bottleneck
PDFs are display files, not machine-readable data. LlamaIndex's CEO explains why document parsing is now the cost and accuracy bottleneck for agent automation.
AI agents need identity-first security, not better sandboxes
Six real AI agent attacks and what would have stopped them. The surprising priority is identity management, not sandboxing.
Get TL;DW
Too Long; Didn't Watch.
A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.
Free. Weekly. No hype.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.