IBM's GB200 rental shows security, not speed, decides GPU cloud success

The decision. IBM Research had just delivered its own H100 cluster, with all the firmware, power, and data center pain that involved. For the next generation it chose to rent GB200s from CoreWeave instead of building again. The deployment grew to 39 racks and more than 4,000 GPUs, and IBM ran MLPerf with Nvidia before production training. That makes this a signal worth reading: a conservative enterprise research lab treated rented GPU infrastructure as primary capacity, not spot overflow.

The security stack. Model speed was not the bottleneck. IBM built a global UID and group namespace, used Red Hat Directory Server as the identity source of truth, rolled out short-lived certificates with step-ca, and tied budgets and groups to LDAP. That creates one choke point when someone leaves, and it satisfies auditors. The key design choice is federation: CoreWeave is not the system of record, IBM keeps the control.

The trade-offs. Renting did not remove operational risk; it moved it. IBM brought its own VAST storage and GPFS experience, coordinated kernel versions with CoreWeave's agents, and took on storage risk that other tenants avoid. The cluster has four networks, controlled ingress, and restrictive egress. The relationship is closer to a negotiated partnership than a self-service cloud. The speaker compares this with another cloud where a YAML limitation was undocumented and cost a week of support tickets.

The agent era. IBM's first security incident came from an eval that injected commands into a user's bashrc; a simple ls deleted a directory. Then coding agents began scanning petabytes of storage, and RL training added inference and execution workloads that are sometimes deliberately malicious. The team now needs sandboxed container execution, placement constraints on NVLink domains, and new monitoring rules. That is a meaningful change for any group running training workloads, not just a niche security problem.

The weak evidence. This is a vendor-customer session, and CoreWeave's product lead frames everything as a trust and product story. There are no numbers on cost, no independent benchmarks beyond a 91% scaling result and a 27-minute MLPerf run, and no details on how the security incidents were resolved. Confidential computing on GPUs, telemetry relay, and the sandbox product are roadmaps with IBM as a design partner. Treat product claims as directionally useful, not proven.

The planning lesson. The most useful line is at the end: a new workload needed weeks of rework because its security profile was deliberately not designed for. Teams adopting rented GPU capacity should decide identity, storage, egress, and agent execution models before the contract is signed. Vendor relationships, not just SLAs, decide whether first-generation hardware is usable. The rest is a well-produced case study.