NVIDIA Vera Rubin Crushes Records With Explosive 72-GPU Rack
NVIDIA confirmed on July 21 that its Vera Rubin GPU platform is now in gigascale production, with 72-GPU racks running live at cloud partners including CoreWeave and Google Cloud.
The company’s developer post published this hour describes a 40x increase in AI token throughput over prior generations and a 14.4 petaflop peak compute figure per Rubin GPU.
Also Read: ChatGPT Small Business Program Reaches Explosive 10 Million Users
The ramp marks the first time NVIDIA has shipped a combined CPU-GPU platform at this scale, and signals a direct attempt to own every computing layer inside AI data centers.
NVIDIA (NVDA) has been building toward this moment through successive architecture generations, and the developer post offers the most detailed public breakdown yet of what makes the Vera Rubin architecture a meaningful departure from prior designs. The platform’s arrival at CoreWeave and Google Cloud also validates NVIDIA’s strategy of treating cloud partners as the primary launch vehicle for new silicon generations.
NVIDIA Vera Rubin Packs 72 Explosive GPUs Into a Single Rack
The NVIDIA Vera Rubin NVL72 configuration places 72 Rubin GPUs and 36 Vera CPUs inside one rack unit, connected over NVIDIA’s fifth-generation NVLink fabric.
NVLink is a high-bandwidth interconnect that lets chips share memory and pass data between themselves far faster than a standard PCIe bus can. In the NVL72 configuration, that interconnect runs at 1.8 terabytes per second of bidirectional bandwidth across the full rack.
Each Rubin GPU carries HBM4 memory, the latest generation of high-bandwidth memory stacked directly on the processor die.
HBM4 is relevant because the single biggest constraint on AI inference speed is how fast a chip can read model weights from memory. Rubin’s HBM4 stack delivers more than 8 terabytes per second of memory bandwidth per GPU, roughly double what its predecessor the H100 could achieve.
The Vera CPU alongside the GPU is equally significant.
Previous NVIDIA racks paired its GPUs with third-party processors from AMD or Intel for host-side tasks. Vera is NVIDIA’s own ARM-based CPU, purpose-built to feed the NVIDIA Vera Rubin GPU pipeline.
The combination means NVIDIA now controls both the compute and the host layer, reducing the bottlenecks that emerge when heterogeneous chips from different vendors have to synchronize.
NVIDIA Vera Rubin and the Shift to Agentic AI Infrastructure
The platform’s timing tracks with a shift in how AI workloads are structured. Training large language models was the dominant use case for GPU clusters through 2023 and 2024.
That work is computationally intensive but episodic: a model trains, and the cluster sits idle until the next run.
Agentic AI changes the equation. An AI agent, unlike a chatbot responding to a prompt, runs continuously, takes actions over minutes or hours, queries external tools, and loops back to check its own outputs.
That kind of workload requires sustained inference rather than burst training. NVIDIA’s developer post frames NVIDIA Vera Rubin explicitly as an “AI factory” platform, a term the company uses to describe infrastructure optimized for always-on intelligence production rather than intermittent training jobs.
The 40x token throughput claim reflects this design philosophy.
Token throughput measures how many text tokens a system can generate per second across concurrent users. A higher number means more simultaneous AI agent sessions the hardware can sustain without degrading response times.
For hyperscalers running thousands of enterprise agent deployments, that figure directly translates into revenue per rack.
From Graphics Cards to the Dominant AI Stack
NVIDIA’s original product was a discrete graphics card for gaming. The company’s pivot toward parallel computing, formalized with its CUDA software platform in 2006, gave researchers a programmable GPU they could use for scientific workloads.
When deep learning took off after 2012, CUDA was already the dominant programming environment for GPU compute, and NVIDIA’s hardware was the default training substrate.
The H100, launched in 2022 and still the backbone of most hyperscale AI clusters, cemented that position. But it was still a GPU that needed a third-party CPU alongside it.
NVIDIA’s Hopper and then Grace Hopper architectures began combining its own chip designs, but the NVL72’s pairing of Vera and Rubin at gigascale is the fullest expression yet of the company’s ambition to supply the complete stack. The NVIDIA blog published alongside the developer post confirms production racks are already live at CoreWeave and Google Cloud, with additional partners coming online through the second half of this year.
Chinese AI labs building competitive frontier models face a specific constraint here.
U.S. export controls restrict the sale of NVIDIA’s most capable chips to Chinese entities. The NVIDIA Vera Rubin platform, given its performance tier, will almost certainly fall under those controls, widening the hardware gap between U.S.-based and China-based AI developers at the exact moment agentic workloads are scaling fastest.
What Comes After the NVL72 Ramp
NVIDIA has already disclosed a roadmap successor called Feynman, expected in 2027.
The NVIDIA Vera Rubin ramp suggests the company can now sustain roughly annual generational updates, compressing the historical two-year GPU cycle. For customers buying rack infrastructure today, that cadence matters: a NVL72 purchased now will face a more capable successor within 18 months.
The more immediate question is whether the 40x throughput gain translates into proportional cost reduction for inference workloads.
NVIDIA’s developer post emphasizes performance per watt alongside raw throughput, which suggests the company is aware that hyperscalers will benchmark total cost of ownership rather than peak flops. Partners like CoreWeave, which lease GPU capacity to AI developers, will be first to publish real-world numbers, and those figures will shape enterprise procurement decisions through the end of this year.
Read Next: NEAR Protocol Makes a Breakthrough With Quantum-Safe Mainnet Signing
