Vera Rubin NVL72 is the rack-scale baseline. Most of its newest performance case is still NVIDIA's.

Vera Rubin NVL72 is not simply NVIDIA's next GPU server. It is the third generation of a design in which the rack becomes one scale-up computer. That continuity matters because operators are buying more than 72 accelerators: they are buying an interconnect domain, a Linux software stack, a management model, a supply chain, and a path for scaling many racks into one cluster.

What Vera Rubin NVL72 is

The rack contains 72 Rubin GPUs and 36 Vera CPUs, connected inside one NVLink 6 scale-up domain. ConnectX-9 SuperNICs and BlueField-4 DPUs handle networking and infrastructure functions, while Quantum-X800 InfiniBand or Spectrum-X Ethernet can connect racks into a larger cluster. The rack is liquid cooled and built around NVIDIA's third-generation MGX NVL72 mechanical design.

That generational history distinguishes Vera Rubin from a first attempt at rack integration. Grace Blackwell established the NVL72 operating model, and Blackwell Ultra extended it. Rubin retains the 72-GPU domain while changing the GPU, CPU, memory, links, trays, networking, and system software. As a result, customers and system builders can reuse some facility and operational knowledge even though they must still qualify a substantially different computer.

CNBC's inspection provides a useful physical check on NVIDIA's description. Its report counted roughly 1.3 million components in the system and described 18 compute trays with field-replaceable superchips. The modular trays remove more quickly than the soldered assemblies in the previous generation, which should improve serviceability. However, easier replacement does not by itself establish fleet uptime; that requires failure-rate, repair-time, degraded-mode, and software-recovery data from production installations.

Why NVLink remains NVIDIA's architectural center

Each Rubin GPU has 3.6 TB/s of bidirectional NVLink bandwidth, and NVLink switches connect all 72 GPUs with a published aggregate of 260 TB/s. The purpose is to make the rack behave like one very large accelerator for model parallelism. When a model's layers, experts, or state are distributed across devices, that communication fabric can determine whether additional GPUs reduce latency or merely add synchronization overhead.

NVIDIA's advantage is not only the raw link rate. NVLink, NCCL collectives, CUDA libraries, topology-aware scheduling, monitoring, and application frameworks have been developed together over several generations. That integration gives NVIDIA more opportunities to optimize communication automatically. It also creates dependence on NVIDIA's complete platform, because the benefits are hardest to reproduce after replacing one layer with a different vendor's component.

The scale-up fabric stops at the rack boundary. Larger deployments still need a scale-out network, storage, checkpointing, control services, and failure management. NVIDIA specifies 28.8 TB/s of aggregate scale-out bandwidth and offers both InfiniBand and Ethernet paths. Operators should therefore test the rack as part of the intended cluster topology, not only as an isolated NVLink domain.

Memory, inference, and the meaning of efficiency

A Vera Rubin NVL72 rack is specified with 20.7 TB of HBM4, or 288 GB per GPU, at up to 22 TB/s of bandwidth per GPU. Compared with Blackwell, that combination increases the working set that stays near the compute units and increases the rate at which weights and inference state can move. Both changes are especially relevant during long-context inference and mixture-of-experts routing.

NVIDIA says the rack can deliver ten times as many tokens per megawatt and one-tenth the cost per million tokens as GB200 NVL72 on a specified Kimi K2 Thinking workload. It also says mixture-of-experts training can meet a target with one-fourth as many GPUs. Those are more informative claims than peak FLOPS because they name a workload direction and an economic outcome. Yet they remain projected, vendor-selected comparisons whose results are marked subject to change.

CNBC adds an important facility-level qualification: NVIDIA said Vera Rubin will consume about twice the power of Grace Blackwell even while delivering much more work per watt. Efficiency therefore does not mean lower rack power. A datacenter may get better throughput from each megawatt while still needing denser power delivery, more cooling capacity, and a larger electrical commitment for every installed rack.

Does Vera Rubin run Linux?

Yes. NVIDIA's BaseOS and DGX OS documentation is explicit: DGX OS 7 is based on Ubuntu 24.04 and Linux kernel 6.8, with NVIDIA drivers, diagnostics, monitoring, networking, and system-specific configuration. NVIDIA also supports its BaseOS software on Red Hat Enterprise Linux and Rocky Linux through DGX EL. Partner-built MGX and cloud systems can use different qualified images, so the exact distribution depends on the product and operator.

Above Linux, CUDA remains the core programming environment, while NCCL, TensorRT-LLM, NIXL, and other libraries address communication and model execution. NVIDIA Dynamo adds the distributed inference layer: request routing, disaggregated serving, key-value cache management, autoscaling, observability, and fault handling. Its production path is Kubernetes-native and supports vLLM, SGLang, and TensorRT-LLM rather than requiring one inference engine.

This software depth is NVIDIA's most defensible competitive advantage. It reduces the number of components an operator must integrate independently and gives application teams a large pool of existing code and experience. It does not eliminate qualification work. New hardware, Arm-compatible Vera CPUs, kernel and firmware combinations, container versions, and topology-aware scheduling still need to be tested as one supported release train.

What AMD Helios changes

AMD Helios makes the comparison more substantive because it targets the same 72-accelerator rack boundary. Both systems publish 260 TB/s of aggregate scale-up bandwidth and both combine host CPUs, accelerators, DPUs, high-speed networking, liquid cooling, and a Linux software environment. This is no longer a comparison between one NVIDIA rack and a collection of unrelated AMD servers.

Helios has more published accelerator memory, slightly more per-GPU memory bandwidth, and a more open mechanical and fabric story. Vera Rubin has a mature scale-up software path and a broader existing application ecosystem. Those differences suggest workload-dependent outcomes, not a universal ranking. Our separate AMD Helios review examines that system in depth, while the side-by-side rack comparison covers the full decision matrix and explains why direct exaFLOPS comparisons are misleading.

What is established, and what remains NVIDIA's claim

Our take

Vera Rubin is the safer architectural bet because NVIDIA has already taught the market how to operate an NVL72 rack. The company has system vendors, cloud providers, network products, Linux images, libraries, management tools, and trained engineers aligned around the same platform. That accumulated operating environment is more difficult for competitors to match than any single GPU specification.

However, the safest architecture can still be the wrong economic choice for a particular workload. NVIDIA controls more layers of the stack, limits substitution, and prices the value of its software and supply position into the system. AMD, custom accelerators, and cloud silicon now give buyers credible reasons to measure alternatives rather than accept one vendor's cost-per-token chart.

We would treat Vera Rubin NVL72 as the default system to beat in a 2026 rack-scale evaluation. We would not treat it as the automatic winner. The procurement process should replay representative models and quality settings, sweep concurrency, record time to first token and time per output token, measure wall power, induce failures, and count the engineering effort required to sustain the target result. NVIDIA's platform advantage should appear in that evidence rather than be assumed outside it.

Questions a cluster operator should ask

1. Which exact vendor implementation, Linux image, firmware bundle, driver branch, CUDA release, and container set are supported together?
2. What measured throughput-latency curve does our workload achieve, and how does it compare at the same output quality and rack power?
3. How much of the published HBM4 and NVLink bandwidth is usable under our model-parallel topology?
4. What power, coolant, floor-loading, service-clearance, and facility changes are required beyond the previous NVL72 generation?
5. Can a failed GPU, superchip, tray, NVLink switch, NIC, or host be isolated without draining the entire rack?
6. How do Kubernetes and Slurm workflows divide responsibility for serving, training, maintenance, and diagnostics?
7. Which Dynamo features are production supported for our inference engine, and which remain optional or experimental?
8. What is the exit plan for models or services that become more economical on AMD, custom silicon, or a managed cloud endpoint?

Sources and further reading

1. NVIDIA Vera Rubin NVL72 product page — rack components, specifications, performance assumptions, and projection footnotes.
2. AMD Instinct MI455X specifications — current memory specifications used in the Helios comparison.
3. NVIDIA: Vera Rubin production ramp — supply-chain status, manufacturing partners, and fall 2026 shipment target.
4. CNBC: First look at Vera Rubin — physical construction, serviceability, facility power, timing, and analyst context.
5. NVIDIA NVLink and NVLink Switch — scale-up topology, bandwidth, and preliminary NVLink 6 specifications.
6. NVIDIA BaseOS documentation — Ubuntu, Red Hat Enterprise Linux, and Rocky Linux software paths.
7. NVIDIA Dynamo documentation — distributed inference, Kubernetes integration, routing, cache management, and supported engines.
8. NVIDIA and Noetra national AI infrastructure — a named multi-rack Vera Rubin deployment plan.
9. MLPerf Endpoints — a framework for evaluating inference across latency, concurrency, and throughput.