AMD Helios is a credible second rack-scale platform. Software and measurements will decide the rest.

AMD spent years selling accelerators that operators installed inside someone else's server design. Helios changes that boundary. It combines AMD GPUs, CPUs, networking, software, power, and liquid cooling into a rack-scale reference architecture, so AMD can now compete for the whole scale-up domain rather than for one component inside it.

What Microsoft actually announced

Microsoft said it will deploy Helios through a new Azure ND MI455X v7 offering for production-scale inference. AMD was more specific about the intended work: Microsoft plans to use the racks for frontier-model inference, Azure AI services, and customer applications. Neither company disclosed the number of racks, committed power, purchase value, availability date for Azure customers, or measured application performance.

Those omissions matter, but they do not make the announcement empty. A public commitment from the operator that first deployed AMD MI300X in Azure gives Helios a credible path into production. It also shifts the competitive question. AMD no longer needs to prove that one Instinct accelerator can execute a model; it needs to prove that a complete Helios fleet can meet service-level objectives at a better total cost than the alternatives.

CNBC's visit to AMD's Texas lab adds useful physical evidence. The report describes a working rack under development and testing, with 18 compute trays and four MI455X accelerators per tray. It also reports that Helios can weigh up to 7,000 pounds and that Microsoft joins announced users including Meta, OpenAI, and Oracle. Even so, a lab rack and customer commitments are evidence of execution and demand, not independent benchmark results.

What AMD has integrated

Helios is a double-wide Open Rack Wide design rather than a conventional 19-inch server rack. Within that larger mechanical envelope, AMD combines 72 MI455X GPUs, sixth-generation EPYC "Venice" host CPUs, Pensando Vulcano AI NICs, Pensando Salina DPUs, scale-up switches, power shelves, and liquid cooling. AMD calls the result a reference design because OEM and ODM partners will build and sell their own implementations; Helios itself is not one fixed appliance ordered directly from AMD.

The distinction gives operators both flexibility and work. Open Rack Wide, UALink, and Ultra Ethernet reduce dependence on one proprietary mechanical or networking design, while partner implementations can differ in firmware, service contracts, storage, management software, and supported configurations. Therefore an operator should evaluate the exact vendor system being offered, not assume that every product carrying the Helios name is operationally identical.

Inside the rack, UALink over Ethernet connects the 72 GPUs into one scale-up domain with a claimed 260 TB/s of aggregate bandwidth. Pensando networking then supplies a claimed 43 TB/s of scale-out bandwidth for connecting racks. These two fabrics solve different problems: scale-up communication lets one model span accelerators within a rack, while scale-out communication coordinates replicas or larger jobs across racks. The headline bandwidth values are useful design targets, but application performance depends on topology, latency, collective efficiency, congestion, and software behavior under contention.

Why memory is Helios's strongest argument

Each MI455X is specified with up to 432 GB of HBM4, giving a full rack approximately 31 TB of accelerator memory. That capacity can hold larger models, more experts, longer contexts, or more key-value cache without crossing a slower boundary. Inference often becomes a memory-system problem during decode, so usable capacity and bandwidth can matter more than peak arithmetic.

AMD specifies 23.3 TB/s of memory bandwidth per GPU, roughly 6% more than NVIDIA publishes for Rubin. Combined with 50% more HBM4 capacity, that gives Helios the stronger published memory specifications. It does not guarantee better application performance: Rubin's mature software paths may use its memory system more effectively, and both platforms still need comparable workload tests.

This is also why the published exaFLOPS should not be used as a direct scorecard. AMD quotes FP4 and FP8 values using supported OCP and MX formats, while NVIDIA emphasizes NVFP4 and other dense specifications. Different numerical formats, sparsity rules, accuracy behavior, and workload mappings can make nominal operations incomparable. The useful test is the same model, quality target, request distribution, latency objective, and measured rack power on both systems.

Does AMD Helios run Linux?

Yes, at the host and cluster-software layers. Helios is powered by ROCm, and AMD's current ROCm documentation supports Linux distributions including Ubuntu, Red Hat Enterprise Linux, SUSE Linux Enterprise Server, Debian, Rocky Linux, and Oracle Linux. AMD also provides a Kubernetes GPU Operator for driver management, device discovery, scheduling resources, metrics, health checks, and node remediation.

That evidence establishes Linux as the operating foundation, but it does not identify one universal Helios distribution. Microsoft's Azure image may differ from an HPE, Meta, or other partner deployment, and AMD has not yet published a public MI455X production support matrix that ties exact kernels, firmware, ROCm releases, container images, and orchestration versions to each shipping system. Operators should require that matrix because "runs ROCm" is not the same as a tested lifecycle policy.

ROCm's broader framework support is no longer the narrowest part of AMD's case. PyTorch, JAX, TensorFlow, vLLM, SGLang, Triton, and other common tools are named for Helios. The harder question is depth: which model paths are optimized on day one, how quickly regressions are fixed, which profilers and debuggers work across the rack, and how much application-specific tuning is required to approach the advertised hardware limits?

The fair comparison with NVIDIA

The table explains why Helios is credible without proving it is superior. AMD has an on-paper memory-capacity advantage and a more explicitly open rack and fabric story. NVIDIA has a longer rack-scale lineage, a broader established software ecosystem, and more public evidence that its supply chain is already manufacturing the new generation. Our separate Vera Rubin NVL72 review examines that system in depth, while the side-by-side rack comparison covers the full decision matrix.

What is established, and what remains projected

Our take

Helios matters because it turns AMD from an accelerator alternative into a system alternative. Hyperscalers want more capacity and negotiating leverage, but they also need a platform that can be deployed repeatedly without creating a separate engineering project for every rack. The Microsoft announcement indicates that AMD has at least one partner capable of doing that integration work at large scale.

The opportunity and the risk come from the same openness. Standardized racks and fabrics can broaden the supplier ecosystem and reduce lock-in. Yet the operational experience can fragment if firmware, diagnostics, qualification, and service procedures vary by partner. AMD must make the open system feel coherent in production, not merely modular on a diagram.

We would shortlist Helios for a controlled inference or training pilot once the exact partner configuration and software matrix are available. Workloads constrained by model capacity, long contexts, or accelerator memory deserve particular attention. We would not select it from peak specifications or customer logos alone. A procurement decision should depend on measured application quality, latency under load, sustained throughput, rack power, recovery behavior, and the engineering effort required to reach those results.

Questions a cluster operator should ask

1. Which company is the system vendor, and which parts of firmware, support, and lifecycle management belong to AMD versus that partner?
2. Which Linux distribution, kernel, ROCm release, firmware bundle, and container images are qualified together?
3. What throughput-versus-latency curves can we reproduce for our models, sequence lengths, concurrency, and quality targets?
4. How much of the 31 TB HBM4 is usable by the workload, and how does performance change when a model spans the full 72-GPU domain?
5. What are measured rack power, coolant flow, supply temperature, pressure, floor-loading, and service-clearance requirements?
6. How do UALoE collectives behave under contention, and how are failed GPU, NIC, switch, tray, or host components isolated?
7. Which Slurm, Kubernetes, observability, profiling, debugging, and rolling-upgrade workflows are supported rather than merely possible?
8. What happens to price, support, and workload portability if a later Helios partner implementation differs from the first one?

Sources and further reading

1. CNBC: AMD Helios and the Microsoft commitment — independent lab visit, physical design details, customer context, and analyst estimates.
2. Microsoft: Expanding Azure AI and HPC infrastructure with AMD — Azure ND MI455X v7 scope and intended inference workloads.
3. AMD: Expanded Microsoft strategic partnership — deployment timing, system components, and AMD's description of the Azure use case.
4. AMD Helios product page — reference-design status, rack specifications, partner model, and internal-analysis footnotes.
5. AMD Instinct MI455X specifications — current per-GPU memory capacity, memory bandwidth, and accelerator specifications.
6. ROCm Linux system requirements — currently supported Linux distributions and kernels.
7. AMD GPU Operator for Kubernetes — driver, device, metrics, and cluster-management integration.
8. MLPerf Endpoints — a useful model for comparing throughput, concurrency, and latency rather than relying on one peak number.