Etched calls its product a frontier inference cluster. Strip away the new label and the underlying idea is familiar: specialize every layer of a cluster for one workload, then sell the result as an integrated system rather than a box of interchangeable parts.
What Etched means by an inference cluster
Inference is the repeated work of running an already-trained model to answer requests. Training builds the model; inference serves it. An inference cluster therefore emphasizes token throughput, response latency, memory capacity and bandwidth, request scheduling, power efficiency, and continuous availability. Those priorities are not identical to a training cluster, where time-to-train and large synchronized updates dominate the design.
Etched uses the term more specifically. The company is not merely clustering accelerators; it says it co-designs the ASIC, package, boards, interconnect, power delivery, cold plates, rack, compiler/runtime, and manufacturing process as one system. Its first public product is rack-scale, with a longer-term ambition to deploy at much larger datacenter scale.
Seen in that wider frame, the inference cluster is less a new category than a return to vertically integrated supercomputing. The rack becomes the useful design boundary because performance depends on data movement, cooling, power delivery, scheduling, and failure handling as much as it depends on arithmetic inside one chip.
With the rack treated as the primary design boundary, its operating environment becomes part of the architecture rather than an installation detail. That makes the Linux question relevant even though Etched's distinctive technology is proprietary silicon.
Is the Etched cluster running Linux?
Yes, on the host side, with qualifications. Etched does not publish a conventional product sheet naming a Linux distribution, but two of its current job postings reveal more about the operating environment. Its Accelerator Software Engineer posting describes work on Linux kernel drivers, PCIe-visible memory management, command queues, and SR-IOV virtual-function management. A separate Etched posting for a Senior Supercomputing Software Engineer in Taipei says the role is responsible for foundational software powering the company's server infrastructure. The Taipei posting lists Linux as the preferred operating system and covers NetBoot, BIOS and BMC firmware, OpenBMC, Kubernetes, and Docker.
Together, those postings provide enough evidence to call Etched's system a Linux-hosted accelerator cluster. They do not tell us whether customers receive root access, which distribution and kernel are supported, how updates are managed, or whether the on-chip embedded processors run Linux, an RTOS, or proprietary firmware. Nor do they prove that Kubernetes integration is a finished, supported product rather than an engineering direction.
The host stack explains how an operator might control Etched's system, but not why the hardware might justify adopting a new software environment. Etched makes that case through two linked architectural claims: one addresses sustained power, and the other addresses memory movement.
The two architectural ideas worth watching
Low Voltage Inference
Etched argues that sustained AI performance is often limited by power and heat before it is limited by the number of arithmetic units on a die. Its Low Voltage Inference design runs math blocks at less than half the voltage of conventional AI chips, according to the company, and spreads the consequences across circuit design, tiling, scheduling, voltage regulation, packaging, power delivery, and liquid cooling.
The system-level reasoning is credible: a chip that only reaches its headline FLOPS briefly is less useful than one that can sustain a high fraction of peak output. Etched says sparse trillion-parameter mixture-of-experts models can run above 80 percent of peak FLOPS without thermal throttling. That exact result remains a vendor claim until workload definitions, measurement methods, wall power, and independent results are published.
Power is only one ceiling. During autoregressive decode, moving model state and cached context can limit output even when arithmetic units and cooling capacity remain available. Etched's second architectural idea addresses that memory ceiling.
Cluster Scale Memory
To address the memory ceiling, Etched says its Cluster Scale Memory combines HBM and SRAM with a proprietary low-latency, high-bandwidth interconnect to create a shared memory pool across the scale-up domain. The intent is to reduce the hierarchy and switching delays encountered when model weights, key-value caches, or routed experts are spread across several chips.
This is the more consequential cluster claim. If the memory system behaves predictably under real model traffic, it could improve both throughput and time per token without relying entirely on large batches. The unanswered questions are the size of the scale-up domain, consistency and allocation behavior, failure isolation, usable bandwidth under contention, and what happens when a model extends beyond that domain.
Low Voltage Inference and Cluster Scale Memory therefore form a coherent system argument: sustain more computation within the rack's power envelope, then keep that computation fed with a flatter memory system. The evidence question is how much of this advantage survives real models, service-level objectives, failures, and measured rack power.
Before weighing that evidence, the product name requires care. Etched called its 2024 transformer ASIC Sohu, but its current site presents the rack-scale system without using that name. A July 2026 report says Etched retired the Sohu name and told the publication that its current hardware is not limited to transformer models. Etched has not yet published a formal product name or enough specifications to confirm the extent of that broader support. This review therefore uses Sohu only for the earlier chip and its associated claims.
What is established, and what is still claimed
| Area | Public evidence as of July 22, 2026 | Status |
|---|---|---|
| Physical silicon | Etched says its first A0 silicon returned from TSMC's N4P process and that complete racks are being validated with customers. | Credible vendor evidence |
| Commercial demand | The company reports more than $1 billion in signed customer contracts and $800 million raised. Customer identities, delivery terms, and recognized revenue are not public. | Reported, details private |
| Shipping | First racks were announced for summer 2026. We found no named customer confirming a production deployment. | Not independently confirmed |
| 500,000+ tokens/second | Etched's 2024 Sohu material claimed this for an eight-chip server running Llama 70B in FP8. The H100 comparison used TensorRT-LLM 0.10.0; the B200 result was estimated. The current launch does not provide a reproducible physical-system result for outsiders. | Historical vendor benchmark |
| Product identity | Etched's current site does not use the Sohu name. One publication reports that Etched retired it and says the current hardware supports models beyond transformers, but formal specifications are still pending. | Transition not fully documented |
| Linux | Current Etched roles explicitly describe Linux kernel drivers and Linux-preferred rack software, plus common Linux orchestration and observability tools. | Strong supporting evidence |
| Software compatibility | Public preview documentation requires sign-in. Supported models, operators, quantization formats, APIs, upgrade policy, and orchestration integrations are not documented openly enough for procurement review. | Insufficient public detail |
| Independent benchmark | We found no Etched result in the current public MLPerf material and no customer-reproducible benchmark from its current physical hardware. | Missing |
The 2024 claim of more than 500,000 tokens per second still attracts attention, but the more important 2026 milestone is simpler: Etched says A0 silicon works and first racks exist. First-pass silicon and a complete liquid-cooled rack demonstrate substantial execution. They do not establish cost per token, tail latency, fleet reliability, or broad model support.
Our take
Etched's strongest case is not that one chip beats one GPU. It is that the complete system can beat a mature GPU cluster on the throughput-latency-power curve for valuable, high-volume inference workloads.
High-volume inference can reward specialization. Model operators may accept a constrained platform when the savings are large, predictable, and easy to capture. If Etched can materially lower cost while preserving latency and output quality, it could also make workloads such as real-time voice, video, and multi-sample reasoning practical at greater scale. Linux host drivers, familiar datacenter controls, virtualization support, and standard orchestration hooks could make the appliance operationally recognizable even when the accelerator is proprietary.
The same comparison exposes the risk. NVIDIA's advantage is not only general-purpose silicon. It also includes CUDA, libraries, debuggers, profilers, deployment patterns, available engineers, multiple server vendors, cloud capacity, and a long record of production behavior. Etched must replace enough of that operating environment to make migration cheaper than the tokens it saves.
We would engage Etched for a tightly scoped pilot when the workload is large, transformer-dominant, latency-sensitive, and expensive enough to justify a dedicated rack. We would not design a general AI platform around Etched's system yet. The product needs proof under changing models, mixed sequence lengths, failures, rolling upgrades, and real power constraints, not merely a peak throughput demonstration.
Where Etched fits among 2026 rack-scale systems
The pilot recommendation becomes clearer when Etched is placed beside AMD Helios and NVIDIA Vera Rubin NVL72. Those general-purpose GPU systems support training and inference across a broad set of models. Each creates a 72-accelerator scale-up domain and then uses a separate network to connect racks. Etched makes a narrower wager: a rack specialized for frontier-model inference can give up some flexibility in exchange for better sustained economics on a high-volume workload.
The resulting choice is more important than any nominal peak-operations comparison. A useful evaluation asks whether the model runs at the required quality, how latency changes with concurrency, how much power the service draws, which failures reduce capacity, and how much software work is required to keep it operating. Our Helios versus Vera Rubin comparison and the separate AMD and NVIDIA reviews apply the same evidence standard to the general-purpose alternatives.
A serious pilot should convert these architectural differences into measurements. Before committing budget or datacenter floor space to an Etched rack, operators should require clear answers to the following questions.
Questions a cluster operator should ask
- Which exact models, attention variants, MoE routing schemes, activations, and quantization formats are supported today?
- What are the full throughput-versus-latency curves for prefill and decode, including p95 and p99 latency at rising utilization?
- What quality checks accompany lower-precision results, and can the same model output be compared against a GPU reference?
- What is measured wall power for the rack, and what facility water, power-delivery, floor-loading, and cooling requirements apply?
- How does the cluster behave when a chip, tray, link, host, or rack fails during a long-running service?
- Which Linux distribution, kernel, BMC, firmware, container runtime, scheduler, and observability interfaces are customer-controlled?
- Can a customer run an existing serving stack, or must the application target Etched's compiler, runtime, and API?
- What benchmark can a prospective customer reproduce before signing a purchase contract?
Sources and further reading
- Etched: Frontier Inference Clusters — current product claims, Low Voltage Inference, Cluster Scale Memory, rack validation, funding, and shipment schedule.
- Let's Data Science: Etched books $1 billion in orders — July 9 editor's note reporting that Etched retired the Sohu name and broadened its hardware positioning.
- Etched launch announcement on X — June 30, 2026 launch claims and public engagement.
- Etched Accelerator Software Engineer — Linux kernel drivers, PCIe memory management, command queues, and SR-IOV.
- Etched Senior Supercomputing Software Engineer (Taiwan) — foundational server software, Linux, NetBoot, BIOS and BMC firmware, OpenBMC, Kubernetes, and Docker.
- Bloomberg: Etched funding and planned shipments — independent reporting on the June 2026 announcement.
- MLCommons MLPerf Endpoints — why inference comparisons need latency-throughput curves, quality targets, and reproducibility.