AI Accelerator Showdown: Google Ironwood vs AMD MI350 vs NVIDIA GB10

AI Accelerator Showdown: Google Ironwood vs AMD MI350 vs NVIDIA GB10

The AI accelerator market is no longer defined by a single hardware architecture.

At the 2025 Hot Chips conference, Google, AMD, and NVIDIA presented three distinctly different approaches to AI computing. Google’s Ironwood TPU targets hyperscale inference, AMD’s MI350 family focuses on large-scale accelerator clusters, and NVIDIA’s GB10 brings Blackwell-class AI compute into a compact workstation platform.

These products occupy very different positions in the AI infrastructure stack, but their architectures reveal several common priorities: higher memory capacity, greater bandwidth, improved energy efficiency, scalable interconnects, advanced packaging, and hardware designed around increasingly demanding AI models.

The comparison also illustrates an important shift in AI hardware design. Performance is no longer determined solely by accelerator compute throughput. Memory hierarchy, chip-to-chip communication, power delivery, cooling, software ecosystems, and system-level scalability increasingly determine how effectively an AI accelerator can run real workloads.

🧠 Google Ironwood TPU: AI Inference at Hyperscale

#

Google’s Ironwood TPU represents a major evolution of its Tensor Processing Unit architecture.

Unlike earlier generations that were heavily associated with AI training, Ironwood was designed specifically around the increasingly important requirements of large-scale AI inference. The architecture targets workloads such as large language models (LLMs), Mixture-of-Experts (MoE) models, and other inference-heavy applications.

Ironwood is not a conventional accelerator product sold across the open market. Instead, it is deployed within Google’s infrastructure and exposed through Google Cloud services.

Its defining characteristic is scale.

A single Ironwood SuperPod can integrate up to 9,216 TPU chips, delivering a reported peak FP8 compute capacity of 42.5 exaflops. The system contains approximately 1.77 PB of directly addressable HBM, while total system power reaches roughly 10 MW.

Despite this enormous scale, Google reports approximately 2× better energy efficiency than its previous-generation Trillium TPU.

Massive SuperPod Architecture

#

One of Ironwood’s most important innovations is its approach to interconnect and memory scaling.

Google uses Optical Circuit Switches (OCS) to connect accelerator resources across racks. This allows the system to scale beyond the previous 4,096-chip configuration to as many as 9,216 chips.

The optical fabric also provides flexibility when dealing with failures. When hardware becomes unavailable, the system can reconfigure resources and use checkpoint-based recovery mechanisms to maintain workload continuity.

This is particularly important at hyperscale. As accelerator clusters become larger, individual component failures become statistically inevitable. System architecture must therefore assume failure rather than treating it as an exceptional event.

HBM3e and Memory Capacity

#

Each Ironwood TPU includes eight groups of HBM3e memory.

The accelerator provides:

• 192 GB HBM3e per TPU
• Up to 7.3 TB/s memory bandwidth
• Approximately 1.77 PB of shared HBM capacity across a 9,216-chip SuperPod

This enormous memory subsystem reflects the changing requirements of modern AI models.

Model size, context length, activation memory, and inference-time working sets continue to grow. Increasing compute throughput without providing sufficient memory bandwidth can leave accelerator resources underutilized.

Reliability and Security

#

Ironwood also emphasizes enterprise-scale reliability, availability, and serviceability (RAS).

The platform incorporates features including:

• Integrated root of trust
• Secure boot
• Hardware self-test
• Silent data corruption detection
• Runtime arithmetic verification
• Confidential-computing capabilities

These features become increasingly important as AI systems move into production environments where a silent numerical error can potentially propagate through a large inference or training workload.

Liquid Cooling and Power Management

#

Ironwood uses Google’s third-generation liquid-cooling architecture.

Its multi-loop water design is intended to keep cold plates clean while preventing blockages. Google also designed hardware and software together to smooth power-consumption fluctuations.

At hyperscale, power delivery is itself an infrastructure constraint. A system capable of delivering enormous compute throughput must also avoid creating unpredictable electrical loads that complicate data-center operation.

Ironwood Architecture

#

Each Ironwood chip contains two compute chiplets, allowing Google to move beyond the limitations of a single large silicon die.

The TPU architecture also incorporates Google’s latest SparseCore technology, targeting workloads involving embeddings and collective communication.

Google further reported that AI-assisted design methods were used during Ironwood’s development. The AlphaChip team contributed to circuit and layout optimization, demonstrating how AI is increasingly being applied to the design of AI hardware itself.

Rack-Level Configuration

#

Ironwood uses a tray-based architecture.

Each tray contains four liquid-cooled TPU chips, while a rack can accommodate 16 trays, resulting in 64 TPUs per rack. CPU host racks provide the associated system infrastructure.

Copper cables handle local rack connections, while optical circuit switching provides connectivity between racks.

The resulting architecture is designed around one fundamental objective: scaling AI compute without allowing communication, memory, cooling, or power constraints to overwhelm accelerator performance.

🚀 AMD MI350: CDNA 4 for Large AI Clusters

#

AMD’s MI350 series represents a different strategy.

Rather than targeting primarily internal hyperscale infrastructure, AMD is positioning MI350 as a high-performance accelerator platform for large AI clusters and enterprise-scale deployments.

The architecture is based on CDNA 4 and continues AMD’s aggressive use of chiplets and advanced 3D packaging.

A single MI350 contains as many as eight Accelerator Compute Dies (XCDs) stacked on top of two I/O base dies, resulting in approximately 185 billion transistors.

The compute chiplets use TSMC’s N3P process, while the I/O base dies are manufactured using a 6nm process.

MI350X and MI355X

#

The MI350 family is divided into two major configurations:

• MI350X: Optimized for air-cooled systems
• MI355X: Designed for direct liquid cooling and higher-power operation

The two platforms share the same fundamental memory architecture and capacity, while differences in operating frequency and thermal design allow the liquid-cooled MI355X to deliver higher performance.

The MI355X can reach approximately 1.4 kW of board-level power, illustrating how rapidly AI accelerator power envelopes are increasing.

CDNA 4 Compute Improvements

#

CDNA 4 substantially increases AI-oriented arithmetic throughput.

The architecture adds native support for lower-precision formats including:

• FP4
• FP6
• FP8
• Other AI-oriented numerical formats

Lower precision is increasingly important for inference because many AI workloads can achieve strong performance without using traditional high-precision arithmetic for every operation.

By increasing the amount of useful AI computation performed per watt and per unit of memory bandwidth, lower-precision execution can significantly improve accelerator efficiency.

Infinity Fabric

#

AMD also redesigned its Infinity Fabric interconnect for the MI350 architecture.

The two-base-die configuration reduces the number of chip-to-chip connections while allowing wider, lower-frequency die-to-die links.

Each socket provides seven Infinity Fabric links.

Compared with the third-generation implementation used by MI300, AMD reports approximately 2 TB/s more aggregate Infinity Fabric bandwidth.

This is important because large AI models increasingly require multiple accelerators to cooperate on the same workload. The performance of the overall system therefore depends not only on individual GPU throughput but also on how quickly accelerators can exchange data.

Memory and Cache Architecture

#

The MI350 architecture increases local data capacity throughout the hierarchy.

The Local Data Share (LDS) capacity is doubled relative to MI300, while each XCD contains a 4 MB L2 cache.

The architecture can also be configured using different NUMA and compute-partitioning strategies.

A system can expose the entire accelerator as one compute domain or divide individual XCDs into separate logical GPUs.

This flexibility allows system designers to balance memory locality, workload isolation, and application requirements.

Multi-GPU Scaling

#

AMD’s reference platform can integrate up to eight MI350 accelerators on a single baseboard.

Infinity Fabric provides the high-speed accelerator interconnect, while PCIe connects the accelerator platform to host CPUs and networking devices.

AMD uses the standard OAM accelerator module format, allowing the MI350 to fit into established data-center accelerator infrastructure.

A universal baseboard can support up to eight OAM modules, providing a practical migration path for existing MI300 and MI325 systems.

Rack-Scale Deployment

#

The difference between air-cooled and liquid-cooled configurations becomes particularly important at rack scale.

AMD indicates that liquid-cooled systems can support approximately 96 or 128 GPUs per rack, while air-cooled configurations can support around 64 GPUs per rack.

AMD is also offering reference rack designs containing its own CPUs, GPUs, and networking components.

This reflects a broader industry trend toward vertically integrated AI infrastructure, where accelerator vendors increasingly optimize the complete compute stack rather than selling individual chips in isolation.

ROCm and Software

#

Hardware alone does not determine accelerator competitiveness.

AMD continues to develop its ROCm software ecosystem to improve application compatibility, compiler performance, libraries, and AI framework integration.

As accelerator architectures become increasingly capable, software optimization can produce gains comparable to hardware improvements.

For large AI clusters, the ability to efficiently utilize every accelerator is particularly important because underutilized compute translates directly into wasted capital and power.

Looking Toward MI400

#

AMD also positioned MI350 as part of a longer accelerator roadmap.

The next major generation, MI400, is expected to continue the company’s focus on large-scale AI infrastructure and substantially improve performance for advanced AI workloads.

The longer-term strategy is clear: increase compute density, memory capacity, interconnect bandwidth, and system-level efficiency while maintaining compatibility with an expanding ROCm ecosystem.

💻 NVIDIA GB10: Blackwell AI in a Compact Workstation

#

NVIDIA’s approach with GB10 is fundamentally different from both Ironwood and MI350.

Rather than targeting massive data-center accelerator clusters, GB10 brings Blackwell-based AI compute into a compact workstation platform.

The chip is the foundation of NVIDIA’s DGX Spark, designed as a local AI development and experimentation system.

GB10 is a multi-chip, single-package SoC combining a Blackwell GPU with an Arm CPU supplied through NVIDIA’s collaboration with MediaTek.

Both components use TSMC’s 3nm process.

Blackwell GPU Architecture

#

The GPU portion of GB10 retains important Blackwell capabilities despite its compact implementation.

Among the most significant is support for FP4 computation, which is particularly relevant to modern AI inference.

The GPU provides approximately:

• 31 TFLOPS FP32 performance
• Up to 1,000 TFLOPS FP4 performance
• A 24 MB L2 cache
• Support for hardware-level CPU/GPU cache coherency
• NVDEC and NVENC media engines
• Ray tracing and display capabilities

The architecture demonstrates how NVIDIA is extending technologies originally designed for large-scale AI systems into much smaller physical platforms.

Unified Memory Architecture

#

One of GB10’s defining features is its unified memory design.

The system provides 128 GB of LPDDR5X memory shared between CPU and GPU resources.

This differs significantly from conventional discrete GPU systems that use separate system memory and GPU HBM or GDDR memory.

A unified address space simplifies data movement and allows the CPU and GPU to access the same physical memory pool.

NVIDIA positions the system as capable of supporting fine-tuning workloads involving models with up to approximately 70 billion parameters, depending on the specific workload and configuration.

CPU Architecture

#

The CPU portion of GB10 contains 20 Arm CPU cores, arranged into two 10-core clusters.

The cores implement the Arm v9.2 architecture, with each core featuring private L2 cache.

The CPU also integrates the memory controller, making the CPU portion of the SoC an important component of overall memory-system performance.

This creates a close dependency between NVIDIA’s GPU architecture and MediaTek’s CPU and memory implementation.

Chip-to-Chip Interconnect

#

GB10 introduces a low-power chip-to-chip (C2C) interconnect between the CPU and GPU components.

The two dies are integrated using a 2.5D interposer, allowing the system to behave more like a unified processor than a conventional discrete CPU-plus-GPU configuration.

Hardware cache coherency further reduces the software overhead associated with managing data shared between CPU and GPU resources.

Connectivity and DGX Spark Scaling

#

Each DGX Spark includes a ConnectX-7 networking component.

Two systems can be interconnected to handle larger models and workloads than a single unit can support.

The SoC connects to the network adapter through a PCIe 5.0 x8 path, providing up to approximately 200 Gbps of one-way bandwidth.

This is not intended to compete directly with high-end multi-rack accelerator fabrics. Instead, the architecture provides a practical bridge between local AI development and larger cloud infrastructure.

Power Efficiency and Desktop Deployment

#

GB10’s total thermal design power is approximately 140 watts.

That is dramatically lower than the power envelopes associated with flagship data-center accelerators.

The result is a compact AI workstation that can operate from a standard electrical outlet rather than requiring server-class power infrastructure.

This is one of the platform’s most important differentiators.

Developers can prototype, fine-tune, validate, and test AI workloads locally before moving production-scale deployment to cloud or data-center infrastructure.

⚖️ Ironwood vs. MI350 vs. GB10

#

These three platforms demonstrate three different interpretations of AI acceleration.

Platform	Primary Target	Architecture	Memory	Scaling Focus	Power Profile
Google Ironwood	Hyperscale AI inference	TPU with compute chiplets	192 GB HBM3e per TPU	Up to 9,216 TPUs	~10 MW per SuperPod
AMD MI350	Large-scale AI clusters	CDNA 4 chiplet/3D stack	High-capacity HBM	Multi-GPU and rack-scale	Up to ~1.4 kW per MI355X
NVIDIA GB10	Local AI development	Blackwell GPU + Arm CPU	128 GB unified LPDDR5X	Multi-system	~140 W

The comparison reveals why there is no single optimal AI accelerator.

Ironwood prioritizes hyperscale inference efficiency and system-level scalability.

MI350 prioritizes high-performance accelerator compute, memory capacity, and cluster scalability.

GB10 prioritizes local AI development, unified memory, low power, and compact deployment.

🔌 Memory and Interconnect Are Becoming Critical

#

One of the strongest common themes across these architectures is the increasing importance of memory.

Modern AI models are not limited simply by arithmetic throughput. Moving model weights, activations, embeddings, and intermediate data can become a dominant performance and energy cost.

This is why all three platforms invest heavily in different forms of memory and interconnect technology:

• Ironwood uses HBM3e and optical circuit switching at enormous scale.
• MI350 combines HBM with Infinity Fabric and chiplet-based architecture.
• GB10 uses unified LPDDR5X memory and low-power C2C connectivity.

The architectures differ, but the objective is similar: keep compute resources supplied with data while minimizing communication overhead and energy consumption.

❄️ Cooling Is Now a Core Architectural Constraint

#

As accelerator power continues to increase, thermal management has become part of the processor architecture itself.

Ironwood uses advanced multi-loop liquid cooling.

MI355X adopts direct liquid cooling and reaches approximately 1.4 kW per accelerator.

GB10 takes the opposite approach by limiting power consumption to a compact workstation-class envelope.

These approaches reflect their respective deployment environments.

Hyperscale AI systems can justify complex liquid-cooling infrastructure because the compute density is extremely high. Local workstations must instead optimize for simplicity, acoustic performance, electrical requirements, and physical size.

🧮 Precision Is Changing AI Performance Metrics

#

Another important trend is the increasing use of low-precision arithmetic.

FP4, FP6, FP8, and other reduced-precision formats can dramatically increase AI throughput while reducing memory traffic.

However, headline performance figures using different precisions cannot be compared directly.

A reported FP4 throughput number and an FP32 throughput number describe fundamentally different operating points.

For real-world evaluation, developers should consider:

• Model architecture
• Numerical precision
• Effective throughput
• Memory bandwidth
• Batch size
• Sequence length
• Interconnect overhead
• Software optimization
• Power consumption
• Actual application latency

The useful metric is not necessarily the accelerator with the largest theoretical FLOPS number, but the system that delivers the required workload performance most efficiently.

🌍 The AI Hardware Market Is Splitting Into Multiple Tiers

#

The architectures represented by Ironwood, MI350, and GB10 suggest that the AI hardware market is becoming increasingly specialized.

At one end are enormous hyperscale systems designed to operate thousands of accelerators as a single logical computing environment.

In the middle are high-performance accelerator platforms such as MI350 that can scale from individual GPUs to complete AI racks and clusters.

At the other end are compact systems such as DGX Spark, which bring meaningful AI development capabilities directly to engineers and researchers.

This segmentation is likely to continue.

Different workloads have radically different requirements for compute density, memory capacity, latency, power, networking, and cost.

🔮 The Next Phase of AI Accelerator Design

#

The next generation of AI accelerators will likely focus less on isolated chip performance and more on complete system architecture.

Future improvements are expected across several dimensions:

• Higher-bandwidth memory
• Larger shared memory pools
• More efficient low-precision computation
• Advanced chiplet architectures
• Optical and electrical interconnect improvements
• Better accelerator-to-accelerator communication
• More sophisticated power management
• Advanced liquid cooling
• Hardware-assisted security and reliability
• Tighter hardware-software co-design

The most competitive AI platforms will increasingly be those that optimize the entire path from model parameters to silicon, memory, networking, cooling, and software.

🏁 Three Architectures, One AI Hardware Direction

#

Google Ironwood, AMD MI350, and NVIDIA GB10 are not direct competitors in every deployment scenario.

Ironwood is built around hyperscale inference. MI350 is designed for high-performance AI clusters. GB10 brings Blackwell-based acceleration to compact local development systems.

Yet they share the same underlying direction.

AI hardware is becoming a system-level engineering problem.

Compute density must increase without allowing memory bandwidth, interconnects, power delivery, cooling, or software overhead to become limiting factors. As AI models continue to grow in parameter count, context length, and computational complexity, these constraints will become increasingly important.

The next phase of the AI accelerator race will therefore not simply be about who builds the fastest chip. It will be about who can build the most efficient complete computing system for a particular class of AI workloads.