TLDR: Amazon Web Services (AWS) has announced the launch of its new P6e-GB200 UltraServers, designed to support NVIDIA’s most advanced Blackwell GPUs. To address the significant heat generated by these powerful chips, AWS has also developed a custom liquid-cooling solution, the In-Row Heat Exchanger (IRHX), which can be integrated into both new and existing data centers.
Amazon Web Services (AWS) made a significant announcement on Wednesday, revealing its latest P6e-GB200 UltraServers, engineered to handle the immense computational demands of modern artificial intelligence, including the development of trillion-parameter foundation models and agentic AI. These new servers are equipped with up to 72 of NVIDIA’s latest Blackwell GPUs, interconnected with fifth-generation NVIDIA NVLink, enabling them to function as a single, powerful compute system.
A key innovation accompanying the new servers is a custom-built cooling system called the In-Row Heat Exchanger (IRHX). Recognizing the substantial energy consumption and heat output of NVIDIA’s new chips, AWS engineers developed this solution to avoid the time and expense of building entirely new liquid-cooled data centers. The IRHX integrates with existing data center infrastructure to efficiently manage the heat from the densely packed GPU configurations.
Dave Brown, AWS Vice President of Compute and Machine Learning Services, explained the necessity for this custom solution. He stated that off-the-shelf cooling options were insufficient for the demands of NVIDIA’s GB200 NVL72 systems. Brown elaborated that traditional cooling methods “would take up too much data center floor space or increase water usage substantially. And while some of these solutions could work for lower volumes at other providers, they simply wouldn’t be enough liquid-cooling capacity to support our scale.”
In conjunction with the new hardware, AWS is launching new EC2 P6e and P6e Ultra instances. These will provide customers with access to NVIDIA’s HGX B200 and the powerful GB200 NVL72 stack, respectively. This move places AWS in direct competition with other cloud providers like Microsoft and CoreWeave, which have also introduced offerings based on NVIDIA’s GB200 NVL72 architecture.
Also Read:
- Nvidia Intensifies Focus on GPU-as-a-Service Amid Surging AI Demand
- Nvidia Unveils ‘Helix Parallelism’ for Instant AI Processing of Vast Datasets
The announcement also highlighted AWS’s Nitro infrastructure platform, which underpins the new instances. The Nitro system provides essential networking and monitoring capabilities, allowing for firmware updates without service interruptions and delivering up to 28.8 terabytes per second of networking bandwidth per UltraServer, a critical feature for AI workloads that scale across thousands of GPUs.
