At its virtual GPU Technology Conference, Nvidia launched its new Ampere graphics architecture — and with it, the most powerful GPU ever made: The DGX A100. It’s the largest 7nm chip ever made, offering 5 petaFLOPS in a single node and the ability to handle 1.5 TB of data per second.
Of course, unless you’re doing data science or cloud computing, this GPU isn’t for you. The purpose of the DGX A100 is to accelerate hyperscale computing in data centers alongside servers. In fact, the United States Department of Energy’s Argonne National Laboratory is among the first customers of the DGX A100. It will leverage this supercomputer’s advanced artificial intelligence capabilities to better understand and fight COVID-19.
“Nvidia is a data center company,” Paresh Kharya, Nvidia’s director of data center and cloud platforms, told the press in a briefing ahead of the announcement. That statement is a far cry from the gaming-first mentality Nvidia held in the old days. Still, Nvidia noted that there was plenty of overlap between this supercomputer and its consumer graphics cards, like the GeForce RTX line. An Ampere-powered RTX 3000 is reported to launch later this year, though we don’t know much about it yet.
The DGX A100 is now the third generation of DGX systems, and Nvidia calls it the “world’s most advanced A.I. system.” The star of the show are the eight 3rd-gen Tensor cores, which provide 320GB of HBM memory at 12.4TB per second bandwidth. And while HBM memory is found on the DGX, the implementation won’t be found on consumer GPUs, which are instead tuned for floating point performance.
The system also uses six 3rd-gen NVLink and NVSwitch to make for an elastic, software-defined data center infrastructure, according to Huang, and nine Nvidia Mellanox ConnectX-6 HDR 200Gb per second network interfaces.
Each GPU instance gets its own dedicated resources — like the memory, cores, memory bandwidth, and cache. Each instance is like a stand-alone GPU and can be partitioned with up to 7 GPUs with various amounts of compute and memory. Nvidia claimed that every single workload will run on every single GPU to swiftly handle data processing. This provides a key functionality for building elastic data centers. The entire setup is powered by Nvidia’s DGX software stack, which is optimized for data science workloads and artificial intelligence research.
All of this power won’t come cheap. Despite coming in at a starting price of $199,000, Nvidia stated that the performance of this supercomputer makes the DGX A100 an affordable solution. In fact, the company said that a single rack of five of these systems can replace an entire data center of A.I. training and inference infrastructure. This means that the DGX solution will utilize 1/20th the power and occupy 1/25th the space of a traditional server solution at 1/10th the cost.
While the DGX A100 can be purchased starting today, some institutions — like the University of Florida, which uses the computer to create an A.I.-focused curriculum, and others — have already been using the supercomputer to accelerate A.I.-powered solutions and services ranging from healthcare to understanding space and energy consumption.
If none of that sounds like enough power for you, Nvidia also announced the next generation of the DGX SuperPod, which clusters 140 DGX A100 systems for an insane 700 petaFLOPS of compute. This performance is equivalent to thousands of servers.