Supermicro Launches First NVIDIA HGX™ B200 Systems, Showcasing AI Performance Leadership in MLPerf® Inference v5.0 Results
Supermicro Unveils Record-Breaking Performance with NVIDIA B200 Systems
Introduction to Supermicro’s Achievement
Super Micro Computer, Inc. (SMCI), based in San Jose, California, has recently made headlines with its impressive performance in the world of artificial intelligence (AI) and machine learning (ML). The company has achieved leading scores in the MLPerf Inference v5.0 benchmarks, using the new NVIDIA HGX™ B200 8-GPU configurations. Notably, these cutting-edge systems have demonstrated more than three times the token generation per second compared to older models, marking a significant leap in performance metrics.
Benchmark Performance Overview
The latest benchmarks indicate that Supermicro’s 4U liquid-cooled and 10U air-cooled systems have outperformed previous generation systems significantly. In particular, the results for the Llama2-70B and Llama3.1-405B benchmarks revealed the B200 systems achieving over 129,000 tokens per second in certain tasks.
Key Performance Insights:
- Token Generation: The NVIDIA B200 systems consistently generate over three times the tokens compared to H200 8-GPU systems.
- Effective Cooling Solutions: The introduction of advanced liquid cooling and air cooling technologies has optimized performance while maintaining system stability.
Statements from Supermicro Executives
Charles Liang, the president and CEO of Supermicro, emphasized the company’s commitment to innovation and collaboration with NVIDIA. He stated that Supermicro’s modular architecture allows for the rapid development of systems tailored to various ML workloads, thus securing a strong position within the AI market.
About MLPerf Inference v5.0
The MLPerf benchmarks are critical in the AI community as they provide reliable, reproducible, and publicly audited performance results. Supermicro’s groundbreaking performance reflects the effectiveness of their system optimizations and adherence to MLCommons’ strict testing protocols.
Technical Specifications and Features
Supermicro’s systems boast an impressive range of features:
- Optimized Systems: The company offers both air-cooled and liquid-cooled NVIDIA HGX™ B200 8-GPU systems, with solutions tailored for various workloads.
- Advanced Cooling Technology: New cold plates and a 250kW coolant distribution unit have doubled the cooling capacity compared to previous models.
- Rack-Scale Design: These systems come in configurations that enable efficient use of rack space, allowing up to 12 systems with 96 GPUs in a single rack.
Performance Metrics on Key Benchmarks
Recent results highlight significant improvements over older GPU systems. The SYS-421GE-NBRT-LCC and SYS-A21GE-NBRT demonstrated leadership in performance, with over 1,000 tokens per second for the large Llama3.1-405B model. Furthermore, for the smaller LLAMA2-70B benchmark, the Supermicro systems stood out as the highest performing from a Tier 1 supplier.
Industry Recognition
David Kanter, the Head of MLPerf at MLCommons, recognized Supermicro’s noteworthy contributions to the MLPerf benchmarks, commending the remarkable performance improvements validated through neutral and reproducible testing.
A Comprehensive AI Portfolio
Supermicro’s offerings extend beyond the B200 systems, as they provide a robust collection of over 100 GPU-optimized systems. This includes a variety of configurations, sizes, and cooling options tailored for different applications and workloads. Their solutions range from single-socket systems to extensive 8-way multiprocessor configurations, ensuring versatility for their customers.
Commitment to Sustainable Innovations
Supermicro emphasizes its commitment to environmentally responsible computing solutions. The company designs and manufactures its products in-house across multiple locations, implementing strategies aimed at reducing the overall total cost of ownership (TCO) while minimizing environmental impact through what they refer to as Green Computing practices.
Summary
Supermicro’s achievements with the NVIDIA B200 systems illustrate a monumental advancement in AI and ML technology, signifying a new era of performance in this rapidly evolving field. Their commitment to innovation and sustainability promises to keep them at the forefront of the technology sector.
