MLPerf Training v6.1 adds post-training test as Rubin, TPUv7 debut
MLCommons says 24 organizations submitted 250 results. Nvidia claims the fastest times, Lambda reports its own gains, and the first Rubin and Google TPUv7 entries appear.
Key takeaways
- MLCommons published MLPerf Training v6.1 results on Nov. 16, with 250 results from 24 organizations.
- The round adds a recommendation benchmark and the suite's first LLM post-training benchmark, based on Qwen3.5-397B.
- First entries using Nvidia's Rubin-generation GPUs and Google's TPUv7 (Ironwood) appear in this round, MLCommons says.
- Nvidia claims the fastest time on every benchmark; Lambda reports its own gains, partly from an unofficial rerun.
- MLCommons does not rank vendors in its announcement and points readers to the full results.
MLCommons, the industry consortium that runs the MLPerf benchmarks, published results for MLPerf Training v6.1 on Monday, Nov. 16, according to the posts of submitters. The round adds the suite's first benchmark for post-training a large language model, and it includes the first entries using Nvidia's Rubin-generation GPUs and Google's TPUv7, also called Ironwood, according to MLCommons.
In MLPerf Training, each system is timed until the model it trains hits a fixed quality target. MLCommons describes the suite as open source and peer reviewed.
What is new in this round?
MLCommons says version 6.1 brings two new benchmarks. The first, DLRMv4-HSTU, is a recommendation test that replaces the older DLRM-dcnv2 benchmark; it uses an attention-based architecture called HSTU and the Yambda dataset. The second, built on Qwen3.5-397B, tests large-scale post-training using Group Relative Policy Optimization (GRPO), which Nvidia describes as a reinforcement learning method. MLCommons says the model learns to reason about software engineering tasks, training on tasks such as tool-calling.
"Post-training has added a new level of complexity to the architecture of AI training," said Shriya Rishab, a co-chair of the MLPerf Training working group, in the MLCommons announcement.
Who took part?
According to MLCommons, 24 organizations submitted 153 systems, which produced 250 results; Lambda's post puts the number of participants at 25. They include AMD, Azure, Cisco, CoreWeave, Dell, Google, HPE, Lambda, Nvidia, Oracle and Supermicro, as well as the University of Florida and tiny corp. Crusoe took part in MLPerf Training for the first time, MLCommons said.
MLCommons says 57 of the systems, or 37%, ran in the cloud, compared with 29% in the previous round. Twelve systems used at least 1,000 accelerators, and the biggest used 10,240. The systems ran on hardware from Nvidia, AMD and Google, MLCommons said. Its announcement does not rank the vendors; it points readers to the full results on its Training benchmark page, where new tools let readers compare and rank results for systems of similar scale.
What does Nvidia claim?
In a blog post, Nvidia says its platforms had the quickest training time on each of the round's benchmarks, and that its platform was the only one with results on all seven. These are Nvidia's own readings of the published results.
Nvidia says its Vera Rubin NVL72 system was fastest on six of the seven benchmarks in its first round, with up to twice the per-GPU performance of its GB300 NVL72 system. On DeepSeek-V3 671B, the one benchmark where Rubin did not post the fastest time, Nvidia says Vera Rubin NVL72 finished in 4.14 minutes on 1,024 GPUs, against 8.03 minutes for GB300 NVL72 at the same size. The fastest time on that benchmark, 1.38 minutes, came from a GB300 NVL72 cluster of 10,240 GPUs, Nvidia says, and Microsoft Azure also ran at that scale on GB200 systems.
Nvidia also published faster reruns of three benchmarks on newer software. It says these are not MLCommons submissions, have not been verified or reviewed, and may not follow the MLPerf rules for verified results.
What do other submitters report?
Lambda, a cloud provider that uses Nvidia hardware, says it trained GPT-OSS-20B in 19.697 minutes on 64 Blackwell Ultra GPUs, which it puts at 2.86 times faster than a Cisco result on 64 H100 GPUs in the previous round. Lambda also reports a separate Flux.1 image-model run of 26.36 minutes, which it says is an unofficial reproduction made after the submission deadline. In that run, Lambda says, its time was 1.12 times faster than AMD's official result on 64 Instinct MI355X GPUs for the same workload. Lambda notes its results are subject to final review by MLCommons.
AMD and Google are listed as submitters, but the sources reviewed for this article do not describe their results. For each submitter's results, MLCommons points readers to supplemental information on its site.
Questions readers are asking
What does MLPerf Training measure?
The time a system needs to bring a model up to a fixed quality target. MLCommons runs the open, peer-reviewed benchmark suite.
What is new in MLPerf Training v6.1?
Two benchmarks: DLRMv4-HSTU, a recommendation test, and a large-scale LLM post-training test based on Qwen3.5-397B that uses the GRPO reinforcement learning method.
Did Nvidia win every benchmark?
Nvidia says its platforms had the fastest time on all seven benchmarks. That is Nvidia's reading of the results; MLCommons' announcement does not rank vendors.
Sources
- MLCommons: MLPerf Training v6.1 Results: Record Participation
- NVIDIA: NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance
- NVIDIA Developer: NVIDIA Vera Rubin NVL72 Achieves up to 2x Blackwell Ultra Performance in MLPerf Training v6.1
- Lambda: Lambda delivers faster AI training for language and image models
- #mlperf
- #mlcommons
- #ai-training
- #benchmarks
- #gpus
Get the day's top AI & tech stories on WhatsApp: Follow the NextFish channel
Get stories like this every morning.
Related
Chips & Infra2 min
Nvidia unveils Rubin HPC, a science-focused GPU due in 2028
Nvidia says the chip will have more than 50,000 CUDA cores but has not given performance, memory or FP64 figures. ServeTheHome says FP64 is the number to watch.
Chips & Infra2 min
China's LineShine stays No. 1 on TOP500 as new entries hit a record low
The November list keeps the same top 10 as June and adds just 27 new systems, the fewest ever. Tom's Hardware ties the drop to new hardware going to AI data centers.
Chips & Infra2 min
US issues rule to stop remote access workarounds to AI chip export limits
The Commerce Department's BIS says the rule targets data centers in third countries. Reports say it adds license rules for five countries, including Singapore and Malaysia.
Chips & Infra2 min
Nvidia board is not considering Huang's return as CEO, CNBC reports
A person familiar with the matter says the board is concentrating on its inquiry into Huang. Tech figures and a petition have called for his return.