Aiming for a million GPUs: Meta offers a look inside its Prometheus AI supercluster
Meta has described its gigawatt-scale Prometheus supercluster in Ohio in detail. Phase one is complete and trained the Muse models – eventually around a million GPUs are to work together.
Photo: Gary Halvorson, Oregon State Archives / Wikimedia Commons, CC BY 4.0 (cropped and resized)
Video: YouTube · Open Compute Project · youtube.com
Meta on Monday published a detailed account on its engineering blog of how its Prometheus AI supercluster in New Albany, Ohio, is built. According to the company, the first phase is complete and has already been used to train the Muse models, including the flagship Muse Wonder. At full scale, around one million distributed GPUs are to compute in lockstep.
Hundreds of thousands of GPUs are already running there in traditional, AI-optimised and rapidly deployed buildings. Meta relies on Nvidia GB200 NVL72 and GB300 NVL72 systems, and in future also Vera Rubin NVL72 with more than 170 kilowatts per rack. An Ethernet network links up to 20,736 GPUs per building, while connections between sites span from 2 to more than 100 kilometres.
A key challenge was reliability: in mid-2025 Meta at times recorded almost 1,000 interruptions per 8,000 GPUs per day, a rate it says has since fallen by 99 per cent. In pre-training Muse Wonder, the share of usable compute time was above 85 per cent, according to Meta. Hyperion in Louisiana is planned as the next large site. A video on the Open Compute Project channel, referenced by Meta in its post, looks at building AI infrastructure at this scale.