World’s fastest supercomputers are helping to sharpen climate forecasts and design new materials

DENVER— To really understand how a material behaves, researchers need to simulate its whirling electrons, which govern most of its chemical and electronic properties. But they have traditionally faced a trade-off. They could simulate up to a couple of hundred electrons with near-perfect accuracy. Or they could simulate a much larger number—while accuracy fell off a cliff.

The world’s most powerful supercomputers, operating at the far frontier of speed known as the exascale, have now begun to eliminate that trade-off.

At SC23, a supercomputing conference here, researchers this week reported simulating the behavior of up to 600,000 electrons within a microscopic chunk of a magnesium alloy with nearly the accuracy of a quantum Monte Carlo simulation, the gold standard for much smaller numbers of electrons. “We broke through the accuracy-length scale barrier,” says Sambit Das, a mechanical engineer at the University of Michigan and a member of the team presenting the work.

The simulations showed how defects form in the alloys, which could open the way to designing novel lightweight alloys for fuel efficient cars and airplanes. Applying the techniques to materials called quasi-crystals, ordered solids that lack repeating atomic arrangements, also revealed why they take the unusual shapes they do, an advance that could lead to novel magnetic materials and superconductors.

In other feats of computation, researchers at SC23 reported efforts to predict airflow and noise from a fuel efficient jet engine design, and how heat would pulse through the core of a small modular nuclear reactor, an advance that could inform safer designs. All are among the early results emerging from one of the world’s first exascale supercomputers, Frontier, at the Department of Energy’s (DOE’s) Oak Ridge National Laboratory. Capable of 1.1 exaflops (10 18 flops), or 1.1 billion billion operations per second, Frontier is more than twice as fast as the fastest machine from just 2 years ago.

These results and others on the way from exascale machines coming online in the next few years promise to open a new window into materials, climate science, biology, and medicine. “There is a new science era that is unfolding,” says Ceren Susut, associate director for DOE’s Advanced Scientific Computing Research program.

A pair of Chinese supercomputers is widely credited as the first to cross the exascale threshold in 2022, and research results from those machines, such as the first global climate model to incorporate the cooling effects of specific volcanic eruptions, were presented at the meeting. But Chinese officials have declined to share details about their computers and don’t offer access to scientists outside China through an open review process. “China is opaque to us,” says Eric Stromaier, who helps assemble the biannual TOP500 list of the world’s most powerful supercomputers.

That’s left Frontier as the world’s only official exascale supercomputer. Completed in May 2022, Frontier opened for general scientific use in April. Weighing nearly 270 tons, Frontier contains more than 40,000 processors that make it about 1 million times more powerful than an average desktop computer. It consumes 21 megawatts of power—enough for more than 15,000 homes—and needs to be cooled with four 350-horsepower pumps, powerful enough to fill an Olympic-size pool in 30 minutes, that continuously circulate water.

Researchers used Frontier, the world’s first exascale supercomputer, to simulate a magnesium system of nearly 75,000 atoms and the National Energy Research Computing Center’s Perlmutter supercomputer to simulate a quasicrystal structure, above, in a ytterbium-cadmium alloy.
Researchers simulated 40,000 electrons in a quasi-crystal made of 2000 ytterbium and cadmium atoms. Vikram Gavini/University of Michigan

Frontier is expected to be eclipsed within weeks by Aurora, a second U.S. exascale behemoth now completing its final debugging phase at Argonne National Laboratory. Even though only partially installed, Aurora has already weighed in as the world’s second most powerful computer and should soon top out at more than 2 exaflops. When it opens up for scientific proposals, it is expected to guide engineers in designing more fuel efficient airplanes, aid the quest for green energy catalysts, and propel efforts to predict patient responses to cancer treatments by simulating the spread of metastases through the bloodstream. El Capitan, a third U.S. exascale machine being installed at Lawrence Livermore National Laboratory (LLNL), is expected to come online in the middle of 2024 and help nuclear weapons scientists simulate explosions from the aging U.S. stockpile. The three U.S. machines were supported by $4 billion provided to DOE and the National Nuclear Security Administration, with about half the money dedicated to building the machines and half going to software development and personnel.

Meanwhile, other countries are pressing ahead with their own exascale efforts. Jupiter, an exascale machine in Germany, is slated to come online at the end of 2024. An exascale upgrade to the Fugaku supercomputer in Japan is planned for 2029. And France is currently planning to build an exascale system called Jules Vernes, although a release date has yet to be announced.

The new machines are the culmination of the seemingly relentless 1000-fold jumps in supercomputing speed and power that have occurred every decade or so since the early 1990s. But this last leap from petascale (10 15 flops) to exascale required some design changes. With Frontier, researchers decided to incorporate vast numbers of graphical processing units (GPUs), the high-speed chips at the heart of gaming consoles, bitcoin mining, and artificial intelligence (AI). They also soldered 128 gigabits of memory onto each GPU chip to reduce the time and energy needed to shuffle data back and forth between processors and memory chips.

The payoff was the ability to track events not only at ultrahigh resolution, but over broader spatial or timescales. “What exascale computers give us is an ability to get higher resolution for longer periods of time,” says Lori Diachin, principal deputy director of computing at LLNL.

At the meeting, Luca Bertagna, an applied mathematician at Sandia National Laboratory, reported how Frontier enabled him and his colleagues to sharpen the resolution of DOE’s global climate model from 100 kilometers to just 3 kilometers. That allowed the model to simulate the fine-scale atmospheric processes that give rise to clouds, which in coarser models have to be estimated. Because the behavior of clouds in a warming world represents one of the biggest uncertainties in climate change, the higher resolution should help researchers sharpen their predictions of how rising greenhouse gas concentrations will warm the planet, says John Taylor, a high performance computing expert at Australia’s Commonwealth Scientific and Industrial Research Organisation.

Having conquered the exascale, researchers are already eyeing the next leap in supercomputing: zettascale (10 21 flops) machines. But it’s going to be hard, says Christine Chalk, program manager for DOE’s Exascale Computing Project (ECP). “All the low-hanging fruit is gone.” The biggest issue, hardware experts say, is that the decadeslong trend of steady shrinkage of transistors and other computing devices, known as Moore’s Law, has slowed considerably in recent years.

One popular idea for squeezing more processing power out of current designs is to relax the mathematical precision with which current computer chips make calculations, a change that could lead to a 10-fold improvement in processing power. But such a change could allow any errors to compound, undermining reliability. Other ideas include creating hybrid machines that would incorporate still emerging technologies such as quantum computers and systems tailored for machine learning and AI. “I don’t know what’s coming next, but I hope it will give us another boost,” says Jack Dongarra, a high performance computing expert at the University of Tennessee, Knoxville.

But all that would take a new influx of money, which doesn’t appear to be in the offing. With its work largely complete, DOE’s ECP is due to sunset next month, which could leave hundreds of computer scientists and engineers out of a job. “The fear is that talent will leave the DOE and go to companies like NVIDIA, Microsoft, and Facebook,” Dongarra says. “It’s a very hard thing to replace.”