Climate and weather modeling has long been a staple of high-performance computing, but as meteorologists seek to improve the speed and resolution of forecasts, machine learning is increasingly finding its place in the field.
In an article published This week in the journal Nature, a team from Google and the European Centre for Medium-Range Weather Forecasts (ECMWF) detailed a new approach that uses machine learning to overcome the limitations of existing climate models and attempt to generate forecasts faster and more accurately than existing methods.
Dubbed NeuralGCM, the model was developed using historical weather data collected by ECMWF and uses neural networks to complement more traditional HPC-style physics simulations.
As Stephan Hoyer, one of the team behind NeuralGCM, wrote in a recent article reportMost current climate models make predictions by dividing the globe into cubes 50 to 100 kilometers on each side, then simulating how air and moisture move within them based on known laws of physics.
NeuralGCM works in a similar way, but the added machine learning is used to track climate processes that are not necessarily as well understood or that occur at smaller scales.
“Many important climate processes, including clouds and precision, vary on scales much smaller (from millimeters to kilometers) than the cube dimensions used in current models and therefore cannot be calculated based on physics,” Hoyer wrote.
Traditionally, these small-scale phenomena are tracked using a series of simpler secondary models, called parameterizations, Hoyer explained. He noted that the problem is compounded by the fact that “these simplified approximations inherently limit the accuracy of physics-based climate models.”
In other words, these parameterizations are not always the most reliable and can degrade the overall accuracy of the model.
NeuralGCM works by exchanging these parameterizations for a neural network. Three models were trained on existing weather data collected by ECMWF between 1979 and 2019 at 0.7, 1.4, and 2.8 degrees of resolution.
According to the study, the results are very promising. Using Google’s WeatherBench2 framework, the team claims that NeuralGCM was able to achieve parity with existing state-of-the-art forecast models up to five days at 0.7-degree resolution, while at 1.4-degree resolution, five- to 15-day forecasts were even more accurate.
Meanwhile, at 2.8 degrees, the team found that the model was able to predict the average global temperature between 1980 and 2020 with an average error rate one-third that of existing atmospheric-only models.
NeuralGCM also proved very competitive with more targeted models like X-SHiELD, which, as Hoyer explains, offers much higher resolution at the cost of greater computational intensity.
Compared to X-SHiELD, the researchers found that NeuralGCM’s 1.4-degree model was able to predict 2020 humidity and temperature with 15 to 20 percent less error. In the same test, they were able to predict tropical cyclone patterns that matched the number and intensity of those observed that year.
Accelerate forecasts
The team didn’t simply replace these parameterizations with neural networks. The entirety of NeuralGCM was written in Google JAX, a machine learning framework for transforming numerical functions for use in Python.
According to Hoyer, moving to JAX had many benefits, including greater numerical stability during training and the ability to run the model on TPUs or GPUs. In contrast, weather models have traditionally run on CPUs, although GPUs are increasingly being used—more on that later.
Because NeuralGCM runs natively on accelerators, Google claims its system is significantly faster and cheaper to run.
“Our 1.4-degree model is more than 3,500 times faster than X-SHiELD, meaning that if researchers simulated the atmosphere for a year with X-SHiELD, it would take them 20 days compared to just eight minutes with NeuralGCM,” Hoyer wrote.
Additionally, Hoyer claims that the simulation can be run on a single TPU as opposed to the 13,000 CPUs needed to run X-SHiELD, and you can even run NeuralGCM on a laptop if you want.
While promising, it’s important to note that NeuralGCM is just a starting point, with Hoyer readily admitting that it’s not a complete climate model. However, that appears to be the long-term goal.
“We hope to eventually include other aspects of the Earth’s climate system, such as the oceans and the carbon cycle, in the model. Doing so will enable NeuralGCM to make predictions on longer time scales, going beyond forecasting weather over days and weeks to make predictions on climate time scales,” Hoyer wrote.
To support these efforts, the model’s source code and weights have been made public on GitHub under non-commercial license. Amateur meteorologists can have fun with it.
ML Gains Ground in Climate Modeling
This isn’t the first time we’ve seen machine learning used in climate modeling. Nvidia’s Earth-2 climate model is another one. example of how AI and HPC can be combined to not only improve forecast accuracy, but also speed up forecasting.
Announced at GTC this spring, Earth-2 is essentially a massive digital twin designed to use a combination of HPC and AI models to generate high-resolution simulations up to two kilometers in resolution.
This is made possible in part by a model called CorrDiff, a diffusion model that Nvidia says can generate weather images with 12.5 times the resolution and up to 1,000 times the speed of other numerical models. The result is a model that is fast and accurate enough to attract the interest of Taiwan, which is eyeing the platform to improve its typhoon forecasts.
At the same time, more and more climate research centers have begun to adopt GPU-accelerated systems. Climate research is one of the many areas of study targeted by the 200 petaFLOP (FP64) Isambard-AI system deployed at the University of Bristol.
Earlier this year, the Euro-Mediterranean Center on Climate Change in Lecce, Italy, tapped Lenovo for its new super Cassandra that will be powered by Intel Xeon Max processors and a small complement of Nvidia H100 GPUs, which the lab aims to use to run a variety of AI-based climate simulations. ®