Solicited Reports

Weather, machine learning, and GenCast

Peter Battaglia

The GenCast team
Google Deepmind

Weather forecasting is undergoing a revolution, with machine learning-based weather prediction (MLWP) methods rapidly advancing and outperforming NWP-based methods in many application areas. Rather than implementing approximations to the atmospheric fluid questions, these new MLWP methods are fit to historical weather data, predicting future states from current ones on the basis of statistical regularities evident in the training data. They promise to be both more accurate, and more efficient than traditional NWP, opening doors to better, faster, more accessible weather forecasting. This talk will describe our group's work on GenCast, a probabilistic AI-based weather model that outperforms ECMWF's ENS, has good calibration, and shows promising results for predicting extreme weather, tropical cyclones, and wind power. It will also describe our efforts to take our research out of the lab, and provide impact in important real-world problems.

Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales

Noah Brenowitz

Director of Intelligent Earth
NVIDIA

Data assimilation of observational data into full atmospheric states is essential for weather forecast model initialization. Recently, methods for deep generative data assimilation have been proposed which allow for using new input data without retraining the model. They could also dramatically accelerate the costly data assimilation process used in operational regional weather models. Here, in a central US testbed, we demonstrate the viability of score-based data assimilation in the context of realistically complex km-scale weather. We train an unconditional diffusion model to generate snapshots of a state-of-the-art km-scale analysis product, the High Resolution Rapid Refresh. Then, using score-based data assimilation to incorporate sparse weather station data, the model produces maps of precipitation and surface winds. The generated fields display physically plausible structures, such as gust fronts, and sensitivity tests confirm learnt physics through multivariate relationships. Preliminary skill analysis shows the approach already outperforms a naive baseline of the High-Resolution Rapid Refresh system itself. By incorporating observations from 40 weather stations, 10% lower RMSEs on left-out stations are attained. Despite some lingering imperfections such as insufficiently disperse ensemble DA estimates, we find the results overall an encouraging proof of concept, and the first at km-scale. It is a ripe time to explore extensions that combine increasingly ambitious regional state generators with an increasing set of in situ, ground-based, and satellite remote sensing data streams.

AI's Potential and Limitations in Transforming Climate Modeling and Prediction

Tapio Schneider

Theodore Y. Wu Professor of Environmental Science and Engineering
California Institute of Technology, Environmental Science and Engineering

While climate change is certain, precisely how climate will change is less clear, necessitating transformative advancements in climate modeling and prediction. This talk will explore how AI can be leveraged to integrate diverse data sources into climate models, including high-resolution simulations, laboratory measurements, and statistics of weather data. I will address both the limitations of learning holistically from weather data and the opportunities to learn process-level information from diverse data. By combining AI with physical process knowledge, we can achieve substantial improvements in the accuracy of simulations of critical yet uncertain processes, such as those governing cloud formation and behavior. This synergistic approach, uniting AI tools with domain expertise, promises transformative breakthroughs in climate modeling and prediction, empowering us to make more informed decisions in the face of climate change.

Novel Approaches to Constrain Next Generation High-resolution Climate Models

Philip Stier

Director of Intelligent Earth
Department of Physics, University of Oxford

A new generation of high-resolution climate models is emerging. These models explicitly resolve the coupling between clouds and circulation as well as convective cloud dynamics that must be parameterised in traditional climate models. The simulated atmospheric structures can "look" virtually indistinguishable from satellite observations. But how good are these models really?
In this presentation we postulate that traditional approaches for climate model evaluation, generally based on observations highly aggregated in space and time, provide necessary constraints – but exploit only a small fraction of the information content available. This is because the spatio-temporal evolution of the atmosphere, encoding much of the underlying physics, is neglected. We will introduce novel spatio-temporal evaluation approaches combining more traditional methods, such as multi-fractal analysis and cloud tracking, with ML/AI methods, such as the analysis of clouds in latent spaces, encoding atmospheric data using computer vision methods in lower dimensional embedded spaces in which distance represents similarity. To demonstrate the potential to constrain the next generation of high-resolution climate models, we apply our methods to results from the DYAMOND global km-scale model intercomparison and geostationary satellite datasets at matching resolution.