in partnership with

MOOC
Machine Learning in
Weather & Climate

TRANSCRIPT

Five years of ML in weather and

climate modelling

Expert: Matthew Chantry

 Five years of ML in weather and climate modelling

Expert: Matthew Chantry

Introduction
Hello and welcome to this podcast, part of the first module of the Machine Learning in Weather
and Climate MOOC.

I’m your host, Lisa Burke and with me is Matthew Chantry, scientist for machine learning at
ECMWF.

In this podcast we’ll be taking a whistlestop tour of the last 5 years of machine learning in
weather and climate.

Lisa: Matthew, take me back 5 years to 2018, what was the state of the field?

Matthew: Well, Lisa, before 2018, there had been limited use of sophisticated machine
learning techniques in the field.

Some exceptions to this are the works of Chevallier & Krasnopolsky, who independently
worked on using shallow neural networks to recreate components of weather forecasting
models.

In 2018 we saw a revisiting of this topic with the work of Rasp and colleagues, who used a
neural network to represent several physical processes in a climate model. They used a 9-
layer multi-layer perceptron, also known as a fully-connected neural network, to learn a
mapping from the current state of the atmosphere to the contributions to this atmosphere state
resulting from physical processes taking place below the grid-scale of the climate model.

Also in 2018, Peer Nowack and colleagues built an ozone parametrization scheme which could
predict ozone distributions from temperature alone. This is particularly useful in contexts where
climatological ozone is currently being used.

2
These papers, building on the prior work of Chevallier & Krasnopolsky, have led to a booming
research area aiming to replace physical components of weather and climate models with
machine learning.

Lisa: What do you think have been the main developments over these last 5 years?

Matthew: Firstly, we have seen significant steps to incorporate more physical knowledge into
these, so called, black boxes. This could be by choosing neural network layers that impose the
same physical priors as your system or building in analytical constraints, for example, to ensure
that a physical process conserves energy. Tom Beucler and coauthors described a framework
for doing this for arbitrary constraints. As a result of these steps, the latest solutions are now,
in my opinion, less black boxes and more translucent boxes.

An alternative approach to using machine learning for learning model components is to use
machine learning to discover equations themselves. In 2020, Zanna and Bolton used this
approach in the context of the ocean to learn an equation-based scheme.

Lisa: The topic of machine learning to learn specific physical processes will be revisited in a
future course. What about unsupervised learning, what’s happened in the last 5 years?

Matthew: Clouds have a huge impact on the earth’s climate, and understanding how cloud
structures change as the climate evolves is key to understand the feedback in the climate. In
2019, Leif Denby used unsupervised learning to cluster cloud types into groups with different
spatial structure. Such a tool can be deployed in observations or model simulations to analyse
whether the distribution of these patterns is changing as the climate changes. However, in
general unsupervised learning has found fewer applications thus far in weather and climate
applications than a supervised approach.

Lisa: Matthew, tell me about benchmark datasets, what are they and how have they featured
in the past 5 years?

Matthew: Benchmark datasets have been key to machine learning progress in other fields.
They provide a complete dataset, description on how to split this into training and testing
components and provide metrics for assessing performance. An early example of this was the
MNIST dataset, consisting of handwritten digits, with the aim to read the correct value.

2020 saw the publication of WeatherBench, a benchmark dataset for data-driven weather
forecasting, focusing on predicting the weather several days into the future. This established
guidelines on how to approach the task of replacing an entire weather forecasting model, albeit
in a stripped-down setting for useability. This benchmark achieved what it set out to do,
focussing attention on a shared task allowing researchers to explore the possible machine
learning solutions. WeatherBench and related efforts will again be a future topic for the MOOC.

3
In 2022 we saw the introduction of ClimateBench, a benchmark problem tasking machine
learning to capture the spatial changes in temperature and rainfall given different emission
scenarios.

The community is increasingly embracing these benchmarks, which will provide a concrete
way to establish the best machine learning tools for specific tasks. With colleagues I have
recently published a paper outlining some of the challenges and benchmark datasets needed
to progress.

Lisa: Nowcasting, which concerns the forecasting over the next few hours and is typically
focused on rainfall. What has been the story here for machine learning?

Matthew: Well, this has been a challenging task for conventional numerical techniques due to
the short time span to deliver forecasts and challenges assimilating radar observations. In
2020 researchers from Google developed MetNet, delivering forecasts over the USA for
several hours. In a follow-up paper, they demonstrated that using conventional models as
additional data sources improves the performance on 6 hour timescales.

In 2021 Deepmind collaborated with the UK MetOffice to deliver UK forecasts with a 2-hour
horizon.

Both performed very well compared to existing benchmarks and deployed cutting-edge
machine learning techniques.

For MetNet they used self-attention, popular in language models, to gather spatial information.
For the Deepmind UKMO collaboration they used a Generative Adversarial Network approach,
training multiple neural networks, some of them to generate a forecast and others to critique
that forecast.

Lisa: Have we seen any of these models become deployed operationally?

Matthew: To my knowledge neither of these models have been operationally deployed, but it
seems very plausible that machine learning becomes a common approach for forecasts on
these timescales. The domain of nowcasting has yet to coalesce around a benchmark problem,
partially because the approach can be considered over a local region, meaning that
researchers might be tempted to focus on their locality.

Lisa: What about after a forecast has been made, where we might want to correct the output
using machine learning, what has happened here in the last 5 years?

Matthew: Well Lisa, the topic of superresolution, increasing the spatial fidelity of an image has
been an active area in image-based machine learning. The techniques developed there have
applications for weather and climate forecasting, learning to increase the spatial resolution of
our forecasts.

4
One highlight in this application is the work of Leinonen in 2020, who created a model capable
of increasing the resolution of radar data while ensuring temporal consistency. This task is of
particular interest for increasing the resolution of rainfall or near surface winds. High spatial
resolution in the latter can be extremely valuable for wind energy predictions.

This is just one of a host of postprocessing applications, seeking to improve the output of
weather and climate forecasting models. Alongside increasing the resolution, researchers
have used machine learning to remove model biases or better calibrate probabilistic
predictions, improving their reliability.

Lisa: How do we go about trusting the outputs of machine learning models? Forecasts can
have a major impact on lives.

Matthew: Interpretability and explainability have become important components in the use of
machine learning for weather and climate. These techniques aim to make machine learning
solutions human interpretable, to build trust in them delivering high impact forecasts. This
includes a range of tools aiming to interpret why a machine learning model made a given
decision. For some, such interpretability is a key step before any machine learning application
can enter operations.

Summary
Lisa: So Matthew, to wrap up, how can we summarise the changes between 5 years ago and
now?

Matthew: Much has changed in the past 5 years. In 2018 fully-connected neural networks and
random forests were the state of the art in the domain. These approaches still have their place
depending on the problem.

In 2022 we see the adoption of the latest techniques from image processing, natural language
processing, and other active areas get imported into the weather and climate domain in a
matter of months.

The community now understands how to impose our physical knowledge into these solutions
to help guide machine learning algorithms to the best possible model.

This community has now been bolstered. In 2018 research was carried out by weather and
climate scientists. In 2022 many projects are collaborations between these same domain
scientists and computer scientists, mathematicians and other machine learning experts.

5
Whilst a large amount of exploration has taken place in this time, we have not yet seen many
examples of these exciting machine learning solutions being deployed as operational products.
Over the next 5 years I think that will very much change, we will soon find out.

Lisa: Thank you so much Matthew for accompanying us on this journey on the developments
of Machine Learning over the last 5 years.