Exploring Deep Learning for Earth System Forecasting

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Exploring Deep Learning for Earth System Forecasting"

By

Mr. Zhihan GAO


Abstract:

Conventionally, Earth system (e.g., weather and climate) forecasting relies on 
numerical simulations using complex physical models, making it computationally 
expensive and demanding in terms of domain expertise. However, the explosive 
growth of spatiotemporal Earth observation data over the past decade has 
facilitated the development of data-driven models utilizing Deep Learning (DL), 
which have shown impressive potential in various Earth system forecasting 
tasks, including precipitation nowcasting, El Niño Southern Oscillation 
(ENSO) forecasting, and Earth surface forecasting. Despite these advancements, 
accurately forecasting the future evolution of the Earth system remains 
challenging for data-driven algorithms. The community is particularly concerned 
with three major challenges: designing effective network architectures, 
handling the inherent uncertainty of chaotic systems, and integrating 
established scientific principles into data-driven approaches.

This thesis investigates how recent advancements in DL techniques can address 
these challenges. For network architecture design, we propose Earthformer, a 
space-time Transformer for Earth system forecasting. Earthformer is built on a 
generic space-time attention block, named Cuboid Attention, which decomposes 
the data into cuboids and applies cuboid-level self-attention in parallel, 
enabling efficient exploration of space-time attention design. To better handle 
the uncertainty inherent in chaotic Earth systems, where minor differences in 
initial conditions can result in significantly different outcomes, we introduce 
PreDiff, a conditional latent diffusion model for probabilistic precipitation 
nowcasting. In contrast to deterministic methods that produce single-point 
estimates, PreDiff models the probability distribution of future, capturing the 
intrinsic uncertainty in chaotic systems. To incorporate domain knowledge into 
data-driven models, we propose a generic two-stage pipeline that aligns 
probabilistic Earth system forecasting models with constraints from scientific 
principles and prior knowledge through a knowledge alignment mechanism. 
Empirical studies on both synthetic datasets (MovingMNIST and N-body MNIST) and 
real-world benchmarks for precipitation nowcasting, ENSO forecasting, and Earth 
surface forecasting demonstrate the state-of- the-art performance of 
Earthformer, the uncertainty-handling capabilities of PreDiff, and the 
effectiveness of our knowledge alignment mechanism in integrating prior 
knowledge.


Date:                   Tuesday, 27 August 2024

Time:                   10:00am - 12:00noon

Venue:                  Room 3494
                        Lifts 25/26

Chairman:               Prof. Ian Duncan WILLIAMS (CHEM)

Committee Members:      Prof. Dit-Yan YEUNG (Supervisor)
                        Prof. Albert CHUNG
                        Prof. Chi-Keung TANG
                        Prof. Shing Yu LEUNG (MATH)
                        Prof. Sinno Jialin PAN (CUHK)