Exploring Deep Learning Architectures for Spatiotemporal Sequence Forecasting

PhD Thesis Proposal Defence


Title: "Exploring Deep Learning Architectures for Spatiotemporal Sequence 
Forecasting"

by

Mr. Xingjian SHI


Abstract:

Spatiotemporal systems are common in the real world. Forecasting the 
multi-step future of these spatiotemporal systems based on the past 
observations, or, Spatiotemporal Sequence Forecasting (STSF), is a 
significant and challenging problem. Due to the complex spatial and 
temporal relationships within the data and the potential long forecasting 
horizon, it is challenging to design appropriate Deep Learning (DL) 
architectures for STSF. In this thesis proposal, we explore DL 
architectures for STSF. We first define the STSF problem and classify it 
into three subcategories: Trajectory Forecasting of Moving Point Cloud 
(TF-MPC), STSF on Regular Grid (STSF-RG), and STSF on Irregular Grid 
(STSF-IG). We then propose architectures for STSF-RG and STSF-IG problems.

For STSF-RG problems, we propose the Convolutional Long-Short Term Memory 
(ConvLSTM) and the Trajectory Gated Recurrent Unit (TrajGRU). ConvLSTM 
uses convolution in both the input-state and state-state transitions of 
LSTM and is better at capturing the spatiotemporal correlations that the 
Fully-connected LSTM (FC-LSTM). TrajGRU improves upon ConvLSTM by actively 
learning the recurrent connection structure, which achieves better 
prediction performance with less number of parameters. To better 
investigate the effectiveness of our proposed architectures and other DL 
models for STSF-RG, we choose to tackle the precipitation nowcasting 
problem, which is a representative STSF-RG problem with huge real-world 
impact. By incorporating ConvLSTM into an Encoder-Forecaster (EF) 
structure, we propose the first machine learning based solution to 
precipitation nowcasting that outperforms the operational algorithm. To 
facilitate future researches for this problem and gauge the state of the 
art, we propose the first large-scale benchmark for precipitation 
nowcasting: HKO-7. HKO-7 has new evaluation metrics and has both the 
offline setting and the online setting in the evaluation protocol.

For STSF-IG problems, we convert the sparsely distributed observations 
into a spatiotemporal graph and utilize graph convolution operators, or 
graph aggregators, to build the model. We propose a new graph aggregator 
called Gated Attention Network (GaAN). GaAN not only uses multiple 
attention heads to aggregate information from the neighborhoods but also 
uses another set of gates to control each attention head's importance. 
With experiments on two large-scale inductive node classification 
datasets, we show that GaAN outperforms the baseline graph aggregators. 
Also, we propose a unified framework called Graph GRU (GGRU), which 
transforms any valid graph convolution operators, or graph aggregators, to 
RNNs that are designed for STSF-IG.


Date:			Thursday, 30 August 2018

Time:                  	2:00pm - 4:00pm

Venue:                  Room 5501
                         (lifts 25/26)

Committee Members:	Prof. Dit-Yan Yeung (Supervisor)
 			Dr. Yangqiu Song (Chairperson)
 			Dr. Raymond Wong
 			Prof. Nevin Zhang


**** ALL are Welcome ****