Selective Multi-Source, Non-IID Transfer Learning for Urban Spatio-temporal Forecasting

PhD Thesis Proposal Defence


Title: "Selective Multi-Source, Non-IID Transfer Learning for Urban 
Spatio-temporal Forecasting"

by

Mr. Yilun JIN


Abstract:

Spatio-temporal Forecasting (STF) is a fundamental problem in urban 
computing, supporting a wide range of downstream tasks like transportation, 
energy, meteorology, public security, etc. In recent years, deep learning 
methods have achieved tremendous success in urban STF. However, the constant 
urbanization process creates needs for new cities, new urban services, and 
expansion of existing services. In these scenarios, the problem of data 
scarcity arises, which negatively impacts the accuracy of urban STF and the 
corresponding downstream applications.

Transfer learning is an effective method to address the data scarcity issue 
in fields like computer vision, natural language processing, and 
recommendation. However, the spatial and temporal heterogeneity in urban 
spatio-temporal data poses unique challenges to applying transfer learning 
for urban STF. Specifically, due to such heterogeneity, the source city (or 
cities) contains multiple non-IID latent domains, resulting in multi-source, 
non-IID transfer learning problems. Consequently, it is necessary that some 
latent source domains are helpful to the target city, while others may be 
irrelevant or harmful, which compromises the performance of transfer learning 
for urban STF.

To address this problem, in this thesis proposal, we focus on the problem of 
selective multi-source, non-IID transfer learning for urban STF, whose goal 
is to identify and select helpful latent domains from the source city and 
transfer the knowledge to the target city. We first define the problem of 
selective transfer learning for urban STF and identify three practical 
sub-problems: selective transfer learning for few-shot grid-like urban STF, 
few-shot graph-like urban STF, and zero-shot urban STF. We then present two 
novel selective transfer learning methods, CrossTReS and TransGTR, for 
few-shot grid-like and graph-like urban STF, respectively. CrossTReS selects 
latent source domains by adaptively re-weighting source spatial units. By 
assigning helpful source spatial units with high weights and vice versa, 
latent source domains are selected as source spatial units with high weights. 
TransGTR further selects graph-structured latent source domains by selecting 
both spatial units and the spatial relations between them, which is achieved 
by learning transferable graph structures for the selected spatial units in 
the source city and the target city. Extensive experimental results on 
real-world datasets show the effectiveness of CrossTReS and TransGTR compared 
to existing, non-selective baselines.

In the future thesis work, we will focus on the sub-problem of selective 
transfer learning for zero-shot urban STF, which is more challenging as the 
algorithm should ensure generalization to an unknown distribution of target 
data. Existing works often perform large-scale pre-training over diverse 
spatio-temporal data to ensure generalization, which exacerbates the 
multi-source non-IID issue. Our future thesis work plans to decouple and 
learn from different latent source domains with mixture-of-experts (MoE) and 
select adequate experts at test time with a one-shot transferability 
estimation.


Date:                   Thursday, 5 December 2024

Time:                   2:00pm - 4:00pm

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Prof. Qiang Yang (Supervisor)
                        Prof. Kai Chen (Supervisor)
                        Dr. Yangqiu Song (Chairperson)
                        Dr. Binhang Yuan