Towards Robust Out-of-Distribution Generalization: Data Augmentation and Neural Architecture Search Approaches

MPhil Thesis Defence


Title: "Towards Robust Out-of-Distribution Generalization: Data 
Augmentation and Neural Architecture Search Approaches "

By

Miss Haoyue BAI


Abstract

Deep learning has been demonstrated with tremendous success in recent 
years. Despite so, its performance in practice often degenerates 
drastically when encountering out-of-distribution (OoD) data, i.e. 
training and test data are sampled from different distributions. In this 
thesis, we study ways toward robust OoD generalization for deep learning, 
i.e., its performance is not susceptible to distribution shifts in the 
test data.

We first propose a novel and effective approach to disentangle the 
spurious correlation between features that are not essential for 
recognition. It employs decomposed feature representation by 
orthogonalizing the two gradients of losses for category and context 
branches. Furthermore, we perform gradient-based augmentation on 
context-related features (e.g., styles, backgrounds, or scenes of target 
objects) to improve the robustness of learned representations. Results 
show that our approach generalizes well for different distribution shifts.

We then study the problem of strengthening neural architecture search in 
OoD scenarios. We propose to optimize the architecture parameters that 
minimize the validation loss on synthetic OoD data, under the condition 
that corresponding network parameters minimize the training loss. 
Moreover, to obtain a proper validation set, we learn a conditional 
generator by maximizing their losses computed by different neural 
architectures. Results show that our approach effectively discovers robust 
architectures that perform well for OoD generalization.


Date:  			Friday, 20 May 2022

Time:			10:00am - 12:00noon

Zoom Meeting:
https://hkust.zoom.us/j/94654266672?pwd=WWdVQVhwL01jQnh6SGNoOEdzazZSUT09

Committee Members:	Prof. Gary Chan (Supervisor)
 			Prof. Andrew Horner (Chairperson)
 			Dr. Dan Xu


**** ALL are Welcome ****