Deep Transfer Learning: Generalization on Clean and Adversarial Data

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Deep Transfer Learning: Generalization on Clean and Adversarial 
Data"

By

Miss Yinghua ZHANG


Abstract

Machine learning, especially deep learning, has made remarkable progress 
in the past few years. However, the success of deep learning systems 
heavily relies on massive labeled data, while labeled data are usually 
scarce in real-world applications. Transfer learning, which leverages the 
knowledge in well-annotated source domain(s) and helps to learn in a 
low-resource target domain, can effectively reduce the dependency on 
labeled data.

In this thesis, we study the generalization ability of deep transfer 
learning models on clean and adversarial data and build deep transfer 
learning models that are effective and robust. We start with the 
transductive transfer learning on clean data, where the tasks of the two 
domains are the same and there is only domain discrepancy. We identify 
that the performance bottleneck of this setting lies in the large variance 
of the learned latent representations, and propose a Fisher loss to learn 
discriminative representations. We then consider the inductive transfer 
learning setting where both the tasks and the domain distributions are 
different. We propose a Parameter Transfer Unit (PTU) that learns a 
fine-grained parameter transfer strategy from data. The PTU is a general 
and flexible module that can be used in both Convolutional Neural Networks 
(CNNs) and Recurrent Neural Networks (RNNs). Our proposed methods 
effectively improve the transfer performance on clean data.

While most transfer learning research works focus on the generalization 
ability on clean data, transfer learning models are under the threat of 
adversarial attacks, and such risks have been less studied. To fill the 
gap, we systematically evaluate the robustness of transfer learning models 
under white-box and black-box Fast Gradient Sign Method (FGSM) attacks via 
empirical experiments. The empirical evaluations on the robustness of 
transfer learning models indicate that deep transfer learning models are 
vulnerable to adversarial attacks. We further propose Transferred 
Evolutionary Strategies (TES) that fail fine-tuned models effectively and 
efficiently in black-box attack settings. We introduce the research 
frontier of deep transfer learning in this thesis and identify several 
directions for future exploration.


Date:			Wednesday, 12 May 2021

Time:			10:00am - 12:00noon

Zoom Meeting: 
https://hkust.zoom.us/j/97527487879?pwd=TmJlRjNWNmg2QWhCaW8wSEZQTkwvdz09

Chairperson:		Prof. Volkan KURSUN (ECE)

Committee Members:	Prof. Qiang YANG (Supervisor)
 			Prof. Yangqiu SONG (Supervisor)
 			Prof. Kai CHEN
 			Prof. Qifeng CHEN
 			Prof. Yuan YAO (MATH)
 			Prof. Rung-Tsong Michael LYU (CUHK)


**** ALL are Welcome ****