Sparsity in Deep Learning: From the Efficiency and Generalization Perspective

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Sparsity in Deep Learning: From the Efficiency and Generalization
Perspective"

By

Mr. Xiao ZHOU


Abstract:

Sparsification is a natural idea to boost the inference and training efficiency
and generalization performance of neural networks. For inference efficiency, it
could work on a small sparse model with much less parameter counts and
computational time while preserving comparable or even better generalization
performance. For training efficiency, it works on a small sparse model with
constrained model size during the whole training process, with sparsified
forward and backward propagations. For generalization performance, besides the
already effective IID (Independently Identically Distributed) setting, we also
give a novel view of ultilizing sparsity to boost the generalizaton performance
in OOD (Out of Distribution) setting. We could also sparsity the dataset to
speed-up the training procedure and boost OOD performance.


Date:                   Wednesday, 29 November 2023

Time:                   3:30pm - 5:30pm

Venue:                  Room 4475
                        Lifts 25/26

Chairman:               Prof. David CHANG (HUMA)

Committee Members:      Prof. Tong ZHANG (Supervisor)
                        Prof. Qifeng CHEN
                        Prof. Tim CHENG
                        Prof. Zhiyao XIE (ECE)
                        Prof. Ying TAN (Peking University)


**** ALL are Welcome ****