A Survey on Model Compression Co-design

PhD Qualifying Examination


Title: "A Survey on Model Compression Co-design"

by

Mr. Xijie HUANG


Abstract:

Deep Learning Models such as Convolutional Neural Networks (CNNs) have 
demonstrated remarkable performance over previous methods and revolutionized 
various tasks. As the model size and complexity of deep learning models grow 
progressively, the time latency and energy consumption have become the major 
consideration for the efficient deployment of these models. Meanwhile, emerging 
AI accelerators also enable the deployment of deep learning models on edge 
devices. In recent years, researchers have studied deep neural networks' 
design, training, and inferencing techniques. The model compression techniques 
including quantization, pruning, distillation, and sparsity have been proposed 
to fully leverage the redundancy in deep learning models to accelerate the 
computation and shrink the size. However, most previous model compression 
algorithms only consider the theoretical compression rate and conduct 
experiments only on GPUs. In this summary, we conduct a thorough survey on 
model compression algorithms, AI accelerators, and the hardware-aware model 
compression co-design. We will also show some examples of model compression 
co-design including a hardware-aware mixed-precision quantization framework and 
an application to design an efficient model of people counting. This work is 
believed to be the first comprehensive survey in the efficient deep learning 
field that covers most hardware-aware techniques and could potentially inspire 
the reader to explore more co-deign research.


Date:  			Monday, 15 August 2022

Time:                  	2:30pm - 4:00pm

Zoom Meeting: 		https://hkust.zoom.us/j/8305838050

Committee Members:	Prof. Tim Cheng (Supervisor)
 			Dr. Zhiqiang SHEN (Supervisor)
 			Prof. Kai Chen (Chairperson)
 			Prof. Chi-Ying Tsui (ECE)


**** ALL are Welcome ****