A Survey on Multi-Objective Optimization in Deep Learning

PhD Qualifying Examination


Title: "A Survey on Multi-Objective Optimization in Deep Learning"

by

Mr. Weiyu CHEN


Abstract:

Multi-objective optimization (MOO), which aims to optimize multiple conflicting 
objective functions simultaneously, has become increasingly important in deep 
learning. Common applications include multi-task learning, where a single model 
is required to perform well across various tasks. Additionally, in some 
scenarios, besides accuracy, model fairness and safety are essential to ensure 
ethical and reliable performance. However, traditional MOO methods face 
significant challenges due to the non-convexity and high dimensionality 
inherent in modern deep neural networks, making effective MOO in deep learning 
a complex task.

In this survey, we provide a comprehensive review of recent advancements in MOO 
within the context of deep learning. We begin by introducing the fundamental 
definitions and concepts related to MOO. We then examine methods for finding a 
single Pareto-optimal solution, including both preference-independent and 
preference-dependent approaches. Next, we discuss methods for finding finite or 
infinite sets of Pareto-optimal solutions. Finally, we highlight current 
challenges and suggest promising directions for future research.


Date:                   Wednesday, 30 October 2024

Time:                   10:00am - 12:00noon

Venue:                  Room 5501
                        Lifts 25/26

Committee Members:      Prof. James Kwok (Supervisor)
                        Dr. Dan Xu (Chairperson)
                        Dr. Junxian He
                        Prof. Dit-Yan Yeung