Efficient Multi-Objective Optimization for Deep Learning

PhD Thesis Proposal Defence


Title: "Efficient Multi-Objective Optimization for Deep Learning"

by

Mr. Weiyu CHEN


Abstract:

While traditional deep learning models optimize for a single objective,
real-world applications in multi-task learning and Large Language Models
(LLMs) require balancing multiple, often conflicting, goals. This thesis
addresses this challenge by developing a portfolio of efficient algorithms
for Multi-Objective Optimization (MOO) in deep learning.

This research introduces several key advances. First, we propose a novel
gradient-based algorithm with adaptive reference vectors that dynamically
conforms to the Pareto front's geometry, yielding a more accurate and
diverse set of discrete solutions. Second, we present a distinct,
parameter-efficient method to learn the entire continuous Pareto manifold,
enabling scalable, on-the-fly generation of any trade-off solution. We also
address preference-aware model merging by framing it as an MOO problem,
generating a Pareto set of models from which users can select a solution
aligned with their priorities. Finally, we tackle LLM efficiency by
developing a multi-objective, one-shot pruning method that produces a family
of models representing optimal trade-offs between size, cost, and
performance.

Collectively, these contributions deliver a powerful and versatile set of
methodologies for creating more flexible, personalized, and efficient models
for critical AI applications.


Date:                   Thursday, 27 November 2025

Time:                   10:00am - 12:00noon

Venue:                  Room 5562
                        Lift 27/28

Committee Members:      Prof. James Kwok (Supervisor)
                        Dr. Dan Xu (Chairperson)
                        Dr. Long Chen