Enhanced meta learning for few-shot learning, and recommendation system

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "Enhanced meta learning for few-shot learning, and recommendation 
system"

By

Mr. Runsheng YU


Abstract:

Meta-learning tries to leverage information from similar learning tasks. In the 
commonly used bilevel optimization formulation, the shared parameter is learned 
in the outer loop by minimizing the average loss over all tasks. However, the 
converged solution may be compromised in that it only focuses on optimizing on 
a small subset of tasks. To alleviate this problem, we consider meta-learning 
as a multi-objective optimization (MOO) problem, in which each task is an 
objective. However, existing MOO solvers need to access all the objectives' 
gradients in each iteration, and cannot scale to the huge number of tasks in 
typical meta-learning settings. To alleviate this problem, we propose a 
scalable gradient based solver with the use of mini-batch. We provide 
theoretical guarantees on the Pareto optimality or Pareto stationarity of the 
converged solution. Empirical studies on various machine learning settings 
demonstrate that the proposed method is efficient, and achieves better 
performance than the baselines, particularly on improving the performance of 
the poorly performing tasks and thus alleviating the compromising phenomenon.

Moreover, we introduce a Meta Prompt Learning (MPL) method tailored for online 
recommendation systems. This method leverages a meta prompt to capture useful 
information from historical data efficiently. The key contributions of the MPL 
method include a bi-level optimization strategy to retain essential 
information, a multi-step gradient descent approximation for solution finding. 
Our experiments on datasets such as Tmall, Taobao, and Avazu demonstrate that 
MPL outperforms state-of-the-art models with lower memory usage and training 
time.


Date:                   Wednesday, 29 May 2024

Time:                   10:00am - 12:00noon

Venue:                  Room 5566
                        Lifts 27/28

Chairman:               Dr. Dan XU

Committee Members:      Prof. James KWOK (Supervisor)
                        Dr. Long CHEN