A Probabilistic Framework for Learning Task Relationships in Multi-Task Learning

PhD Thesis Proposal Defence


Title: "A Probabilistic Framework for Learning Task Relationships in Multi-Task 
Learning"

by

Mr. Yu ZHANG


Abstract:

In many real applications, the cost for acquiring labeled data is very 
high since labeling process is laborious and time consuming. So in many 
applications the labeled data is limited, which is called ‘label data 
deficiency problem'. In machine learning area, many efforts are devoted to 
alleviate this problem by requiring the number of labeled data as small as 
possible. For example, even though labeled data is limited, abundant 
supply of unlabeled data is available at very low cost. Two learning 
frameworks, active learning and semi-supervised learning, are developed to 
utilize the information in the unlabeled data to reduce the dependency on 
labeled data. Moreover, besides utilizing unlabeled data, another learning 
framework, multi-task learning, can also improve the performance in this 
situation by utilizing the label information in other related tasks.

Multi-task learning is a learning paradigm to improve the generalization 
performance of a learning task with the help of some other related tasks. 
This learning paradigm has been inspired by human learning activities in 
that people often apply the knowledge gained from previous learning tasks 
to help learn a new task. Many methods have been proposed for multi-task 
learning and we classify most existing methods into five categories: 
common representation approach, task regularization approach, hierarchical 
Bayesian approach, task clustering approach, and task relationship 
learning approach. Common representation approach is to find common data 
representation for multiple tasks. Task regularization approach extends 
the regularized method in single-task learning to multi-task learning by 
encoding the task relationship as a regularization term. Hierarchical 
Bayesian approach utilizes Bayesian hierarchical model for multi-task 
learning. Task clustering approach divides different tasks into several 
task clusters and learns similar (or identical) model or data 
representation for the tasks in the same cluster. Task relationship 
learning approach is to learn the task relationship from data 
automatically which seems as a powerful and adaptive approach over the 
methods belonging to other categories.

In this proposal, we propose a simple but powerful probabilistic framework 
to learn task relationship in multi-task learning where the novelty lies 
in the matrix variate prior used in our framework and then task 
relationship is very natural to be modeled as a parameter in the matrix 
variate prior. Based on this framework, we develop two concrete methods 
for multi-task learning: Multi-task relationship learning (MTRL) and 
Multi-task generalized t process (MTGTP). By utilizing a matrix-variate 
normal distribution as a prior on the model parameters of all tasks, the 
objective function of MTRL can be formulated as a convex problem which has 
a global optimal solution. MTGTP is a Bayesian method by modeling the task 
covariance matrix as a random matrix with an inverse-Wishart prior and 
integrating it out to achieve Bayesian model averaging to improve the 
performance. Experimental results show that our models can achieve 
state-of-theart performance in many real-world applications.


Date:  			Friday, 7 January 2011

Time:           	3:00pm - 5:00pm

Venue:          	Room 3501
 			lifts 25/26

Committee Members:      Prof. Dit-Yan Yeung (Supervisor)
 			Prof. Nevin Zhang (Chairperson)
 			Dr. James Kwok
 			Prof. Qiang Yang


**** ALL are Welcome ****