Efficient Structured Regularization with Proximal Algorithms

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence

Title: "Efficient Structured Regularization with Proximal Algorithms"

By

Mr. Wenliang ZHONG


Abstract

Modern data arising from various domains, such audio, image, text and 
microarray data, are often high-dimensional and contain spurious features with 
various structures. In most cases, a simple model learned from data is at a 
more favorable side than complicated ones, since it can often provide better 
generalization performance, together with intuitive interpretation.

Beside pure sparsity, sophisticated structured-sparsity-inducing regularizers 
are highly desirable when the features have some intrinsic structures. In this 
thesis, several extensions of sparse models are proposed, which can encourage 
1. hierarchical feature selection; 2. group feature selection; 3. graph feature 
selection; and 4. automatic feature clustering during the learning procedure.

As the resulted objectives are often nonsmooth, complicated and even nonconvex, 
optimization can be changeling. We also develop a set of novel algorithms based 
on the proximal method for a wide range of structured regularization problems, 
including convex and nonconvex funtions, deterministic and stochastic settings. 
Theoretically analysis of all these algorithms is presented, guaranteeing that 
their performance is matching or better than state-of-the-art approaches.

Experiments on a number of synthetic and real data sets demonstrate the 
advantage of the novel models and the efficiency of developed algorithms.


Date:			Thursday, 7 August 2014

Time:			2:00pm - 4:00pm

Venue:			Room 3501
 			Lifts 25/26

Chairman:		Prof. Pascale Fung (ECE)

Committee Members:	Prof. James Kwok (Supervisor)
 			Prof. Brian Mak
 			Prof. Dit-Yan Yeung
 			Prof. Daniel Palomar (ECE)
                        Prof. Irwin King (Comp. Sci. & Engg., CUHK)


**** ALL are Welcome ****