PhD Thesis Proposal Defence


"Tracing the Solution Path in the Regularization Optimization
Framework"

by

Mr. Gang Wang


Abstract:

In the area of statistical machine learning, we always consider
the generic regularization optimization problem, which is
comprised of two parts, i.e., the loss and regularization. Besides
the parameters we aim to optimize, there are some other
hyperparameters in the formulation, e.g., the regularization
parameter and the kernel parameter, whose values have to be
specified in advance by the user. The optimal hyperparameter value
is data dependent, and the prior knowledge is often required to
set its value properly. The traditional approach to this model
selection problem is to apply methods like cross validation to
determine the best choice among a number of prespecified
hyperparameter values. Extensive exploration such as performing
line search for one hyperparameter or grid search for two
hyperparameters is usually used. However, this requires training
the model multiple times with different hyperparameter values and
hence is computationally prohibitive especially when the number of
candidate values is very large.

The solution path algorithms attempt to calculate every solution
for all hyperparameter values without having to re-train the model
multiple times. By estimating the generalization errors under
different hyperparameter values, the optimal value can be found
with a low extra computational cost. For a large family of
statistical learning models, the solution paths are piecewise
linear and, hence, it is efficient to explore the entire solution
path by monitoring the breakpoints only. However, the solution
paths are often nonlinear for some hyperparameters in the
regularization problems, and it is challenging to have an
efficient approach to trace the nonlinear solution path. The
solution path algorithms are relatively new techniques in machine
learning, which apply a completely different approach to solving
the optimization problem, and its rapid generation of the full
solution path can save much computational cost. In this thesis
proposal, we introduce the basic definition of the solution path
algorithm, review our recent progress on this topic and point out
some promising directions we plan to pursue.


Date:     		Monday, 22 January 2007

Time:                   2:00p.m.-4:00p.m.

Venue:                  Room 3501
			lifts 25-26

Committee Members:      Prof. Frederick Lochovsky (Supervisor)
			Dr. Dit Yan Yeung (Supervisor)
                        Dr. James Kwok (Chairperson)
			Dr. Nevin Zhang
			Prof. Michael Wong (PHYS)


**** ALL are Welcome ****