Learning the Structure of Neural Networks: A Survey

PhD Qualifying Examination


Title: "Learning the Structure of Neural Networks: A Survey"

by

Mr. Xiaopeng LI


Abstract:

In recent years, deep neural networks have achieved breakthroughs in many 
machine learning tasks. However, it remains an art to design an 
appropriate architecture for a particular application. Typically, 
researchers need to evaluate a long list of candidate architectures before 
finding a satisfactory one, a process that is sometimes called graduate 
student descent. Thus, there is growing interest in learning the structure 
of deep neural networks. Structure learning not only involves architecture 
learning, which automatically determines the number of neurons and the 
depth of networks for a given problem, but also involves connectivity 
learning, which leads to sparse networks. Network sparsity is helpful in 
avoiding overfitting and has the benefit of reduced computational 
complexity and memory cost. This survey aims to review the existing 
methods for learning the structure of neural networks. Existing methods 
can be roughly categorized into constructive methods, network pruning, 
regularization-based methods, and probabilistic methods. Other methods, 
such as reinforcement learning and evolutionary algorithms, have also been 
proposed to address the problem. In this survey, comparisons among 
different methods are made, and their strengths and weaknesses are also 
discussed.


Date:			Wednesday, 24 April 2018

Time:                  	10:30am - 12:30pm

Venue:                  Room 3494
                         Lifts 25/26

Committee Members:	Prof. Nevin Zhang (Supervisor)
 			Prof. James Kwok (Chairperson)
 			Prof. Albert Chung
 			Dr. Yangqiu Song


**** ALL are Welcome ****