More about HKUST
Learning the Structure of Neural Networks: A Survey
PhD Qualifying Examination Title: "Learning the Structure of Neural Networks: A Survey" by Mr. Xiaopeng LI Abstract: In recent years, deep neural networks have achieved breakthroughs in many machine learning tasks. However, it remains an art to design an appropriate architecture for a particular application. Typically, researchers need to evaluate a long list of candidate architectures before finding a satisfactory one, a process that is sometimes called graduate student descent. Thus, there is growing interest in learning the structure of deep neural networks. Structure learning not only involves architecture learning, which automatically determines the number of neurons and the depth of networks for a given problem, but also involves connectivity learning, which leads to sparse networks. Network sparsity is helpful in avoiding overfitting and has the benefit of reduced computational complexity and memory cost. This survey aims to review the existing methods for learning the structure of neural networks. Existing methods can be roughly categorized into constructive methods, network pruning, regularization-based methods, and probabilistic methods. Other methods, such as reinforcement learning and evolutionary algorithms, have also been proposed to address the problem. In this survey, comparisons among different methods are made, and their strengths and weaknesses are also discussed. Date: Wednesday, 24 April 2018 Time: 10:30am - 12:30pm Venue: Room 3494 Lifts 25/26 Committee Members: Prof. Nevin Zhang (Supervisor) Prof. James Kwok (Chairperson) Prof. Albert Chung Dr. Yangqiu Song **** ALL are Welcome ****