Let Tensor Flow (More) Freely: A Survey on Specialized Learning Systems

PhD Qualifying Examination


Title: "Let Tensor Flow (More) Freely: A Survey on Specialized Learning 
Systems"

by

Mr. Huangshi TIAN


Abstract:

The past decade has witnessed the broad application of machine learning to 
manifold domains, ranging from physical security check to physical simulation, 
from fraud detection to molecule detection, from power plant management to 
automated plant farm. The system community contributed their part with learning 
systems (e.g., TensorFlow, MXNet, PyTorch) which simplify the programming and 
enhance the scalability. Although they claim to be general-purpose, not all 
algorithms have received sufficient support. Some dynamic algorithms are unable 
to be implemented due to the assumption of static computation; some ensembling 
algorithms suffer from poor aggregation performance owing to the naively-chosen 
architecture; some online algorithms demand pipeline integration which is 
unthought of in mainstream systems. In light of those problems, researchers 
have proposed specialized learning systems which are dedicated to a subtype of 
machine learning.

This survey revolves around three subtypes of learning algorithms and the 
systems specifically designed for them. First, we describe how dynamic 
learning, previously unsupported, gets supported with dynamic computation graph 
and further optimized with dynamic batching and data decoupling. The second 
part sketches out how researchers ameliorate the scalability of ensemble 
learning in both single-machine and distributed environment. Thirdly, we 
introduce a system for online learning that streamlines the model updating 
process, accelerates data incorporation and mitigates the data skew. We 
conclude the survey by summarizing the lessons learned and draw three 
guidelines for future learning systems: flexibility, composability and 
extensibility.


Date:			Wednesday, 7 November 2018

Time:                  	10:00am - 12:00noon

Venue:                  Room 2408
                         Lifts 17/18

Committee Members:	Dr. Wei Wang (Supervisor)
 			Prof. Gary Chan (Chairperson)
 			Dr. Kai Chen
 			Dr. Yangqiu Song


**** ALL are Welcome ****