System Optimization in Synchronous Federated Training

PhD Qualifying Examination


Title: "System Optimization in Synchronous Federated Training"

by

Mr. Zhifeng JIANG


Abstract:

The unprecedented demand of collaborative machine learning in a 
privacy-preserving manner gives rise to a novel machine learning paradigm 
called federated learning (FL). Given a sufficient level of privacy 
guarantees, the practicality of an FL system mainly depends on its 
time-to-accuracy performance during the training process. Despite bearing 
some resemblance with traditional distributed training, FL has four 
distinct challenges that complicate the optimization towards shorter 
time-to-accuracy: information deficiency, coupling for contrasting 
factors, client heterogeneity, and huge solution space. Motivated by the 
need for inspiring related research, in this paper we survey highly 
relevant attempts in the FL literature and organize them by the related 
training phases in the standard workflow: selection, configuration, and 
reporting. We also review exploratory work including measurement studies 
and benchmarking tools to friendly support FL developers. Although a few 
survey articles on FL already exist, our work differs from them in terms 
of the focus, classification, and implications. To our best knowledge, 
this survey is the first FL review that focuses on general system-level 
efforts for optimizing the synchronous federated training process towards 
better time-to-accuracy.


Date:			Tuesday, 20 July 2021

Time:                  	2:00pm - 4:00pm

Zoom meeting:
https://hkust.zoom.us/j/96511539981?pwd=RFRwQkRkVjJzV3hVN2wzM0dDM1kxUT09

Committee Members:	Dr. Wei Wang (Supervisor)
 			Prof. Bo Li (Chairperson)
 			Dr. Kai Chen
 			Dr. Yangqiu Song


**** ALL are Welcome ****