A Survey on the Evaluation of Federated Learning

PhD Qualifying Examination


Title: "A Survey on the Evaluation of Federated Learning"

by

Mr. Di CHAI


Abstract:

Evaluation is a systematic method to study how well a system achieves its 
goal. Federated learning (FL) is a new paradigm of privacy-preserving 
machine learning, which enables different parties to jointly train models 
without exchanging private data. The evaluation of FL is challenging 
because it is an interdisciplinary area and has many goals, e.g., privacy, 
security, efficiency, and heterogeneity. Meanwhile, the evaluation of FL 
is essential and has many applications. Firstly, FL is an 
application-driven technology, and the evaluation works as access control 
of FL methods in the application such that methods with severe issues are 
filtered out in the application. Secondly, the evaluation system works as 
an incentive mechanism to evaluate each party's contribution and do a fair 
payoff sharing. Thirdly, the evaluation system can work as an online and 
life-long verification to enhance the FL methods' security and privacy. 
Since most of the FL studies assume the participants are semi-honest, 
which cannot be guaranteed in real-world applications. A real-time and 
life-long evaluation is essential to detect malicious behaviors and 
guarantees that the participants strictly follow the predefined secure 
protocol. In this survey, we will first investigate the evaluation aspects 
used in existing works and categorize the evaluation metrics. Afterward, 
we will look into each evaluation aspect and introduce the evaluation 
approaches used in each aspect. Finally, we give a set of challenges and 
future research directions for the evaluation of FL.


Date:			Thursday, 30 September 2021

Time:                  	2:00pm - 4:00pm

Zoom meeting: 
https://hkust.zoom.us/j/92265505543?pwd=cnFFY3AxWDNEeXl0WUh3cC9pWEsxdz09

Committee Members:	Prof. Qiang Yang (Supervisor)
 			Dr. Kai Chen (Supervisor)
 			Dr. Qifeng Chen (Chairperson)
 			Prof. Ke Yi


**** ALL are Welcome ****