Prof. Bo Li Receives IEEE ICDCS 2025 Distinguished Paper Award

Prof. Bo Li, Chair Professor of Department of Computer Science and Engineering, was a co-recipient of IEEE ICDCS (2025) Distinguished Paper Award. The co-authored paper titled “Mast: Efficient Training of Mixture-of-Experts Transformers with Task Pipelining and Ordering” was presented at the 45th IEEE International Conference on Distributed Computing Systems (ICDCS) 2025, Glasgow, Scotland, July 20-23, 2025. IEEE ICDCS is a premier forum on topics related to all aspects of Distributed Computing Systems. This year, three papers received the Distinguished Paper Award out of 104 accepted from 529 submissions.

In this groundbreaking work, Professor Li and his collaborators from HIT-SZ and HKUST-GZ tackled the fundamental challenge in limiting the scaling efficiency of MoE LLM training due to the overwhelming communications incurred, which is one of the Professor Li’s focuses in recent 3-5 years. The team developed Mast—a new framework to efficiently train MoE models by pipelining and re-ordering communication and computation tasks to effectively hide communication costs. They design intelligent task overlaps in both attention layers and MoE layers and develop optimization techniques to derive the near-optimal order for task pipelining with the objective of minimizing iteration time. The results improve the state-of-the-art MoE speedup factor by over 40%.

Congratulations to the authors—students from HIT-SZ and HKUST-GZ, Prof. Shaohuai Shi (former CSE RaP, now a Professor) and Prof. Xiaowen Chu (a CSE Alumni, Head of DSA Thrust, HKUST-GZ). This prestigious recognition highlights the department’s commitment to cutting-edge, impactful research in AI systems and our continued efforts in fostering close collaboration in Greater Bay area.