Dr. Shaohuai SHI, CSE PhD Alumnus Prof. Xiaowen CHU and Prof. Bo LI Received Best Paper Award in IEEE INFOCOM 2021

Dr. Shaohuai SHI and PhD alumnus Prof. Xiaowen CHU, who is currently a Professor at HKBU and Prof. Bo LI, have received the Best Paper Award in IEEE INFOCOM 2021 with their paper, "Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning". Their paper was one of the three best paper awards among 252 accepted papers out of 1,266 submissions.

In the paper, they used simultaneous All-Reduce communications to expand the scheduling design space and demonstrate that simultaneous All-Reduce communications can effectively enhance the communication efficiency of small tensors through theoretical analysis and experiments. To reduce training iteration time while allowing for both tensor fusion and simultaneous communications, they formulated an optimization problem. By using Horovod and PyTorch, they further developed an efficient optimal scheduling solution and implement the distributed training algorithm ASC-WFBP. The team also ran real-world tests on an 8-node GPU cluster with 32 GPUs with 10Gbps Ethernet. On four modern DNNs, experimental findings indicated that ASC-WFBP could achieve a speedup of around 1.09x - 2.48x over the baseline without tensor fusion, and 1.15x - 1.35x over the state-of-the-art tensor fusion solution.

The IEEE International Conference on Computer Communications (IEEE INFOCOM) is a top-ranked conference on networking in the research community covering both theoretical and systems research. It is a major conference venue for researchers to present and exchange significant and innovative contributions and ideas in the field of networking and closely related areas. Due to the COVID-19 pandemic, the conference was held virtually on 10-13 May 2021.

Congratulations to Dr. Shi, Prof. Chu and Prof Li!

For more details, please refer to the event website.

Certificate of the Best Paper Award received in IEEE INFOCOM 2021

More News