A Survey of Network Scheduling for Distributed Data-Parallel Applications

PhD Qualifying Examination


Title: "A Survey of Network Scheduling for Distributed 
Data-Parallel Applications"

by

Mr. Luping WANG


Abstract:

Today’s datacenters employ diverse cluster computation frameworks, such as 
MapReduce to deal with the increasing outsourcing demands. Having various 
distributed data-parallel applications running in the cluster, the 
communication between the successive computa- tion stages often results in 
massive data transfers over the datacenter network. Therefore a broad spectrum 
of network services are expected.

Over the recent years, concentrated efforts in both industry and academia have 
gone into optimizing the network performance, among which coflow is a recently 
proposed networking abstraction to capture communication patterns in 
data-parallel computing frameworks. Specifically, a coflow is defined as a 
collection of parallel flows transferring intermediate data between two groups 
of machines. Not until all parallel flows have completed will a coflow 
complete. To improve application-level networking performance, an ideal coflow 
scheduler should provide isolation guarantees to achieve predictable 
net- working performance, as well as strive to decrease the average coflow 
completion time (CCT). A practical coflow scheduler should also be suitable for 
both clairvoyant coflow that assumes complete coflow knowledge and 
non-clairvoyant coflow scheduling, which lacks of the prior coflow information. 
As a result, recent years have witnessed unremit- ting research efforts 
for designing appropriate coflow schedulers satisfying these 
diverse requirements.

This survey provides a systematical review about state-of-the-art coflow 
schedulers. Given the essential conflict between fairness and performance, 
striking a balance between them is research gap that needs to be bridged. 
Besides, further explorations are also called to provide predictable service 
for non-clairvoyant coflow.


Date:			Friday, 17 January 2020

Time:                  	1:00pm - 3:00pm

Venue:                  Room 3494
                         Lifts 25/26

Committee Members:	Prof. Bo Li (Supervisor)
 			Prof. Lei Chen (Chairperson)
 			Dr. Yangqiu Song
 			Dr. Wei Wang


**** ALL are Welcome ****