Optimizing Resource Scheduling for Cloud Workloads Running in Data Centers

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Optimizing Resource Scheduling for Cloud Workloads Running in Data 
Centers"

By

Mr. Luping WANG


Abstract

With the burst of data volume and application complexity, applications 
running in cloud data centers are scheduled with two categories: 
data-intensive batch jobs that strive for fast completions, and 
customer-facing online services that pursue low response latencies. In 
this dissertation, we aim to separately identify the key factors when 
scheduling each of the two workloads, and optimize their performances 
with tailored scheduling designs.

For data-parallel batch jobs, the communication is often the bottleneck, 
in which a collection of concurrent flows, termed coflow, transfer 
intermediate data between computation stages (e.g., shuffle phase in a 
MapReduce job). Scheduling coflows in a shared cluster is hard, where 
efficiency---minimized average coflow completion times (CCTs) and 
fairness---predictable networking performance  are conflicting with each 
other. In this regard, we make the following contributions. First, we 
present Utopia, a coflow scheduling mechanism that minimizes the average 
CCT while ensuring predictable performance with isolation guarantees. 
Utopia achieves the best of both worlds by preferentially scheduling 
coflows in ascending order of their CCTs under fair-sharing alternatives, 
and providing provable network isolations in the long run. Second, for 
non-clairvoyant coflow scheduling where the coflow size is unavailable in 
advance (e.g., multi-stage applications with pipelines), we present 
non-clairvoyant DRF (NC-DRF), the other scheduling policy that provides 
predictable coflow completions. NC-DRF enforces fair-sharing scheduling 
based on the amount of flows a coflow has on each link, and outperforms 
alternatives by being aware of the coflow-level communication patterns. 
Trace-driven simulations and EC2 deployments have empirically confirmed 
that both Utopia and NC-DRF outperform existing alternatives and achieves 
long-term isolation guarantee.

Online cloud services, on the other hand are deployed as long-running 
applications (LRAs) in containers, where the container placement is of 
paramount importance. Placing LRA containers are known to be difficult as 
they often have sophisticated performance interferences (e.g., resource 
competitions and I/O dependencies) that are hard to be quantitatively 
expressed. We show that optimal LRA placement can be automatically learned 
using deep reinforcement learning (RL) techniques. We first present Metis, 
a general-purpose RL-based scheduler that achieves scalable LRA scheduling 
to large clusters where tens of thousands of LRA containers run on 
thousands of machines. To this end, Metis employs novel hierarchical 
learning techniques that decomposes a complex container placement problem 
into a hierarchy of subproblems with significantly reduced state and 
action space. We show that many subproblems have similar structures and 
can hence be solved by training a unified RL agent offline. We second 
propose the other LRA scheduler, George that achieves high-quality 
container performance subject to operation constraints, such as fault 
tolerance, disaster avoidance and incremental deployment. We design 
a projection-based proximal policy optimization (PPPO) algorithm in 
combination with the Integer Linear optimization technique to 
intelligently schedule LRA containers under operation constraints. In 
order to reduce the training time, we apply the transfer learning 
technique by taking advantage of the similarity in LRA scheduling events. 
We prove theoretically that our proposed algorithm is effective, stable, 
and safe. Both Metis and George are implemented as a plug-in services in 
Docker Swarm. Large-scale EC2 deployments confirm that they 
improve container performance and scale drastically by requiring less than 
1 hour scheduling time in a large cluster with 700 machines.


Date:			Monday, 21 June 2021

Time:			2:00pm - 4:00pm

Zoom Meeting: 		https://hkust.zoom.us/j/5767775326

Chairperson:		Prof. Jiheng ZHANG (IEDA)

Committee Members:	Prof. Bo LI (Supervisor)
 			Prof. Qiong LUO
 			Prof. Yangqiu SONG
 			Prof. Danny TSANG (ECE)
 			Prof. Jianping WANG (CityU)


**** ALL are Welcome ****