A Survey of Reinforcement Learning for Cloud Resource Management

PhD Qualifying Examination


Title: "A Survey of Reinforcement Learning for Cloud Resource Management"

by

Mr. Suyi LI


Abstract:

Computing jobs from academia and industry are moving to the cloud because it 
provides a platform to develop and deploy applications without worrying about 
hardware infrastructure and software setup. The increasing workloads then pose 
challenges to cloud service providers to manage and allocate their computing 
resources such as CPU, memory, storage and network bandwidth. Proper resource 
management, which provides high-quality service at a low operation cost, is the 
key to achieving profitable cloud service.

Through the past years, researchers have been making efforts to optimize 
resource management strategy in the cloud. Typically, researchers rely on their 
domain expertise to develop clever heuristics and painstakingly test and tune 
them for improvement. Intuitive and interpretable as the heuristics are, most 
of them only provide qualitative guidelines rather than quantitative yet can 
still be inaccurate. Later, inspired by the recent advances in reinforcement 
learning, the ideas of formulating resource management as a sequence of 
decision-making problems and solving it by reinforcement learning algorithms 
become increasingly attractive. Reinforcement learning techniques enable cloud 
service providers to deploy an agent in the cloud who automatically learns from 
past experiences and makes intelligent resource allocation decisions. It is 
reported in recent works that the performance of a well-trained agent is 
comparable to the human experts or even better.

This survey serves as a systematical review of the cloud resource management 
topic. We first introduce selected cloud resource management problems and their 
heuristic solutions. Then, we briefly introduce reinforcement learning 
techniques and point out the challenges of applying them. Next, we present some 
state-of-the-art reinforcement learning-based solutions in cloud resource 
management. We focus on how they tailor the reinforcement learning framework to 
solve specific resource management problems. Finally, we conclude the survey 
and hope the unsolved challenges could motivate further research directions and 
industrial-oriented solutions in cloud resource management.


Date:			Wednesday, 24 March 2021

Time:                  	4:00pm - 6:00pm

Zoom meeting: 
https://hkust.zoom.com.cn/j/94505032021?pwd=YlY4b3lsZjBpUjFKWWRwVFVHMGdzQT09

Committee Members:	Dr. Wei Wang (Supervisor)
 			Prof. Qian Zhang (Chairperson)
 			Prof. Bo Li
 			Dr. Qiong Luo


**** ALL are Welcome ****