NETWORK CONGESTION CONTROL WITH DEEP REINFORCEMENT LEARNING

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "NETWORK CONGESTION CONTROL WITH DEEP REINFORCEMENT LEARNING"

By

Mr. Han TIAN


Abstract:

Recent years have witnessed a plethora of learning-based solutions, especially
ones adopting deep reinforcement learning (DRL), for congestion control, which
show outstanding performance improvement compared to traditional TCP schemes.
However, several challenges still remain when incorporating deep reinforcement
learning into the classic control task in networking. Some of them are
intrinsic and have not been solved by the current DRL-based CC schemes, e.g.,
the fairness issue; Some are introduced by learning-based algorithms adopting
deep neural networks, e.g., the overhead issue and the generalization issue;
Furthermore, new demands arise to extend capability and flexibility of CC
schemes, e.g., multiple objectives. These problems hinder network transport
designers and operators from putting DRL-based solutions into practice in the
real world.

This thesis presents our effort in solving the above problems. For the fairness
issue, we propose Astraea, a novel learning-based solution based on multi-agent
reinforcement learning that ensures fast convergence to fairness with
stability; For the overhead issue, we propose Spine, a hierarchical congestion
control algorithm that fully utilizes the performance gain from DRL but with
ultra-low overhead; For the multi-objective requirement from applications, we
propose MOCC, a congestion control scheme based on multi-objective
reinforcement learning that fits various performance objectives in one single
model. For the generalization issue, we propose a transfer learning-based DRL
CC that aligns state features from various network conditions. In the future,
we will continue to work towards providing a practical, efficient, and flexible
DRL-based congestion control scheme with consistently high performance across
various network conditions.


Date: 			Thursday, 25 May 2023

Time: 			10:30am - 12:30pm

Venue: 			Room 2128A
			lift 19

Chairperson: 		Prof. David COOK (ECON)

Committee Members: 	Prof. Qiang YANG (Supervisor)
			Prof. Kai CHEN (Supervisor)
			Prof. Brahim BENSAOU
			Prof. Gary CHAN
			Prof. Xuanyu CAO (ECE)
			Prof. Hong XU (CUHK)


**** ALL are Welcome ****