Toward Next-Generation Host Networking for Cloud and AI: Efficiency, Scalability, and Flexibility

PhD Thesis Proposal Defence


Title: "Toward Next-Generation Host Networking for Cloud and AI: Efficiency, 
Scalability, and Flexibility"

by

Mr. Xinyang HUANG


Abstract:

The host network connects processors, memory, and network interface cards 
(NICs), supporting communication within hosts and across tightly coupled 
accelerator domains. As datacenter networks become faster, memory-access 
latency and data-movement overhead expose last-mile bottlenecks along 
CPU-to-NIC and GPU-to-NIC paths. Meanwhile, AI workloads demand 
high-bandwidth, low-latency processor-to-processor communication across 
larger compute and memory domains, while diverse deployment environments 
require software control over network performance and functionality.

This thesis aims to advance host network architecture for cloud and AI 
through the design, implementation, and evaluation of systems addressing I/O 
efficiency, interconnect scalability, and software flexibility.

For I/O efficiency, the challenge is to make network data promptly available 
to CPU and GPU computation while minimizing the processor cycles and memory 
bandwidth spent on I/O. CEIO coordinates incoming traffic with cache capacity 
and consumption through credit-based admission and elastic buffering to 
improve CPU packet processing. FreeIO coordinates producer and consumer 
transactions at the GPU-network boundary to reduce data shuffling and 
synchronization costs in GPU all-to-all communication. Together, these 
designs demonstrate that memory-aware I/O coordination improves data 
availability and preserves processor resources for useful computation.

For interconnect scalability, the challenge is to expand accelerator and 
memory domains while sustaining high-bandwidth, low-latency 
processor-to-processor communication. EtherNode combines optical I/O, a 
multi-plane Ethernet fabric, and software transaction coordination to support 
a tightly coupled domain of up to 512 GPUs. Its implementation, deployment 
experience, and evaluation on up to 64 GPUs demonstrate the feasibility of 
commodity Ethernet for scale-up communication. This work establishes 
coordinated endpoint, fabric, and communication-software design as an 
architectural foundation for scalable host networks.

For software flexibility, the challenge is to expose capabilities for 
deployment-specific performance tuning and customized I/O functionality. 
PipeTune identifies performance-relevant configuration knobs and guides their 
tuning through a model of contention along CPU-involved I/O datapaths. XDD 
provides runtime support for customized functions on heterogeneous data 
processing units (DPUs), using fragmented ring buffers and shared lock-free 
queues to reduce data-movement and synchronization overhead. These studies 
establish reconfigurability and efficient programmable execution as 
complementary capabilities for adapting host networks to workload and 
deployment requirements.


Date:                   Wednesday, 21 October 2026

Time:                   4:30pm - 6:30pm

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Prof. Kai Chen (Supervisor)
                        Dr. Xiaomin Ouyang (Chairperson)
                        Prof. Gary Chan