SERVERLESS MACHINE LEARNING INFERENCE WITH HETEROGENEITY

PhD Qualifying Examination


Title: "SERVERLESS MACHINE LEARNING INFERENCE WITH HETEROGENEITY"

by

Mr. Sheng YAO


Abstract:

As the demand for machine learning grows rapidly, Serverless Computing has
emerged as a prevalent paradigm for deploying machine learning inference,
offering fine-grained resource scalability and pay-per-use cost model. Unlike
traditional serverless CPU workloads, accelerating serverless machine learning
inference often relies on a diverse set of heterogeneous hardware, such as GPUs
and NICs, each with distinct capabilities. This heterogeneity poses new
challenges to the serverless platform, motivating a heterogeneity-centric
designs to better exploit these resources.

This paper surveys the recent techniques that leverage heterogeneous hardware
for serverless machine learning inference. We begin by introducing the concept
of serverless machine learning inference. Then, we discuss the landscape of
heterogeneous hardware in the serverless platform. Finally, we examine the
state-of-the-art approaches that exploit the heterogeneous hardware to
accelerate serverless machine learning workloads. Our goal is to offer insights
into maximize the utilization of heterogeneous hardware for serverless machine
learning and to foster future research in this field.


Date:                   Wednesday, 24 September 2025

Time:                   1:00pm - 3:00pm

Venue:                  Room 2612B
                        Lifts 31/32

Committee Members:      Dr. Wei Wang (Supervisor)
                        Prof. Bo Li (Chairperson)
                        Prof. Kai Chen