Adaptive Scheduling and Resource Management for Heterogeneous Multimodal Data Pipelines

PhD Qualifying Examination


Title: "Adaptive Scheduling and Resource Management for Heterogeneous 
Multimodal Data Pipelines"

by

Mr. Ding PAN


Abstract:

Modern large language models and multimodal foundation models increasingly 
rely on large-scale text, image, audio, video, and document corpora, making 
data preparation a major bottleneck in the AI development pipeline. Unlike 
traditional stream analytics, multimodal data pipelines combine CPU-bound 
preprocessing, accelerator-backed AI inference, highly variable inputs, and 
large intermediate artifacts, creating new challenges for scheduling and 
resource management. This survey reviews adaptive scheduling for 
heterogeneous multimodal data preparation pipelines, with a focus on three 
core problems. First, we examine why capacity estimation is unreliable for 
asynchronous AI operators, whose observed throughput may be distorted by 
continuous batching, input-dependent inference cost, upstream starvation, 
downstream backpressure, and transient queue dynamics. Second, we discuss 
adaptive configuration tuning under workload regime shifts, where changes in 
document length, image resolution, video duration, token length, or modality 
mix affect both throughput and accelerator memory usage, making out-of-memory 
failures a hard safety constraint. Third, we analyze joint scheduling under 
fixed heterogeneous resources, where parallelism, placement, and inference 
configuration must be coordinated to avoid shifting bottlenecks or increasing 
cross-node data movement. We further review machine-learning-based 
performance modeling, Bayesian optimization, and stream-processing 
autoscaling as key building blocks. The survey concludes that future 
schedulers should move beyond isolated local control toward joint end-to-end 
optimization that is workload-aware, memory-safe, bandwidth-aware, and 
placement-aware.


Date:                   Monday, 11 May 2026

Time:                   3:00pm - 5:00pm

Venue:                  Room 3494
                        Lift 25/26

Committee Members:      Dr. Binhang Yuan (Supervisor)
                        Dr. Wei Wang (Chairperson)
                        Prof. Ke Yi