On the Opportunities of Exploiting LLM Agents for ML System Design and Implementation

Speaker: Dr. Binhang YUAN
Assistant Professor
Department of Computer Science and Engineering
HKUST

Title: On the Opportunities of Exploiting LLM Agents for ML System Design and Implementation

Date: Monday, 21 September 2026

Time: 4:00pm - 5:00pm

Venue: Lecture Theater F (Leung Yat Sing Lecture Theater), near lift 25/26, HKUST

Abstract:

Machine-learning systems are increasingly constrained not by large-scale model architecture alone, but also by the difficulty of designing, optimizing, and continuously adapting their underlying software infrastructure under various deployment scenarios. This talk examines the emerging opportunity to employ LLM agents as active autonomous system engineers rather than conventional code-generation assistants. We discuss two of our recent attempts: (i) Autopoiesis extends this idea from offline implementation to online operation: an LLM agent continuously observes serving behavior and rewrites scheduling policies as workloads and cluster conditions evolve. (ii) ARGUS demonstrates how domain-specific abstractions, data-flow invariants, and compiler-generated counterexamples can provide agents with sufficiently structured feedback to synthesize highly optimized GPU kernels, approaching hand-tuned performance across GEMM, attention, and Mixture-of-Experts workloads. We are also actively building AReaL-AutoPilot: an agentic control layer that translates an RL workload specification into a validated deployment plan, configures and launches the corresponding distributed RL system, monitors its execution, and progressively adapts system policies to changing models, workloads, and hardware conditions.

Biography:

Binhang YUAN is an Assistant Professor at the Department of Computer Science and Engineering (CSE), the Hong Kong University of Science and Technology (HKUST) since 2023. He is currently leading the AReaL open source community. He received his Ph.D. degree from Rice University and his bachelor's degree from Fudan University. Before joining HKUST, he was a Postdoc at the Swiss Federal Institute of Technology Zurich (ETH Zurich). His main research interests are in distributed, decentralized, and heterogeneous machine learning systems for foundation models.