Optimization in the Real World without Gradients: Theory and Practice

Speaker: Dr. Zhongxiang Dai
         MIT

Title:   "Optimization in the Real World without Gradients:
          Theory and Practice"

Date:     Thursday; 29 February 2024

Time:     11:00am - 12 noon

Venue:    Room 2405 (via lift 17/18), HKUST

Abstract:

Many real-world problems, such as automated machine learning (AutoML) and
AI4Science, require optimizing black-box functions, i.e., functions whose
gradients are not accessible. Bayesian optimization has been one of the
most widely used methods to solve such black-box optimization problems.
This is mainly thanks to its high sample efficiency (i.e., it only
requires a small number of queries to the function) and solid theoretical
guarantees (thanks to its equivalence to multi-armed bandits).

In this talk, I will discuss my research efforts aiming to push the
theoretical and practical boundaries of Bayesian optimization (BO) and
multi-armed bandits (MAB). These include (1) applying them to solve novel
challenging optimization problems, (2) advancing their state-of-the-art
performance both in theory and in practice, and (3) pioneering their
extension to the setting of federated learning. To achieve these goals, I
have designed novel BO and MAB algorithms which are both theoretically
grounded and practically effective. In my future work, I plan to tackle
emerging real-world optimization problems based on BO and MAB, with a
particular focus on (1) automating advanced AI algorithms such as
automatically optimizing the prompt for large language models and (2)
solving important AI4Science problems such as optimizing the structure of
molecules. In addition, I will also continue working on fundamental
theoretical problems in BO and MAB.


***************
Biography:

Dr. Zhongxiang DAI is a Postdoctoral Associate in MIT, Laboratory for
Information and Decision Systems (LIDS). Previously, he was a Postdoctoral
Fellow in National University of Singapore (NUS). Before that, he received
his Bachelor of Engineering (First Class Honors) and subsequently his
Ph.D. in Computer Science, both from NUS. During his Ph.D. study, he was
supported by Singapore-MIT Alliance for Research and Technology (SMART)
Graduate Fellowship, and he was jointly advised by Associate Professor
Bryan Low from NUS and Professor Patrick Jaillet from MIT. He received the
Dean's Graduate Research Excellence Award and multiple Research
Achievement Awards from NUS, School of Computing. His research area is AI
and machine learning, and his main research interests include Bayesian
optimization (BO) and multi-armed bandits (MAB). Specifically, he aims to
develop theoretically principled BO and MAB algorithms to solve real-world
black-box optimization problems (e.g., AutoML and AI4Science). His
research works have been published in top AI conferences such as NeurIPS,
ICML and ICLR.