From Deep Reinforcement Learning to LLM-based Agents: Perspectives on Current Research

Speaker: Dr. Stefano Albrecht
University of Edinburgh

Title: From Deep Reinforcement Learning to LLM-based Agents: Perspectives on Current Research

Date: Wednesday, 12 March 2025

Time: 11:00am - 12:00noon

Venue: Rm 1410 (near LTF, lift 25/26), HKUST

Abstract:

Since the recent successes of large language models (LLMs) and their extensions, we are beginning to see a shift of attention from deep reinforcement learning (RL) to LLM-based agents. While deep RL agents are typically trained from scratch to maximise some defined return objective, LLM agents use an existing LLM at their core and focus on clever prompt engineering and downstream specialisation of the LLM via supervised and reinforcement learning techniques. In this talk, I will first provide a broad overview of my research in deep RL, which focuses among other topics on developing sample-efficient and robust RL algorithms for both single- and multi-agent tasks, including industry applications in autonomous driving and multi-robot warehouses. I will then present recent research into LLM agents, where we propose an approach for household robotics that takes into account user preferences to achieve more robust and effective planning. I will conclude by highlighting what I believe is an important limitation in LLM agents, namely that LLMs are not natively designed to maximise objectives for optimal control and decision making. Based on these observations, I believe some fruitful research avenues can be identified.

Biography:

Dr. Stefano Albrecht's research specialises in developing machine learning algorithms for autonomous systems control and decision making, with a particular focus on reinforcement learning and multi-agent interaction. He collaborates closely with industry partners to develop applications in sectors such as autonomous driving and multi-robot warehouses, which has been supported by personal fellowships from the Royal Society and Royal Academy of Engineering. In 2022, his research was nominated for the IJCAI Computers and Thought Award for introducing Stochastic Bayesian Games and optimal solution algorithms, which have since been applied in a range of domains. Dr. Albrecht is founder of the Autonomous Agents Research Group (https://agents-lab.org) at Edinburgh University. Previously, he was a postdoctoral fellow at the University of Texas at Austin, obtained PhD and MSc degrees in Artificial Intelligence from the University of Edinburgh, and a BSc degree in Computer Science from Technical University of Darmstadt. He is co-author of the new MIT Press textbook "Multi-Agent Reinforcement Learning: Foundations and Modern Approaches" which is freely available at www.marl-book.com.