Advancing a Development Platform for Large Language Model Applications

Speaker: Dr. Zheng Wang
Huawei Singapore Research Center

Title: Advancing a Development Platform for Large Language Model Applications

Date: Monday, 7 April 2025

Time: 4:00pm - 5:00pm

Venue: Lecture Theater F (Leung Yat Sing Lecture Theater), near lift 25/26, HKUST

Abstract:

The rapid development of large language models (LLMs) has unlocked new possibilities for enhancing applications across various domains. In this talk, I will present our recent research on an industry development platform that leverages a pre-trained LLM, incorporating key capabilities such as (1) Retrieval-Augmented Generation (RAG), (2) multi-modality, and (3) task planning to create AI agents for a range of applications, including smartphone AI assistants and search engines. In (1), I will detail how RAG improves LLM performance by retrieving relevant information through a multi-partition paradigm, alongside a framework that integrates RAG with an Editable Memory Graph to develop personalized agents, thereby enhancing usability and adaptability for AI assistants. In (2), I will delve into multimodal applications, such as generating query suggestions based on user query images, and employing collaborative prompting for continual learning in Video Question Answering. In (3), I will introduce a novel approach that enhances LLMs as intelligent agents for complex task planning. This method addresses the challenges of enlargability and transferability by retrieving past successful experiences from an external instruction database. Finally, I will outline future research directions: Personal LLM Agents, including the integration of streaming data for online applications and the exploration of LLM-powered personalized multimodal retrieval. Through these discussions, I aim to share valuable insights into the latest advancements in LLM techniques and their potential to drive innovation in real-world products and services.

Biography:

Zheng Wang is currently a Principal Researcher and Huawei TopMinds at Huawei Singapore Research Center. Prior to that, he received his PhD degree at Nanyang Technological University, Singapore, advised by Prof. Cheng Long and Prof. Gao Cong. His current research interests focus on large language models (AI Agents), including retrieval-augmented generation, multimodality, and LLM-based agent planning. During his PhD, he specialized in deep learning, with an emphasis on reinforcement learning for data management and mining. He has published over 30 papers in top-tier conferences and journals in the fields of AI and data science. His research has been recognized with several awards, including one of Best PhD Thesis Awards, Nominated Schmidt Science Fellows, the WAIC Yunfan Award (15 selected worldwide), the Google PhD Fellowship (sole winner from Asia in Structured Data and Database Management), and the AISG PhD Fellowship (one of the top three awardees from NTU).