Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "Evaluating and Enhancing LLMs Agent based on Theory of Mind in 
Guandan: A Multi-Player Cooperative Game under Imperfect Information"

By

Mr. Yau Wai YIM


Abstract:

Large language models (LLMs) have succeeded in simple imperfect information 
games and multi-agent coordination, but their effectiveness in complex 
collaborative environments remains underexplored. This study evaluates 
open-source and API-based LLMs in sophisticated text-based games requiring 
agent collaboration under imperfect information, comparing their performance 
against established baselines. We propose a Theory of Mind (ToM) planning 
technique enabling LLM agents to adapt strategies against various 
adversaries using only game rules, current state, and historical context. An 
external tool addresses the challenge of dynamic and extensive action 
spaces. Results reveal that while LLMs underperform state-of-the-art 
reinforcement learning models, they demonstrate significant ToM 
capabilities. This improves their performance against opposing agents, 
indicating their ability to understand both ally and adversary actions and 
establish effective collaboration. Our findings suggest LLMs' potential for 
complex multi-agent scenarios despite current limitations.


Date:                   Wednesday, 16 July 2025

Time:                   2:00pm - 4:00pm

Venue:                  Room 5501
                        Lifts 25/26

Chairman:               Prof. Kai CHEN

Committee Members:      Dr. Yangqiu SONG (Supervisor)
                        Dr. Binhang YUAN