The Impact of Learning Representation on Agents' Behaviors in Noncooperative Games

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "The Impact of Learning Representation on Agents' Behaviors in
Noncooperative Games"

By

Mr. Stefan Warren JUANG


Abstract:

This thesis delves into the learning representations of Nash equilibria (NE) in
noncooperative games, where players independently optimize their individual
preferences and potentials. We address two challenges: (1) reducing the
computation of self-play algorithms, such as PSRO, which prevent cycles of
strategy interactions, and (2) understanding how noncooperative games affect
the behavior diversity of a population of agents. For the first challenge, we
establish the theoretical equivalence between cyclical strategies and support
strategies of a mixed-strategy NE. Leveraging this insight, we design a
directed graph representation that enhances learning efficiency by six times
compared to the state-of-the-art algorithm, Simplex-NeuPL. For the second
challenge, we examine the phenomenon of Skill Transfer in population learning,
where agents' behaviors in noncooperative games converge to a set of general
and transferrable behaviors under a single conditional neural net. We derive
the Policy Gradient Integration and demonstrate that Skill Transfer results
from the learning representation of Interaction Information maximization (IIM)
among agents' actions. While IIM captures the generality of competitive
behaviors to accelerate population learning, it may not fully reflect the
individual preferences and diverse potentials of the agents. To address this,
we propose Joint mutual Entropy Minimization (JEM) to train a population of
Generalists into Specialists. Our experiments show that our approach
outperforms existing methods with a 15% gain in behavior diversity and a 22%
increase in individual performances. This thesis underscores the importance of
understanding learning representation in noncooperative games.


Date:                   Tuesday, 5 December 2023

Time:                   10:30am - 12:30pm

Venue:                  Room 4504
                        lifts 25/26

Committee Members:      Prof. Nevin Zhang (Supervisor)
                        Prof. Fangzhen Lin (Chairperson)
                        Prof. Dit-Yan Yeung


**** ALL are Welcome ****