Adaptive Virtual Agent in Mixed Reality

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering

Final Year Thesis Oral Defense

Title: "Adaptive Virtual Agent in Mixed Reality"

by

LEE Ting-ting

Abstract:

Human-agent interaction has long been an area of interest in the domain of 
Human Computer Interaction (HCI). However, compared with interacting with 
real humans, current virtual agents have a number of short comings. One is 
the lack of adaptiveness to different environment and scenarios. Another 
is that they cannot express empathy according to the user's emotional 
states. It is thus hard for people to develop relationship and affection 
towards the virtual agents. As technology advances, new solutions are 
emerging to build adaptive virtual agents, such as Mixed Reality, 
emotional analysis, and machine learning to generate human-like responses 
using real world chat histories. In this project, we built a lightweight 
and adaptive virtual agent in Mixed Reality that only requires a smart 
phone and a transparent plastic sheet, aiming to solve the above 
limitations. Beyond executing basic tasks on command, the agent can 
interact with the user as if it is physically in front of the user, 
perform voice chat with human-like responses, and show empathy and 
adaptiveness through emotion detection. Finally, the evaluation results 
show that the overall feedback of the interaction is positive and user 
reports that the agent feels livelier with the avatar, animations, the way 
that it is able to respond to almost all queries, and the mixed reality 
display that make the agent looks as if it is in front of the user. This 
study provides a basis for future research in adaptive agent-human 
interaction in mixed reality.


Date            : 4 May 2021 (Tuesday)

Time            : 14:00-14:40

Zoom Link:
https://hkust.zoom.us/j/96400598559?pwd=NHRrZDM4WG9uMjFQZ0xEZWs5SGZOZz09

Meeting ID      : 964 0059 8559
Passcode        : 671598

Advisor         : Dr. MA Xiaojuan

2nd Reader      : Prof. HORNER Andrew