Towards Modeling and Mutual Understanding of Poses in Human-Robot Interaction

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Towards Modeling and Mutual Understanding of Poses in Human-Robot
Interaction"

By

Mr. Mingfei SUN


Abstract:

Pose modality, referring to human poses and robot poses, is a powerful cue
to reveal humans' internal status and also a communication channel that
naturally arises in motion planning for the robots. However, compared to
other modalities, e.g., audition, vision, poses have been largely
overlooked even though there is an increasing interaction demand, either
from human poses to the robot’s perception, i.e., pose inference, or
from robot poses to the human’s perception, i.e., pose generation.

The thesis considers typical HRI scenarios and proposes novel models to
meet such demand. To infer humans' cognitive/affective status, we
propose a corpus-based state transition model to sense engagement
dynamics, and a learning-based multi-modality fusion models to estimate
emotion intensity. Results show that the models enable robots to be
significantly more intel- ligent in handling complex interactions with
peripheral interference, as well as in perceiving human partners under
incomplete observations. To generate robot poses, we propose mathe-
matical models for robots to simulate humans' behaviour and approximate
human poses with physical constraints. Evaluation suggests that the
generated poses significantly improve interac- tion transparency and
affect humans' perception towards the robot capability and the
interaction outcomes. We further employ Learning from Demonstration (LfD)
to scale up the pose genera- tion, enabling robots to robustly and
efficiently learn from demonstrated poses. Results show the potentials of
LfD in learning from incomplete demonstrations and in generalizing
demonstrated poses to new scenarios.

To sum up, this thesis presents the computational models for
cognitive/affective inference from human body poses, and explores the
generation of human-like poses in HRI. The thesis takes the first step to
systematically investigate the pose modality as a communication channel in
HRI.


Date:                   Thursday, 20 February 2020

Time:                   5:00pm - 7:00pm

Zoom Meeting:           https://hkust.zoom.us/j/508602451

Chairman:               Prof. Zhenyang LIN (CHEM)

Committee Members:      Prof. Xiaojuan Ma (Supervisor)
                        Prof. Pedro Sander
                        Prof. Chiew-Lan Tai
                        Prof. Sai-Kit Yeung (ISD)
                        Prof. Taku Komura (Univ of Edinburgh)


**** ALL are Welcome ****