Reducing the Gulfs in Human-Computer Communication: Exploring the Effects of Non-Verbal Metaphors

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Reducing the Gulfs in Human-Computer Communication: Exploring the 
Effects of Non-Verbal Metaphors"

By

Mr. Zhida SUN


Abstract

Within every interaction between a human and a computer, there exist the twin 
challenges of understanding the computer's current state and figuring out how 
to update it. Previous research has been aware of "the gulf of evaluation" and 
"the gulf of execution", and addressed them by applying metaphors. However, the 
use of non-verbal metaphors has not been fully investigated in developing a 
wide range of applications and enhancing user experience.

This thesis explores the effects of using non-verbal metaphors to reduce the 
mismatches within the communication between humans and computers. To assist 
users in perceiving computer feedback and interpret what it means, we explore 
the effect of using the visual metaphors with familiar objects to interpret, 
introspect, and interact with data. We first present VideoForest, a 
visualization system designed with a tree-like visual summary to support video 
analysis augmented with danmu data. We then examine a postcard metaphor to 
facilitate self-reflection on social media food posts. Next, we experiment with 
metaphorical biofeedback designs to support user engagement during a guided 
stress management practice. To assist users in manipulating computer controls, 
we use embodied metaphors to help designers create meaningful interactions with 
everyday gestures. We introduce Metaphoraction, a creativity support tool that 
formulates interaction design ideas to extend gesture meanings.

To sum up, this thesis takes metaphors' target domain as the way to represent a 
computer's functioning and the source domain as the users' mental images with 
real-world references. The fundamental idea is to investigate the effects of 
non-verbal metaphors in reducing the gulfs, which includes conveying data 
insights, increasing engaging experiences, providing pleasant persuasion, and 
support creative ideation. The studies prove that using metaphors can move the 
represented model closer to the user's mental model, and the resulting 
conclusions and design considerations benefit futuristic digital communication 
between humans and computers.


Date:			Thursday, 16 July 2020

Time:			2:00pm - 4:00pm

Zoom Meeting:		https://hkust.zoom.us/j/98127511433

Chairman:		Prof. Jianzhen YU (CHEM)

Committee Members:	Prof. Xiaojuan MA (Supervisor)
   			Prof. Huamin QU
   			Prof. Sai Kit YEUNG
   			Prof. Ravindra GOONETILLEKE (ISD)
   			Prof. Xiaoru YUAN (Peking University)


**** ALL are Welcome ****