Serendipitous Togetherness: Multimodal Designs for Shared Experiences across Time and Space

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Serendipitous Togetherness: Multimodal Designs for Shared Experiences 
across Time and Space"

By

Mr. Zeyu HUANG


Abstract:

Human beings, as social animals, continually seek various shared experiences 
to connect and communicate with each other, especially after the COVID-19 
pandemic substantially altered our social dynamics. This thesis focuses on 
serendipitous shared experiences—unplanned and ad-hoc interactions that 
offer social opportunities and create casual, poetic interpersonal 
connections. Despite its significance, designing such experiences for 
temporally or spatially separated individuals remains challenging compared to 
spontaneous co-located interactions.

To bridge this gap, I demonstrate how digital interventions can support 
serendipitous shared experiences through creative and multimodal interaction 
design. My research comprises four interconnected works structured around 
three conceptual pillars: (1) creating connections across temporal and 
spatial distributedness; (2) leveraging locations—both geographical and 
metaphorical—as anchors for social experiences; and (3) tailoring multimodal 
experiences to the unique affordances and limitations of each context.

In particular, two studies explored geographical locations-based designs. The 
first work fosters the communal construction of values around the campus 
space. To connect campus residents across time, a mobile application enables 
contribution to and exploration of geocached collective memory, offering 
locative and immersive narratives. The second one focuses on scenic live 
streaming of a specific scenic site and its spatially distributed online 
audience. Leveraging Augmented Reality, I deliver immersive interactions that 
constitute a multi-user experience with local scenic and cultural contexts. 
The other two studies focused on the cases where the interactive medium 
itself brings people together. The third work foregrounds the temporally and 
physically separated viewers of online videos. Conceptualizing the online 
video as a metaphorical location where people gather, I enable viewers to 
share "goosebump feelings" during video consumption with a haptic-visual 
design. The non-intrusive affective communication successfully creates 
emotional contagion and togetherness. The final study uses hammocks at 
public spaces as unexpected location anchors for shared experience. To 
connect facility users across time, interactive bodily traces of previous 
occupants are captured and re-emerge through haptic feedback. The embodied 
experience succeeds in establishing poetic and engaging connections among 
strangers.

This thesis reveals a rich design space where technology transcends 
conventional face-to-face interpersonal interaction paradigms. By 
facilitating serendipitous shared experiences with meaningful resonance, 
emotional synchrony, and sensory engagement, it advances our understanding of 
how subtle yet profound human connections can be fostered across the 
boundaries of time and space.


Date:                   Friday, 14 November 2025

Time:                   9:30am - 11:30am

Venue:                  Room 3494
                        Lifts 25/26

Chairman:               Dr. Qing CHEN (MAE)

Committee Members:      Dr. Xiaojuan MA (Supervisor)
                        Dr. Brian MAK
                        Dr. Arpit NARECHANIA
                        Prof. Richard SO (IEDA)
                        Dr. Kening Ken ZHU (CityU)