Towards Crowd-sourced Semantic-based Multimodal User Interfaces

Speaker:        Dr. Xiaojuan MA
                Researcher
                Huawei Noah's Ark Lab.

Title:          "Towards Crowd-sourced Semantic-based Multimodal User
                 Interfaces"

Date:           Monday, 2 February 2015

Time:           4:00pm - 5:00pm

Venue:          Lecture Theater F (near lifts 25/26), HKUST

Abstract:

Communication of information takes place in both spoken and written forms
on user interfaces of desktop, laptop, and mobile devices. However,
everyone may encounter occasions when receiving and expressing information
via a language is difficult. Therefore, people seek alternative ways to
communicate when they fail to retrieve the exact concept. One of the most
common strategies is semantic paraphrasing, which requires the user
interfaces to be equipped with a well-structured, dense knowledge base
with semantic relations among encoded concepts. Traditionally, manual
approaches for constructing knowledge bases may not scale well, while the
automatic approaches via text mining may not explicitly reflect the
organization of human mental lexicon. We propose a strategy that leverages
collective intelligence to augment semantic network with directed,
weighted, cross-part-of-speech associative links. We integrate such
crowd-sourced method with machine learning and natural language processing
techniques to achieve better scalability. Evaluations show that our method
captures new semantic relatedness information, which can better facilitate
sense making and word finding on multimodal user interfaces.

********************
Biography:

Xiaojuan MA, Ph.D., is a researcher at Huawei Noah's Ark Lab. Before
joining Noah's Ark, she was a postdoctoral researcher in the
Human-Computer Interaction Institute at Carnegie Mellon University (CMU),
and was awarded Computing Innovation Fellow by Computing Research
Association. Dr. MA received her Ph.D. degree from the Computer Science
Department of Princeton University in 2010. She worked as a research
fellow in the Department of Information Systems, National University of
Singapore before joining CMU. Dr. MA is the board member and secretary
general of International Chinese Association of Computer Human
Interaction. Her background is in Human-Computer Interaction. She is
particularly interested in human computation and crowdsourcing, multimodal
interfaces for both human-human, human-computer, and human-robot
interactions, design, and (computational) linguistics.