IMPROVING DEEP CONVERSATIONAL MODELS VIA INPUT AUGMENTATION AND DATA SOURCE EXPANSION

PhD Thesis Proposal Defence


Title: "IMPROVING DEEP CONVERSATIONAL MODELS VIA INPUT AUGMENTATION AND 
DATA SOURCE EXPANSION"

by

Mr. Zhiliang TIAN


Abstract:

Conversational models aim to generate readable textual responses to input 
queries. In recent years, conversational models are typically built using 
deep neural networks, which are called deep conversational models (DCMs). 
DCMs can be used for task-oriented dialogues or chit-chat conversations. 
In this thesis, we investigate ways to enhance DCMs for chit-chat 
conversations via input augmentation and data source expansion.

DCM maps an input query to some output responses. It has been observed in 
previous work that the output is often uninformative and lacks diversity 
due to two reasons: (1) The input does not contain sufficient information 
to determine an appropriate output, and (2) the DCM is trained via 
likelihood maximization and hence captures only the most salient 
input-output relationships. One common method to address the first issue 
is to include background documents as additional inputs to the DCM, and 
one popular way to deal with the second issue is to include retrieved 
responses to similar input queries as additional inputs to the DCM. In 
this thesis, we advance the state-of-the-art in both of those two lines of 
work.  For the first line of work, we propose an output-anticipated memory 
module to enable the DCM to better attend to the relevant information in 
the background documents. For the second line of work, we develop a memory 
module to extract relationships between clusters of similar inputs and 
clusters of outputs (which are more robust than relationships between 
individual inputs and outputs), and use the relationships to improve the 
performance of the DCM.

Nowadays, DCMs are often trained on huge corpora. However, there are still 
scenarios with low resources. One example is an online chatbot that needs 
to quickly adapt to a new user after a few rounds of conversations. Our 
third contribution in this thesis is a meta-learning based method to help 
with the adaption by utilizing data from the user’ friends, who are 
expected to have similar interests and expectations. In the future, we 
plan to investigate how to utilize private data while ensuring the privacy 
of sensitive information in the private data.


Date:			Wednesday, 23 February 2022

Time:                  	10:30am - 12:30pm

Zoom Meeting:		https://hkust.zoom.us/j/7491359443

Committee Members:	Prof. Nevin Zhang (Supervisor)
  			Dr. Brian Mak (Chairperson)
 			Prof. Fangzhen Lin
 			Dr. Yangqiu Song


**** ALL are Welcome ****