MULTI-TASK LEARNING FOR AUTOMATIC SPEECH RECOGNITION

PhD Thesis Proposal Defence


Title: "MULTI-TASK LEARNING FOR AUTOMATIC SPEECH RECOGNITION"

by

Mr. Dongpeng CHEN


Abstract:

It is well-known in machine learning that multi-task learning (MTL) can 
help improve the generalization performance of singly learning tasks if 
the tasks being trained in parallel are related, and the effect is more 
prominent when the amount of training data is relatively small. Recently, 
deep neural network (DNN) has been widely utilized as acoustic model in 
ASR. We propose applying MTL on DNN to exploit extra information from the 
training data, without requiring additional language resources, which is 
great benefit when language resources are limited. In the first method, 
phone and grapheme models are trained together within the same acoustic 
model and the extra information is the phone-to-grapheme mappings, while 
in the second method, a universal phone set (UPS) modeling task is learned 
with language-specific triphones modeling tasks to help implicitly map the 
phones of multiple languages. Although the methods were initially proposed 
for low-resource speech recognition, we also generalized and applied the 
idea to large vocabulary speech recognition tasks. Experiment results on 
three low-resource South African languages in the Lwazi corpus, the TIMIT 
English phone recognition task and the Wall Street Journal English reading 
speech recognition tasks show the MLT-DNNs consistently outperform 
single-task learning (STL) DNN.


Date:			Tuesday, 12 May 2015

Time:                   2:00pm - 4:00pm

Venue:                  Room 5503
                         lifts 25/26

Committee Members:	Dr. Brian Mak (Supervisor)
  			Prof. Nevin Zhang (Chairperson)
 			Prof. Dit-Yan Yeung
  			Dr. Raymond Wong


**** ALL are Welcome ****