Natural Language Processing: A Neural Network View

PhD Qualifying Examination


Title: "Natural Language Processing: A Neural Network View"

by

Mr. Yuchen YAN


Abstract:

We survey modern neural network models in Natural Language Processing 
(NLP), which has made drastic breakthroughs in the field. Neural network 
can model language better than statistical N-gram models, due to the 
special structure of recurrent neural network (RNN). The RNN-based 
architecture can further be extended to handle machine translation. In 
fact, this is the main building block of Google Translation and Microsoft 
Translation. From their models, we illustrate that the alignment problem 
is the bottleneck. Luckily, such problem may be solved by our next model: 
TRAAM.

Neural network can model grammar as good as our hand-crafted grammar 
rules. The model can also be extend to model bi-lingual grammar. Since 
there are no hand-crafted bi-lingual grammar rules, the representational 
power of neural network becomes valuable. In this survey we show that 
TRAAM, bi-lingual grammar augmented with neural network, can be considered 
as a good candidate for the alignment solution.

The ultimate goal of natural language processing to understand natural 
language. Semantic Role Labeling (SRL) is a structural representation of 
semantics. In this survey we study the current best neural network parser: 
SENNA, and compare it against the commonly used parser, ASSERT (which is a 
statistical one). We show the limitation of SENNA, and how it can be 
improved by applying RNN. On the other hand, SRL is a hand-crafted 
structure, although it is carefully designed, we can still go beyond it. A 
recent neural network architecture, generative adversarial network (GAN), 
is capable of generating sentences from random noise. In this survey we 
analyze its potential in semantics representation.

We show the strengths and reveal the limitations of neural networks by 
studying their applications in machine translation, and analyzing their 
performances in semantics representation. Interestingly, in both machine 
translation and semantics representation, generative networks, TRAAM and 
GAN, are showing stronger potential than discriminative networks. 
Generative networks learn patterns from unlabeled data; in contrast, 
discriminative networks have to be provided correct solutions, like 
syntactic tree, or SRL parse. With generative networks, there are still 
plenty of open questions waiting to be solved, plenty of opportunities 
lying to be seized.


Date:			Wednesday, 16 August 2017

Time:                  	2:00pm - 4:00pm

Venue:                  Room 2612A
                         Lifts 31/32

Committee Members:	Prof. Dekai Wu (Supervisor)
 			Prof. Fangzhen Lin (Chairperson)
 			Prof. Andrew Horner
 			Prof. Nevin Zhang


**** ALL are Welcome ****