EVALUATION AND APPLICATIONS OF MEANING REPRESENTATION

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "EVALUATION AND APPLICATIONS OF MEANING REPRESENTATION"

By

Miss Ziyi SHOU


Abstract:

Understanding natural language facilitates communication with humans, 
representing a significant milestone in the field of artificial intelligence. 
Thus, comprehending natural language has been a persistent aim in the domain of 
natural language processing. Meaning representations (MRs) act as a connection 
between linguistic expressions and the underlying meaning of the words used, 
encoding the meaning of language into a discrete, hierarchical structured 
graph. Their interpretability and ease to use, both for machines and human, 
have contributed to their popularity in the research field.

Despite the extensive work on MR parsing, research on MR evaluation has 
considerably trailed behind. A dependable metric is crucial for the design and 
evaluation of meaning representation parsers, as it facilitates the comparison 
of the disparity between the outputs from MR parsers and golden annotations. 
Inspired by plain-text automatic similarity assessment methods, we first 
propose a novel metric for efficient similarity evaluation using 
self-supervised learning methods. Our proposed metric demonstrates substantial 
enhancements in correlating with human semantic scores and maintains robustness 
under diverse challenges.

Secondly, we investigate the potential applications of meaning representation. 
Leveraging the flexibility and modifiability inherent in meaning 
representation, we parse sentences to these representations. These 
representations can then undergo a series of modifications, resulting in a an 
extensive dataset of paraphrased sentences without the need to retrain the 
decoder. Experimental results show that the effectiveness of our data 
augmentation approach using meaning representations in improving performance 
across various downstream tasks.

As we advance towards multimodal models, we investigate the potential 
application of meaning representation in this domain. Vision-language models 
have been criticized for performing akin to a bag-of-words models, lacking 
nuanced semantic understanding. To address this, we modify the structure of 
meaning representation and create negative samples that possess entirely 
different meanings but share close plain paraphrases. Subsequently, 
vision-language models are trained to distinguish between true labels and our 
generated negative samples. Our results indicate that incorporating negative 
samples utilizing meaning representations enhances the models' performance in 
tasks involving attribute and relation understanding.


Date:                   Tuesday, 26 March 2024

Time:                   3:30pm - 5:30pm

Venue:                  Room 2127A
                        Lift 19

Chairman:               Prof. Xueqing ZHANG (CIVL)

Committee Members:      Prof. Fangzhen LIN (Supervisor)
                        Prof. Ke YI
                        Prof. Dan XU
                        Prof. Ling PAN
                        Prof. Weichuan YU (ECE)
                        Prof. Jiamin JI (CUHK)