Constraint Modeling in Natural Language Processing in the Era of Deep Learning

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Constraint Modeling in Natural Language Processing in the Era of Deep
Learning"

By

Mr. Zizheng LIN


Abstract:

The remarkable success of deep learning in Natural Language Processing (NLP) 
stems from neural models' ability to learn complex patterns from vast amounts 
of data. However, this data-driven paradigm often struggles to incorporate 
prior knowledge—ranging from linguistic rules and domain expertise to 
commonsense facts and logical structures—a limitation that becomes 
particularly severe in data-scarce scenarios or tasks requiring rigorous 
adherence to structural and semantic dependencies. This thesis investigates 
constraint modeling, a paradigm that addresses these limitations by 
integrating prior knowledge into models through explicit constraints.

The central idea of this dissertation is that constraint modeling, when 
strategically applied at different stages of the NLP pipeline, provides a 
versatile and powerful framework for enhancing models' performance, 
robustness, and generalizability. We organize our study around three distinct 
paradigms: (1) imposing constraints on learning, where constraints are 
integrated directly into the training objective to regularize the parameter 
space; (2) imposing constraints on inference, where constraints serve as a 
global reasoning layer to ensure outputs adhere to logical or domain-specific 
rules; and (3) imposing constraints on prompts, where instructions specifying 
constraints are inserted to explicitly guide the generation processes of 
Large Language Models (LLMs). We substantiate this taxonomy through three 
original contributions.

First, we present HMTGIN to address heterogeneous multi-task learning in 
community question answering. HMTGIN imposes explicit cross-task constraints 
derived from prior knowledge directly into the joint learning objective, 
demonstrating significant performance improvements across diverse tasks.

Second, we propose GCPEAC, an approach that leverages global constraints with 
prompting for zero-shot event argument classification. GCPEAC adopts 
constraints that encode cross-task, cross-argument, and cross-event relations 
to regularize initial predictions, achieving state-of-the-art zero-shot 
performance without any training.

Third, we develop CCoToM for enhancing theory-of-mind reasoning in LLMs. 
CCoToM adaptively inserts natural language instructions specifying 
definitional and dependency constraints into prompts at each reasoning step, 
enabling more robust and accurate theory-of-mind reasoning in LLMs.


Date:                   Wednesday, 17 June 2026

Time:                   10:00am - 12:00noon

Venue:                  Room 3494
                        Lifts 25/26

Chairman:               Dr. Shenghui SONG (ECE)

Committee Members:      Dr. Yangqiu SONG (Supervisor)
                        Dr. Wei WANG
                        Dr. Binhang YUAN
                        Prof. Can YANG (MATH)
                        Prof. Sinno Jialin PAN (CUHK)