ConstraintChecker: A Plugin for Large Language Models to Reason on Commonsense Knowledge Bases

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "ConstraintChecker: A Plugin for Large Language Models to Reason on 
Commonsense Knowledge Bases"

By

Mr. Van Quyet DO


Abstract:

Reasoning over Commonsense Knowledge Bases (CSKB), i.e. CSKB reasoning, has 
been explored as a way to acquire new commonsense knowledge. Despite the 
advancement of Large Language Models (LLM) and prompt engineering techniques in 
various reasoning tasks, they still struggle to deal with CSKB reasoning. One 
challenge for them is to acquire explicit relational constraints in CSKBs from 
only in-context exemplars, due to their lack of symbolic reasoning 
capabilities. In this thesis, we propose ConstraintChecker, a 
symbolic-reasoning plugin over baseline prompting techniques to provide and 
check explicit constraints. When considering a new knowledge instance, 
ConstraintChecker employs a rule-based module to produce a list of constraints, 
then it uses a zero-shot learning module to check whether this knowledge 
instance satisfies all constraints. The acquired constraint-checking result is 
then aggregated with the output of the main prompting technique to produce the 
final output. Experimental results on CSKB Reasoning benchmarks demonstrate the 
effectiveness of our method by bringing consistent improvements over all 
baseline prompting techniques.


Date:                   Wednesday, 12 June 2024

Time:                   4:00pm - 6:00pm

Venue:                  Room 3494
                        Lifts 25/26

Chairman:               Dr. Long CHEN

Committee Members:      Dr. Yangqiu SONG (Supervisor)
                        Dr. Xiaojuan MA