Data-Based Knowledge Transfer for Large-Small Model Collaboration: From Chain-of-Thought Reasoning to Knowledge Graphs and Executable Skills

PhD Thesis Proposal Defence


Title: "Data-Based Knowledge Transfer for Large-Small Model Collaboration: 
From Chain-of-Thought Reasoning to Knowledge Graphs and Executable Skills"

by

Mr. Shuoling LIU


Abstract:

Large language models (LLMs) provide strong generalization and reasoning 
capabilities, but they are costly to deploy and are often accessible only 
through black-box application programming interfaces. Small language models 
(SLMs) can be deployed locally with lower cost and greater control over 
sensitive data, yet they generally lack broad knowledge and robust reasoning. 
This thesis investigates data-based knowledge transfer as a practical 
foundation for collaboration between server-side LLMs and client-side SLMs 
when model parameters, logits, and gradients are unavailable. It develops a 
unified progression from reasoning, to knowledge, to action through three 
transferable artifacts: chain-of-thought (CoT) traces, knowledge graphs 
(KGs), and executable skills.

First, the thesis studies multi-teacher CoT distillation for domain- 
specialized SLMs. Complementary reasoning and non-reasoning teachers generate 
candidate traces, and an LLM judge selects high-quality supervision for 
training a local financial expert. Second, it externalizes domain knowledge 
into KGs that can be updated without retraining. The proposed LLM-KG 
coordination methods canonicalize ambiguous queries, retrieve and fuse 
evidence across heterogeneous sources, and move frequent pipeline operations 
to compact local models. Third, the proposed chapter extends large-small 
collaboration from models to agents through federated executable skill 
learning. Client agents retain private tasks and trajectories locally, 
exchange only sanitized reusable skills, and combine an execution-validated 
global skill library with client-private overlays. Structure-aware 
consolidation and sandbox execution are used to ensure that redistributed 
skills remain coherent and runnable.

The remaining work will integrate graph queries into executable skills, 
validate skills across clients with heterogeneous local graphs and tool 
configurations, and evaluate privacy leakage from knowledge-bearing skill 
templates. Overall, the thesis aims to establish a data-based framework that 
progressively equips local SLMs with reasoning, updatable knowledge, and 
actionable procedures, while explicitly characterizing the trade-offs among 
effectiveness, efficiency, and privacy.


Date:                   Tuesday, 27 October 2026

Time:                   3:00pm - 4:30pm

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Prof. Qiang Yang (Supervisor)
                        Prof. Kai Chen (Co-supervisor)
                        Dr. Yangqiu Song (Chairperson)
                        Dr. Binhang Yuan