From Code to Capability: Data-Centric Training of Coding Agents

PhD Thesis Proposal Defence


Title: "From Code to Capability: Data-Centric Training of Coding Agents"

by

Mr. Ka Shun SHUM


Abstract:

Coding agents have evolved from generating isolated code fragments to 
performing software engineering tasks within repositories and execution 
environments. Their development involves a connected training pipeline that 
transforms coding data into progressively richer capabilities. Pre-training 
learns from large corpora of raw code and technical text, with cleaning, 
deduplication, and selection supporting the acquisition of programming 
knowledge, implementation patterns, and code generation ability. Mid-training 
extends this foundation through longer repository contexts, synthetic editing 
and code-behavior tasks, and multi-turn engineering trajectories, developing 
cross-file understanding, basic editing behavior, and experience with tools. 
Post-training uses curated demonstrations, executable tasks, and outcome and 
process feedback to improve instruction following, task completion, and the 
reliability of agent behavior.

This thesis presents an integrated account of how data is constructed and 
used across pre-training, mid-training, and post-training to build capable 
coding agents. It reviews the technical components of large-scale 
coding-model training, using Qwen3-Coder-Next as a production-scale case 
study, and develops specific contributions within this broader pipeline. 
PreSelect demonstrates a capability-oriented approach to pre-training data 
selection through agreement between cross-model compression rankings and 
downstream performance rankings. World Rollout scales engineering experience 
for mid-training through a hybrid environment that preserves repository state 
and deterministic operations while simulating runtime observations. SWE-RM 
examines how learned outcome rewards support both solution selection and 
reinforcement learning, emphasizing discrimination and calibration alongside 
ranking performance. Complementary process-level work provides rule-based 
labels for useful and harmful turns without per-turn tree search. These 
studies connect foundational data selection, scalable interaction data, and 
outcome and process supervision into a common technical framework, clarifying 
their respective roles and connections across the training pipeline. 
Together, the overview and original studies provide a data-centric account of 
how large-scale training converts available coding data into capable software 
engineering agents.

The unifying perspective is that coding-agent capability emerges through the 
transformation of code into knowledge, interaction into engineering behavior, 
and verification into effective learning signals. The thesis brings corpus 
construction, trajectory generation, and outcome and process supervision into 
a common technical framework, clarifying their respective roles and 
connections across the training pipeline. Together, the overview and original 
studies provide a data-centric account of how large-scale training converts 
available coding data into capable software engineering agents.


Date:                   Tuesday, 20 October 2026

Time:                   9:00am - 11:00am

Venue:                  Room 5504
                        Lifts 25/26

Committee Members:      Dr. Junxian He (Supervisor)
                        Prof. Raymond Wong (Chairperson)
                        Dr. May Fung