From Long-Horizon Coding to Recursive Self-Improvement: A Survey of LLM Agents for AI Development

PhD Qualifying Examination


Title: "From Long-Horizon Coding to Recursive Self-Improvement: A Survey of 
LLM Agents for AI Development"

by

Mr. Junteng LIU


Abstract:

Large language model (LLM) agents are moving from generating individual 
functions to sustained development across large codebases. Their ability to 
implement, test, and revise AI systems provides a foundation for recursive 
self-improvement (RSI). This survey traces the progression from long-horizon 
coding to agent-driven AI development and RSI. We first review coding 
benchmarks, agent harnesses, context management, and training methods that 
support extended development tasks. We then examine how agents improve models 
and training procedures, evolve their harnesses, and optimize inference 
systems, with particular attention to how resulting changes support subsequent 
improvement cycles. Across these settings, we compare feedback signals ranging 
from binary correctness checks to continuous measures of quality and 
efficiency, and discuss evaluation protocols that account for computational 
budgets and generalization beyond development tasks. Finally, we identify open 
challenges in evaluating agents across broader AI and systems workloads and 
training them to conduct increasingly long-horizon research and development.


Date:                   Tuesday, 27 October 2026

Time:                   1:30pm - 2:30pm

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Dr. Junxian He (Supervisor)
                        Dr. May Fung (Chairperson)
                        Dr. Long Chen