A Survey of LLM Agents for Vulnerability Discovery

PhD Qualifying Examination


Title: "A Survey of LLM Agents for Vulnerability Discovery"

by

Mr. Yuchong XIE


Abstract:

Software vulnerabilities remain the dominant root cause of security 
incidents, yet the classical pillars of automated vulnerability discovery 
(fuzzing, static analysis, and symbolic execution) each trade off precision, 
coverage, and the semantic understanding needed to reason about code the way 
a human auditor does. The emergence of large language models (LLMs), and in 
particular LLM agents that plan, invoke tools, execute code, and act over 
multiple steps, has opened a new and rapidly expanding line of research that 
reframes vulnerability discovery as an interactive, goal-directed task. This 
survey provides a focused review of LLM agents for vulnerability discovery, 
which we define as the problem of finding previously unknown or unconfirmed 
security flaws and confirming that they are real, spanning code and binary 
auditing, vulnerability detection and localization, and the generation of 
proofs-of-vulnerability (PoVs) that reproduce a flaw.

We organize the field along four axes that mirror how research on agentic 
discovery actually progresses. We first survey the benchmarks that define the 
task and measure progress, from repository-level detection datasets to 
real-world PoV benchmarks built on patched CVEs, and we highlight recurring 
methodological pitfalls such as data contamination and unverifiable success 
signals. We then review agent scaffolds: the single-agent and multi-agent 
architectures, tool integrations, and reasoning workflows that turn a 
general-purpose LLM into a vulnerability hunter. We next examine the nascent 
but growing body of work on training models specifically for discovery, 
including specialized reasoning models and reinforcement learning with 
verifiable rewards. We close with an analysis of open challenges and future 
directions. We argue that the ability to build, execute, and verify targets 
is the scarce resource that jointly underwrites trustworthy confirmation, 
honest multilingual and execution-grounded evaluation, and effective 
training, and that the most consequential shift ahead is to treat discovery 
as an open-ended, continuous task rather than a single-target episode, all 
under a dual-use tension that responsible progress cannot ignore. Our aim is 
to give a clear map of a young but fast-moving field and to identify where 
the most consequential open problems lie.


Date:                   Thursday, 30 July 2026

Time:                   1:00pm - 3:00pm

Venue:                  Room 2408
                        Lift 17/18

Committee Members:      Dr. Dongdong She (Supervisor)
                        Dr. Shuai Wang (Co-supervisor)
                        Dr. Mingxun Zhou (Chairperson)
                        Dr. Binhang Yuan