Resource-Bounded and Regression-Safe Evolution of Agent Skills under Non-Stationary Workloads

PhD Qualifying Examination


Title: "Resource-Bounded and Regression-Safe Evolution of Agent Skills under
Non-Stationary Workloads"

by

Mr. Liming ZHANG


Abstract:

Language-model agents increasingly externalize procedural knowledge as reusable
skills. Persistent revisions can improve new tasks while degrading established
behavior, and changing task distributions can alter which capabilities matter.
This survey develops a PhD research framework for resource-bounded skill
evolution, treating regression safety as a conditional objective under explicit
workloads, tolerances, and validity assumptions.

We organize the literature around skill acquisition, retrieval, refinement,
consolidation, and verification, comparing evidence for utility, retention, and
cost. SkillsBench reports that curated skills raise mean task pass rate from
33.9% to 50.5% across 87 tasks and 18 model-harness configurations. Conversely,
a longitudinal pilot in SkillEvoReg reports transfer accuracy declining from
46.67% to 37.50% during later updates to one learned skill trajectory. These
findings motivate separate evaluation of useful adaptation and retained
capabilities, without establishing a universal benefit or failure rate. Existing
replay, source retention, targeted counterexamples, and contract checks offer
protection bounded by their evidence and execution assumptions.

The synthesis defines a staged research agenda. The first study fixes the
candidate updater and executor to test whether validation allocation reduces
harmful admissions at comparable beneficial-update acceptance and total cost.
Subsequent studies address changing task mixtures and the refresh of evidence
invalidated by tool or specification changes. The proposed protocol combines
chronological evaluation, independent audits, unchanged-skill controls, and
explicit protection of selected historical capabilities. Workload shifts change
task demand; environment changes can invalidate execution evidence and require
separate controls. The current contribution is a critical synthesis and
falsifiable evaluation plan. Original performance gains and general safety
guarantees remain to be established. Keywords: Agent Skills; Skill Evolution;
Resource-Bounded Validation; Regression Control; Non-Stationary Workloads.


Date:                   Friday, 23 October 2026

Time:                   11:00am - 12:00noon

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Prof. Kai Chen (Supervisor)
                        Dr. Yangqiu Song (Chairperson)
                        Dr. May Fung