KV-Cache Infrastructure Optimization for LLM Inference Systems

PhD Qualifying Examination


Title: "KV-Cache Infrastructure Optimization for LLM Inference Systems"

by

Mr. Yunxuan ZHANG


Abstract:

Large language models (LLMs) use a key-value cache (KV-cache) to avoid 
recomputing historical attention states during inference. Longer contexts and 
higher request concurrency make cache management a major constraint on 
efficiency. This survey reviews KV-cache infrastructure, covering the 
mechanisms that represent, store, access, and reuse cached keys and values. We 
organize the survey into three categories: representation and compression, 
storage and access, and reuse and execution management. We examine how these 
techniques address capacity pressure, access bandwidth bottleneck, and the 
overhead of constructing or restoring cached state. The review compares 
architectural changes, compression, memory management, offloading, and reuse 
in terms of their benefits, costs, and applicability. We emphasize that 
optimizations can shift bottlenecks and introduce processing, communication, 
or quality trade-offs, motivating coordinated design across model and system 
components. Finally, we discuss future requirements for ultra-long contexts 
and long-lived agentic tasks, including efficient history access and state 
preservation across tool interactions.


Date:                   Wednesday, 28 October 2026

Time:                   11:00am - 1:00pm

Venue:                  Room 5504
                        Lifts 25/26

Committee Members:      Prof. Kai Chen (Supervisor)
                        Dr. Binhang Yuan (Chairperson)
                        Dr. Chaojian Li