A Survey on Open-World Spatial Perception and Reasoning

PhD Qualifying Examination


Title: "A Survey on Open-World Spatial Perception and Reasoning"

by

Mr. Yang CAO


Abstract:

Open-World spatial perception and reasoning is emerging as a core capability 
for AR, VR, and embodied AI, where an agent needs to localize, recognize, and 
reason about objects in 3D space beyond fixed categories, sensors, or query 
types. Riding on foundation models such as CLIP, VGGT, and multimodal LLMs, 
recent research has advanced 3D spatial understanding along three 
complementary axes. A vocabulary axis pursues open-vocabulary 3D object 
detection beyond a fixed category set; a sensor axis moves toward 
sensor-geometry-free detection that reduces reliance on calibrated multi-view 
poses or depth, enabling broader deployment; and a semantic axis targets 
fine-grained spatial reasoning over relational and compositional queries. 
Together they trace a progression from closed to open perception and from 
perception to cognition. We first formalize the tasks and their core 
challenges: category openness, minimal sensor assumptions, and generalization 
to complex queries. We then organize detection methods by input modality 
(multi-view RGB, point clouds) and degree of vocabulary openness, categorize 
spatial reasoning by modeling framework and training strategy, and introduce 
datasets and evaluation protocols alongside the methods. We aim to provide a 
common foundation to catalyze advances in open-world spatial perception and 
reasoning.


Date:                   Wednesday, 2 September 2026

Time:                   10:00 - 12:00

Venue:                  Room 3494
                        Lift 25/26

Committee Members:      Dr. Dan Xu (Supervisor)
                        Dr. Qifeng Chen (Chairperson)
                        Dr. Chaojian Li