A Survey on Knowledge-driven Autonomous Driving

PhD Qualifying Examination


Title: "A Survey on Knowledge-driven Autonomous Driving"

by

Mr. Zhili CHEN


Abstract:

The burgeoning of deep learning algorithms and computational power accelerate 
the process toward full automation for the driving system. Both academics and 
industry have made massive efforts to enhance the robustness of autonomous 
driving systems. The achievements are also attributed to the construction of 
high-quality large benchmark datasets, like KITTI, nuScenes, Waymo, etc. 
However, these data-driven autonomous driving systems are likely to fail when 
encountering data not seen in the training set. It is costly to collect and 
impossible to cover various kinds of corner case data for training. Moreover, 
it is hard to identify and completely resolve the root cause of the failure 
cases by modeling because the systems lack interpretability. Recently, the 
large foundation models learned from web-scale data have presented impressive 
human-like generalization and reasoning capabilities. It provides a potential 
solution for solving the long-tail challenge in autonomous driving by 
leveraging their learned world knowledge.

This survey provides reviews of the recent advancements that incorporate 
foundation models into various tasks of autonomous driving. We started by 
introducing the knowledge-augmented autonomous driving benchmark datasets and 
their evaluation metrics. We then give reviews on the related large language 
model (LLM) and the vision-language model (VLM). Next, we present a 
comprehensive survey of the recent works that incorporate knowledge in boosting 
the performance of various tasks in autonomous driving. Subsequently, we delve 
into the works that explore equipping the end-to-end paradigms with human-like 
ability by using the LLM/VLM. Finally, we conclude this survey with discussions 
on the encountering limitations and future research directions.


Date:                   Wednesday, 17 July 2024

Time:                   4:00pm - 6:00pm

Venue:                  Room 3494
                        Lifts 25/26

Committee Members:      Dr. Qifeng Chen (Supervisor)
                        Dr. Long Chen (Chairperson)
                        Dr. Hao Chen
                        Dr. Junxian He