Automatic Generation of Vector Maps by Imitation Learning for Autonomous Driving

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering

PhD Thesis Defence

Title: "Automatic Generation of Vector Maps by Imitation Learning for 
Autonomous Driving"


Mr. Zhenhua XU


The vector map of road elements is critical for autonomous driving. It contains 
the connectivity and topology information of the road for multiple downstream 
tasks of the autonomous vehicles, such as prediction, planning and motion 
control. To assure the normal and safe operations of autonomous vehicles, 
vector maps, including the SD (standard-definition) map and the HD 
(high-definition) map need to be pre-built before the deployment of autonomous 
vehicles in the target region. However, manually annotating the vector map is 
labor-intensive, expensive and time-consuming, which severely restricts the 
fast application and deployment of the autonomous vehicle. Therefore, automatic 
approaches to effectively and efficiently generate the vector map of the target 
region are needed. The source data may come from high-resolution aerial images 
or vehicle-mounted sensors, such as cameras or LiDARs. Aerial images can 
directly provide high-quality BEV (bird's-eye view) images around the vehicle, 
while additional steps for perspective transformation are required for 
vehicle-mounted sensors. Previous works on our task could be classified into 
three categories: segmentation-based approaches, two-stage graph-based 
approaches and decision-making graph-based approaches. Compared with 
segmentation-based approaches and two-stage graph-based approaches, the 
decision-making graph-based approaches are more powerful and stable. Therefore, 
this thesis mainly focuses on decision-making graph-based approaches. These 
approaches are trained by imitation learning.

In this thesis, we will discuss three decision-making graph-based systems for 
vector map generation with details. First, we propose to detect the graph of 
road curbs from small aerial image patches with a decision-making network 
trained by imitation learning. Road curbs possess simple topology without 
complicated intersections or overlapping. Then, we manage to detect the graph 
structure of the road network, whose topology is much more complicated than 
road curbs and road boundaries, such as intersections of arbitrary numbers of 
roads and overpasses. A decision-making system based on the transformer is 
proposed to generate the vector map vertex by vertex. The proposed system can 
handle road networks with any topological changes, which presents 
state-of-the-art performance compared with previous works. Finally, we adapt 
and further improve our approaches to generate the vector map of road lane 
centerlines with data collected by vehicle-mounted sensors. We first convert 
the data from perspective views to BEV by perspective transformation. And then 
generate the global consistent vector map by the proposed iterative 
vectorization approach. The proposed system achieves state-of-the-art 
performance in evaluation experiments. In the end, we analyze the remaining 
problems that cannot be well solved by existing approaches as potential future 
works, such as the map update problem, adapting more advanced decision-making 
algorithms to our task, and graph merge.

Date:			Friday, 6 January 2023

Time:			2:00pm - 4:00pm

Venue:			Room 3494
 			lifts 25/26

Chairperson:		Prof. Zhihong GUO (CHEM)

Committee Members:	Prof. Ming LIU (Supervisor)
 			Prof. Huamin QU (Supervisor)
 			Prof. Qifeng CHEN
 			Prof. Qiong LUO
 			Prof. Yansong YANG (ECE)
 			Prof. Yong WANG (Harbin Institute of Technology)

**** ALL are Welcome ****