3D SEMANTIC SEGMENTATION OF INDOOR AND OUTDOOR SCENES

PhD Thesis Proposal Defence


Title: "3D SEMANTIC SEGMENTATION OF INDOOR AND OUTDOOR SCENES"

by

Mr. Zeyu HU


Abstract:

3D semantic segmentation serves as an indispensable cornerstone of various 
downstream applications like AR/VR, robotics and autonomous vehicle. It 
aims to parse a scene and assign a class label to each 3D point providing 
point-wise perceptional information for a thorough 3D scene understanding.

According to the scanning environments, 3D scenes can be roughly divided 
into two categories: indoors and outdoors. Indoor scenes have small ranges 
and closely located objects. They are generally reconstructed by RGB-D 
scanners with densely captured point clouds making point edge extraction 
and accurate mesh reconstruction possible. In contrary, outdoor scenes 
have much wider scanning ranges than indoor scenes and are thus often 
measured by Light Detection and Ranging (LiDAR) sensors, which provide 
continuous LiDAR frames consisting of sparse point clouds. This 
dissertation presents methods that provide improvements to the state of 
the art in 3D semantic segmentation of both indoor and outdoor scenes 
leveraging their unique properties.

More specifically, first, we take advantage of the densely captured point 
clouds of indoor scenes to extract meaningful point edges. Leveraging the 
duality between segmentation and edge detection tasks, we propose a novel 
framework for joint learning of semantic segmentation and semantic edge 
detection bringing joint improvements to both tasks. Second, we further 
exploit the geodesic information embedded in the reconstructed meshes of 
indoor scenes. A novel deep architecture, which operates on the voxel and 
mesh representations, is proposed to leverage both the Euclidean and 
geodesic information for geodesic-aware 3D semantic segmentation. Finally, 
we focus on the continuous scanning pattern of LiDAR sensors used for 
outdoor scenes. To reduce the annotation costs, we propose a novel active 
learning strategy for 3D LiDAR semantic segmentation by estimating model 
uncertainty based on the inconsistency of predictions across frames.


Date:			Thursday, 12 May 2022

Time:                  	3:00pm - 5:00pm

Zoom Meeting:		https://hkust.zoom.us/j/4715932509

Committee Members:	Prof. Chiew-Lan Tai (Supervisor)
  			Dr. Minhao Cheng (Chairperson)
 			Dr. Qifeng Chen
 			Prof. Pedro Sander


**** ALL are Welcome ****