Accurate and Realistic Dense 3D Reconstruction from Multiple Images

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Accurate and Realistic Dense 3D Reconstruction from Multiple 
Images"

By

Mr. Jingyang ZHANG


Abstract

3D modeling is an important task to preserve and visualize real-world 
scenes by computer. The applications include but are not limited to 
heritage preserving, city-scale surveys and AR/VR applications. Typical 
image-based 3D reconstruction methods include SfM, MVS, meshing and 
texturing. This thesis aims at improving the last three steps with neural 
network techniques so that the reconstruction pipeline is capable to 
produce a 3D model with high accuracy and realistic appearance. First, we 
introduce visibility handling into learning-based stereo matching systems 
to improve the quality of estimated depth maps. We propose to explicitly 
detect the visibility and recover the erroneous pixels by neighborhood or 
other views. Second, we adopt differentiable rendering in neural implicit 
surface optimization to simultaneously obtain accurate geometry and 
realistic appearance. We investigate effective geometric prior and 
critical regularizations to improve the ability of generalization and 
robustness of the system. Third, we decompose the appearance into 
environmental lighting and physical-based material to support efficient 
rendering in arbitrary novel environments. We discuss possible techniques 
to reduce the ambiguity between environment and material, and provide an 
approximated indirect illumination handling method to improve the 
estimation quality in complex scenes. The proposed modules are extensively 
evaluated on multiple datasets, including both synthetic and real-world 
data.


Date:			Thursday, 14 April 2022

Time:			2:30pm - 4:30pm

Zoom Meeting: 
https://hkust.zoom.us/j/91287553271?pwd=SFpWYnFwODdOeldOa2o1WWRyS2NCZz09

Chairperson:		Prof. Yilong HAN (PHYS)

Committee Members:	Prof. Long QUAN (Supervisor)
 			Prof. Qifeng CHEN
 			Prof. Pedro SANDER
 			Prof. Kai TANG (MAE)
 			Prof. Yizhou YU (HKU)


**** ALL are Welcome ****