Reconstruct the world in 3D from images

Speaker:        Dr. Tian FANG
                Department of Computer Science and Engineering
                Hong Kong University of Science and Technology

Title:          "Reconstruct the world in 3D from images"

Date:           Friday, 19 September 2014

Time:           11:00am - 12 noon

Venue:          Room 1504 (near lifts 25/26), HKUST

Abstract:

3D Reconstruction from images is a fundamental problem in computer vision.
A general pipeline of three steps is used for that. First, local relative
camera poses and 3D points are recovered from images by solving minimal
multiple view geometry problems with sparse image feature matching. Then,
such local reconstructed cameras and 3D points are merged into a global
reconstruction optimized by bundle adjustment. Finally, stereo
reconstruction is carried out to generate dense point clouds and surfaces.
Although the principles behind these techniques have been studied
exhaustively over the past decades, the non-linear properties of the
optimization and the complex connections in the parameter network make it
very challenging to reconstruct the world in 3D from millions of images.
In this talk, a series of works on adaptively resampling and partitioning
the reconstruction into smaller problems are presented to handle the
large-scale 3D reconstruction. First, a stochastic sampling strategy is
presented to resample the redundant 3D points according to quality
assessment scores without compromising the quality of the local
reconstruction. Then such an idea is extended to the space of cameras.
Based on an image quality graph, a graph simplification procedure
maintaining the accuracy of the estimated cameras and completeness of the
reconstruction is carried out to remove the redundant and bad cameras,
yielding a faster and more robust global bundle adjustment. Such global
bundle adjustment provides a global consistent coordinate frame for dense
point and surface reconstruction, but global bundle adjustment usually
converges to a non-zero residual and produces sub-optimal camera poses for
local areas, which leads to loss of details for high-resolution
reconstruction. Instead of trying harder to optimize everything globally,
we propose a segment-based approach to readjust the camera poses locally
and improve the reconstruction for fine geometry details. In the end,
several photo realistic results of city-scale 3D reconstruction will be
demonstrated in the talk.


**********************
Biography:

Dr. Tian FANG now is a post-doctoral fellow in the Department of Computer
Science and Engineering, the Hong Kong University of Science and
Technology, where he received the Ph.D. degree in 2011 under the
supervision of Prof. Long Quan. His research interests are computer vision
and graphics, especially in real-time SLAM and large-scale 3D
reconstruction from images. His research papers are published in the top
journals and conferences, e.g. ACM TOG, IEEE TVCG, IEEE TRGS, IEEE TIP,
ECCV and CVPR. His collaborative work with Beijing Normal University on
"Theories and Methods of Earth Surface Feature Modeling and Visualization
Based on Multi-Sensor Spatial Data" won a second class award in Higher
Education Outstanding Scientific Research Output Award (Natural Science)
presented by Ministry of Education, China. His inventions with colleagues
were filed as four US patents, two of which were granted. He also serves
as reviewers in major top journals and conferences, including IJCV, ACM
TOG, and SIGGRAPH/SIGGRAPH Asia. He is now coordinating a team on
reconstructing 3D city models from images, inducing wide ranges of
research projects such as large-scale stereo, joint 2D-3D semantic
segmentation, model abstraction and vectorization, surface reconstruction,
mesh processing, real-time rendering, and distributed computing.