Accurate, Scalable and Parallel Structure from Motion

PhD Thesis Proposal Defence


Title: "Accurate, Scalable and Parallel Structure from Motion"

by

Mr. Siyu ZHU


Abstract:

Structure from motion (SfM) is a photogrammetric technique for estimating 
three-dimensional structures from two-dimensional images. Thanks to the 
rapid development of portable photo acquisition equipments cameras and the 
explosion of on-line image collections, large-scale SfM has achieved 
extraordinary progress in the past few years. However, large-scale SfM is 
still challenging in three aspects, namely accuracy, scalability and 
efficiency. The target of this work is to handle highly accurate and 
consistent large-scale SfM problems in a parallel and scalable manner.

First, we tackle the accurate and consistent Structure from Motion (SfM) 
problem, in particular camera registration, far exceeding the memory of a 
single computer in parallel. Different from the previous methods which 
drastically simplify the parameters of SfM, we propose a camera clustering 
algorithm to divide a large SfM problem into smaller sub-problems in terms 
of camera clusters with overlapping while preserving as many connectivity 
among cameras and tracks as possible. We next exploit a hybrid formulation 
leveraging the relative motions from local incremental SfM into a global 
motion averaging framework to produce superior accurate and consistent 
initial camera poses. Our scalable formulation in terms of camera clusters 
is highly applicable to the whole SfM pipeline including track generation, 
local SfM, 3D point triangulation and bundle adjustment, and able to 
reconstruct camera poses of a city-scale data-set containing 665K 
high-resolution images with the state-of-the-art accuracy and robustness 
evaluated on both the benchmark and Internet data-sets.

Then we propose a divide-and-conquer algorithm to solve large-scale motion 
averaging problems in a highly parallel scheme. First, we partition the 
full camera set into clusters in which local SfM is performed to provide 
robust relative motions and initialize global camera poses. Then the full 
motion averaging problem is decoupled into several sub-problems with 
respect to their local coordinate frame encoded by a similarity 
transformation for independent optimization in parallel. Finally, we can 
merge sub-problems globally without caching the whole reconstruction in 
memory at once. A hierarchical system is subsequently proposed not only 
able to solve large-scale motion averaging problems including one 
consisted of 665K images in an inherently parallel scheme but also 
simplifies challenging translation averaging to a well-posed similarity 
averaging problem. Experiments on benchmark and Internet data-sets confirm 
that our unified system improves accuracy over the state-of-the-art 
methods with comparable efficiency.

Global bundle adjustment usually converges to a non-zero residual and 
produces sub-optimal camera poses for local areas, which leads to loss of 
details for high-resolution reconstruction. Instead of trying harder to 
optimize everything globally, we argue that we should live with the 
non-zero residual and adapt the camera poses to local areas. To this end, 
we propose a segment-based approach to readjust the camera poses locally 
and improve the reconstruction for fine geometry details. The key idea is 
to partition the globally optimized structure from motion points into 
well-conditioned segments for re-optimization, reconstruct their geometry 
individually, and fuse everything back into a consistent global model. 
This significantly reduces severe propagated errors and estimation biases 
caused by the initial global adjustment. The results on several datasets 
demonstrate that this approach can significantly improve the 
reconstruction accuracy, while maintaining the consistency of the 3D 
structure between segments.

To the best of my knowledge, ours is the first pipeline able to 
reconstruct highly accurate and consistent camera poses from 665K 
high-resolution images in a parallel manner.


Date:			Wednesday, 22 March 2017

Time:                  	1:30pm - 3:30pm

Venue:                  Room 1504
                         (lifts 25/26)

Committee Members:	Prof. Long Quan (Supervisor)
  			Prof. Huamin Qu (Chairperson)
 			Dr. Pedro Sander
 			Prof. Chiew-Lan Tai


**** ALL are Welcome ****