EFFICIENT SEMI-AUTOMATIC TECHNIQUES FOR IMAGE AND VIDEO MORPHING

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "EFFICIENT SEMI-AUTOMATIC TECHNIQUES FOR IMAGE AND VIDEO MORPHING"

By

Miss Jing LIAO


Abstract

This thesis proposes new methods for creating continuous mappings between 
images and videos, as well as the adaptive transition control schemes that 
create smooth transitions.

The main challenge in achieving good image morphs is to create a map that 
aligns corresponding image elements. Our aim is to help automate this 
often tedious task. We compute the map by optimizing the compatibility of 
corresponding warped image neighborhoods using an adaptation of structural 
similarity. The optimization is regularized by a thin-plate spline, and 
may be guided by a few user-drawn points. We parameterize the map over a 
halfway domain and show that this representation offers many benefits. The 
map is able to treat the image pair symmetrically, model simple occlusions 
continuously, span partially overlapping images, and define extrapolated 
correspondences. Moreover, it enables direct evaluation of the morph in a 
pixel shader without mesh rasterization. We improve the morphs by 
seamlessly extending content beyond the image boundaries. We parallelize 
the algorithm on a GPU to achieve a responsive interface and demonstrate 
challenging morphs obtained with little effort.

Extending image morphing techniques to video presented some added 
challenges. Because motions are often unsynchronized, temporal alignment 
is also necessary. Applying morphing to individual frames leads to 
discontinuities, so temporal coherence must be considered. Our approach is 
to optimize a full spatiotemporal mapping between the two videos. We 
reduce tedious interactions by letting the optimization derive the 
fine-scale map given only sparse userspecified constraints. For 
robustness, the optimization objective examines structural similarity of 
the video content. We demonstrate the approach on a variety of videos, 
obtaining results using few explicit correspondences.

We further explore the optimized transition paths for both geometry 
warping and color blending in order to reduce deformation and blurriness 
artifacts during morphing. We also provide convenient tools for the user 
to intuitively control the transition paths and rates, resulting in more 
interesting morphs.


Date:			Wednesday, 17 December 2014

Time:			10:00am - 12:00noon

Venue:			Room 4483
 			Lifts 25/26

Chairman:		Prof. Henry Lam (CBME)

Committee Members:	Prof. Pedro Sander (Supervisor)
 			Prof. Huamin Qu
 			Prof. Chiew-Lan Tai
 			Prof. Ajay Joneja (IELM)
                        Prof. Hongbo Fu (Creative Media, CityU)


**** ALL are Welcome ****