Visual Enhancement Using Multiple Cues

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Visual Enhancement Using Multiple Cues"

By

Mr. Jia Chen


Abstract

Despite the advances in imaging technology, fundamental limitations of 
camera still exist and captured photographs can be defective. Two major 
types of defects are blur and noise. Enhancement of image and video is an 
important topic in computer vision and graphics, because it can serve 
either as a pre-processing step for other algorithms or as a 
post-processing step to directly produce enhanced output for users.

In this thesis, I will explore the issues and propose solutions to visual 
enhancement given images corrupted by blur and noise. There are a great 
number of previous works and most of them use a single image for the 
purpose of enhancement. This thesis will study the problem from a 
different perspective: using multiple cues for visual enhancement. 
Different ways are proposed to construct useful cues in addition to the 
source image itself. In deblurring, we introduce one more shot thus 
converting the deblurring problem from using a single image to using two 
images. In denoising problem, we use the noise layer as well as the noisy 
image for image noise removal. We formulate video denoising by optimizing 
one frame with multiple temporal observations.

While it is intuitive that using multiple cues will provide us with more 
information for enhancement, many research challenges remain to be solved. 
First of all, it is important to construct and collect multiple cues in 
proper ways. Second, the observations should be linked in a computational 
framework. The theme of this thesis is centered at a unified multi-cue 
enhancement approach, where we emphasize the importance of invariant 
quantity in linking multiple cues. We will also derive specific 
optimization procedures for integrating prior knowledge with these 
observations.

I will first analyze image deblurring problem using two observations. 
Given that the two inputs are taken from the same static scene, the 
invariant quantity is the common clear image. Since the two input images 
have different motion blur defects, their frequency responses are 
complementary to each other. A feedback algorithm is proposed that 
effectively combines two independently blurred images. This approach 
introduces an image prior and a motion prior in the context of multiple 
observations. Consequently, the visual quality of enhancement is greatly 
improved compared to approaches using single images.

  The second part of this thesis will focus on denoising. Removing noise 
from images is a topic which has been studied for decades. However, there 
are limitations inherent in most previous automatic approaches, which 
usually take the image itself as the processing target. We show that even 
with a single input image, an auxiliary observation, namely the noise 
layer can be constructed. Using an extracted noise layer, the artifacts of 
automatic denoising algorithms can be easily visualized and optimization 
can be performed on both image layer and noise layer. We propose an 
interactive system based on this representation, which allows a user to 
achieve high quality image noise separation results.

The image denoising system will be extended to video to denoise multiple 
frames which already exist as observations. The key issue this thesis is 
how to set up connections between these observations. Classical method of 
finding inter-frame correspondences is optical flow estimation which gives 
pixel-wise motion field. An extended motion field, which is called 
probabilistic motion field will be introduced to characterize soft 
temporal correspondences. The corresponding pixels will be placed inside a 
spatio-temporal Markov Random Field where the denoised frames are 
optimized using multiple observations.


Date:			Tuesday, 25 August 2009

Time:			2:00pm-4:00pm

Venue:			Room 3501
 			Lifts 25-26

Chairman:		Prof. King-Lun Yeung (CBME)

Committee Members:	Prof. Chi-Keung Tang (Supervisor)
 			Prof. Long Quan
 			Prof. Chiew-Lan Tai
 			Prof. Weichuan Yu (ECE)
 			Prof. Pheng-Ann Heng (Comp. Sci. & Engg., CUHK)


**** ALL are Welcome ****