SHADOW REMOVAL AND OBJECT PROPOSAL GENERATION FOR RGB-D IMAGES

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "SHADOW REMOVAL AND OBJECT PROPOSAL GENERATION FOR RGB-D IMAGES"

By

Mr. Yao XIAO


Abstract

In recent years, the emergence of reliable and low-cost RGB-D sensors
(e.g., Microsoft Kinect) has expended the dimension of a single image from
2D to 3D. With the aiding of additional depth channel the 3D spatial
information is discovered to compensate 2D image plane, in which way many
computer vision tasks are boosted. In this thesis, we present the research
of extending two vision tasks, shadow removal and object proposal
generation, from RGB to a single RGB-D image.

Shadow removal is a classical and challenging computer vision problem.
First we propose an automatic method to remove shadows from single RGB-D
images. Using normal cues directly derived from depth, we can remove both
hard and soft shadows while preserving surface texture and shading.  Our
key assumption is: pixels with similar normals, spatial locations and
chromaticity should have similar colors.  A modified nonlocal matching is
used to compute a shadow confidence map that localizes well hard shadow
boundary, thus handling hard and soft shadows within the same framework.
Then the detected shadows will be removed by a constrained linear
optimization to reconstruct a shadow-less image.

Our second task is to generate object proposals from RGB-D images. But
before that we present a novel method to produce proposals for 2D images.
Object proposals are the potential object candidates in the detection
pipeline. Besides, distance metric plays a key role in grouping
superpixels to produce the proposals for object detection. We observe that
existing distance metrics work primarily for low complexity cases. In this
paper, we develop a novel distance metric for grouping two superpixels in
high-complexity scenarios. Combining them, a complexity-adaptive distance
measure is produced that achieves improved grouping in different levels of
complexity.

Next we focus on the task of extracting 3D region proposals from indoor
RGB-D images, which aims to produce bounding boxes of candidate objects.
3D voxel grid contains large amount of redundant space. To rule out
less-informative voxels and to simplify the problem we introduce a space
compression procedure to squash 3D space to 2D "tile" grid. After each
tile is layered in vertical direction individually, we propose Structural
Constrained Parametric Min-Cuts (S-CPMC) to group the tilted space. The
extracted tiles are further processed to reconstruct 3D bounding boxes
through geodesic distance transformation (GDT) from the generated tile
hypotheses. Finally the object hypotheses are ranked by a trained ranker.


Date:			Wednesday, 12 July 2017

Time:			3:00pm - 5:00pm

Venue:			Room 2612B
 			Lifts 31/32

Chairman:		Prof. Jimmy Fung (MATH)

Committee Members:	Prof. Chi-Keung Tang (Supervisor)
 			Prof. Huamin Qu
 			Prof. Long Quan
 			Prof. Shing-Yu Leung (MATH)
 			Prof. Michael Brown (York University)


**** ALL are Welcome ****