Deep Contextual Modeling: Exploiting Context in Spatial and Temporal Domains

PhD Thesis Proposal Defence


Title: "Deep Contextual Modeling: Exploiting Context in Spatial and Temporal 
Domains"

by

Mr. Yongyi LU


Abstract:

Context plays a critical role in perceptual inference as it provides useful 
guidance to solve numerous tasks both in spatial and temporal domains. In this 
dissertation, we study two fundamental computer vision applications, i.e., 
object detection and image generation, by exploiting different spatial-temporal 
contexts to boost their performance.

Driven by the recent development of deep neural nets, we propose deep 
contextual modeling. Context here refers to one of the following application 
scenarios, e.g., (1) temporal coherence and consistence for object detection 
from video frames; (2) spatial constraint for conditional image synthesis, 
i.e., generating image from sketch and (3) domain-specific knowledge such as 
facial attributes for natural face image generation.

We first study the problem of exploiting temporal context for video object 
detection, where applying single framework object detector directly in video 
sequence tends to produce high temporal variation on frame-level output. We 
advocate to the detection of long-range video object pattern for temporal 
coherence and consistence by proposing association LSTM. Next we investigate 
image generation guided by hand sketch in spatial domain. We propose contextual 
GAN to use sketch as weak spatial constraint, where the output images do not 
necessarily follow the input edges. Finally we explore domain-specific context, 
i.e., face attribute and attribute-guided face generation: we condition the 
CycleGAN and propose conditional CycleGAN, which is designed to allow easy 
control of the appearance of the generated face via the input context.


Date:			Friday, 6 July 2018

Time:                  	10:00am - 12:00noon

Venue:                  Room 3494
                         (lifts 25/26)

Committee Members:	Prof. Chi-Keung Tang (Supervisor)
 			Prof. Huamin Qu (Chairperson)
 			Prof. Long Quan
 			Dr. Pedro Sander


**** ALL are Welcome ****