Unsupervised Video Super-Resolution with Temporal Consistency using GAN

MPhil Thesis Defence


Title: "Unsupervised Video Super-Resolution with Temporal Consistency using 
GAN"

By

Mr. Song WEN


Abstract

Video super-resolution (VSR) means recovering a high-resolution (HR) video from 
its low-resolution (LR) counterpart. Convolutional neural network (CNN) models 
have been recently shown to be promising for VSR. However, these models are 
based on supervised learning, i.e., trained on synthesized LR/HR pairs with 
prior knowledge of degradation operation from HR sequence to the LR version. In 
reality, the degradation operation is usually not available, which limits the 
performance of these supervised models. Moreover, previous approaches often 
generate frames independently. Because the temporal dependency among generated 
HR frames has not been captured properly in the video sequence, they suffer 
from poor temporal consistency in the form of flickering artifacts.

We propose VistGAN, the first unsupervised video super-resolution with temporal 
consistency using cycle Generative Adversarial Network (GAN) architecture. By 
employing CNN to degrade some externally inputted HR videos, VistGAN learns in 
an unsupervised way the inherent degradation operation and its inverse for the 
test (LR) video. It then super-resolves the test video using the inverse. 
Furthermore, to achieve temporal consistency, VistGAN uses a frame-recurrent 
framework based on HR flow estimation which fuses the current LR test frame 
with the previous super-resolved HR frame. We conduct extensive experiments on 
benchmark datasets to validate VistGAN performance. Compared with 
state-of-the-art schemes, VistGAN achieves much better performance in terms of 
temporal consistency and PSNR, and it is also effective for videos with blur 
and scene changes.


Date:			Thursday, 26 September 2019

Time:			10:00am - 12:00noon

Venue:			Room 2408
 			Lifts 25/26

Committee Members:	Prof. Gary Chan (Supervisor)
 			Dr. Raymond Wong (Chairperson)
 			Prof. Andrew Horner


**** ALL are Welcome ****