SinIR: Effcient General Image Manipulation withSingle Image Reconstruction

MPhil Thesis Defence


Title: "SinIR: Effcient General Image Manipulation withSingle Image 
Reconstruction"

By

Mr. Ji Hyeong YOO


Abstract

Focusing on internal visual information included in one single image, 
internal methods have been widely studied in the computer vision 
community. Especially as it removes the necessity for collecting 
large-scale datasets which is usually accompanied with intensive human 
labor for labelling, deep internal learning has came into the limelight 
very recently. However, in terms of practical usage of deep internal 
learning, there are still many obstacles to be overcomed. For example, 
most existing deep internal methods are (1) image-specific or 
task-specific, or (2) requires long training time.

In this thesis, we push the limits of deep internal learning by proposing 
SinIR, a reconstruction-based framework trained on a single image for 
general image manipulation. SinIR is trained on a single image with 
cascaded multi-scale learning, where each network at each scale is 
responsible for image reconstruction. Having reconstruction as its 
training objective, SinIR is trained way faster and robustly. However, 
naively using reconstruction leads unsatisfactory visual quality due to 
its innate characteristics. Thus, to mitigate this problem, we apply 
random pixel shuffling, a simple solution to effectively enrich the 
learning process, inspired by the Denoising Autoencoder. SinIR solves 
various computer vision problems including super-resolution, editing, 
harmonization, paint-to-image, photo-realistic style transfer, and 
artistic style transfer. Quantitative evaluation shows SinIR has 
competitive performance comparable to those of dedicated external methods. 
Also it is found that SinIR is trained 33.5 times faster than SinGAN (for 
\(500 \times 500\) images) that solves similar tasks.


Date:  			Tuesday, 3 August 2021

Time:			2:00pm - 4:00pm

Zoom meeting: 
https://hkust.zoom.us/j/98020863319?pwd=c3F2V2QraUVFeldVcG1pZytHL1MxZz09

Committee Members:	Dr. Qifeng Chen (Supervisor)
 			Prof. Raymond Wong (Chairperson)
 			Prof. Chi-Keung Tang


**** ALL are Welcome ****