ENTED: Enhanced Neural Texture Extraction and Distribution for Reference-based Blind Face Restoration

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "ENTED: Enhanced Neural Texture Extraction and Distribution for 
Reference-based Blind Face Restoration"

By

Mr. Yuen Fui LAU


Abstract:

We introduce ENTED, an innovative framework for blind face restoration that 
aims to enhance high-quality and realistic portrait images. Our approach 
involves restoring a single degraded input image by leveraging information 
from a high-quality reference image. To achieve this, we employ a texture 
extraction and distribution framework, facilitating the exchange of 
high-quality texture features between the degraded input and the reference 
image.

In the context of blind face restoration research, architectures similar to 
StyleGAN prove beneficial for generating high-quality portrait images with 
rich texture details. However, these StyleGAN-like architectures have a 
drawback: they rely on high-quality latent codes to produce realistic images. 
Unfortunately, latent codes extracted from degraded input images often 
contain corrupted features, posing a challenge in aligning semantic 
information from the input with the high-quality textures from the reference.

To address this challenge, we employ two specialized techniques. First, 
inspired by vector quantization, we replace corrupted semantic features with 
high-quality code words. Second, we generate style codes that carry 
photorealistic texture information from a more informative latent space, 
developed using the high-quality features present in the reference image's 
manifold. Our extensive experiments on synthetic and real-world datasets 
demonstrate that our method yields results with more realistic contextual 
details, outperforming state-of-the-art approaches. A thorough ablation study 
further confirms the effectiveness of each proposed module.


Date:                   Tuesday, 30 July 2024

Time:                   2:00pm - 4:00pm

Venue:                  Room 5501
                        Lifts 25/26

Chairman:               Prof. Gary CHAN

Committee Members:      Dr. Qifeng CHEN (Supervisor)
                        Dr. Dan XU