Analysis and synthesis of 3D human face

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Analysis and synthesis of 3D human face"

By

Mr. Jiaxiang SHANG


Abstract

Face reconstruction and reenactment have been extensive research for decades 
due to their fundamental research status and wide applications Previous 
reconstruction methods request multi-view input or different illumination 
conditions. Current applications require less con- strained settings, even from 
a monocular image. These approaches rely on face priors, com- monly known as 3D 
morphable models (3DMMs). This thesis aims to use the analysis-by- synthesis 
idea to reconstruct 3DMMs from monocular input. We propose a self-supervised 
training architecture considering the 3DMM as a face prior. We analyze the 3D 
face by Deep neural network (DNN) and then synthesize the landmark2d and image 
by projection and dif- ferentiable rendering. Then, we leverage the synthesis 
results to constrain the DNN training. However, the 3DMM is a statistic model 
learned from datasets of 3D scans, which contains limited expressions and loses 
face detail. Another goal of this thesis is to construct a digital avatar 
construction pipeline, which provides avatars with vivid face shapes and 
expressions to build a face model. Besides, it can also offer face details. 
Face reenactment synthesizes images combining the source image identity and the 
facial expression and pose of driving video frames. Recent methods regard 2D 
landmarks as face shape, expression, and pose representations. This thesis 
pushes the state-of-the-art in high-quality face reenactment in a 
synthesis-by-analysis strategy. We synthesize images under the analysis of the 
3D face, which is a more robust rep- resentation than facial landmarks. Hence, 
this thesis’s key idea is to analyze and synthesize human faces. Finally, we 
construct a facial expression recognition network, where face analysis and 
synthesis are applied to improve the recognition results.


Date:			Tuesday, 12 July 2022

Time:			4:00pm - 6:00pm

Zoom Meeting: 
https://hkust.zoom.us/j/96191752507?pwd=bWg4M0tvT0VzL1Z6dzdvdTNzYzcwUT09

Chairperson:		Prof. Chi Ying TSUI (ISD)

Committee Members:	Prof. Long QUAN (Supervisor)
 			Prof. Xiaojuan MA
 			Prof. Pedro SANDER
 			Prof. Ajay JONEJA (ISD)
 			Prof. Pong Chi YUEN (HKBU)


**** ALL are Welcome ****