Single Portrait Image Matting and Bokeh Effect Synthesis via Multiplane Images

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "Single Portrait Image Matting and Bokeh Effect Synthesis via Multiplane
Images"

By

Mr. Zhefan RAO


Abstract:

High-quality portrait photography has become an essential function in our daily
lives. However, due to the limited aperture and focal length of a smartphone
camera, images captured by a smartphone could not match the same level of bokeh
effect by a digital single-lens reflex camera. A common solution on a
smartphone is to simulate out-of-focus effects from an all-in-focus image,
where the key is robust depth estimation and portrait matting. To achieve this,
we propose a multi-stage, multi-branch matting network to estimate a
strand-level portrait alpha mask, which is then used to refine the coarse depth
map from the pre-trained model. By integrating the input portrait image with
the estimated depth map and alpha mask, we propose a multi-plane image (MPI)
representation that includes multiple layers of disk-blurred images, where the
kernel size is proportional to the absolute depth distance to the focus layer.
Then a depth-aware blurring process is applied to enforce the bokeh effect.
Besides, each MPI layer has an alpha channel controlling the visibility
according to the corresponding depth. Finally, an image with bokeh is rendered
by compositing all MPI layers. We conduct comprehensive experiments to evaluate
our method, which demonstrates that our method can generate more accurate alpha
masks and more realistic images with bokeh, compared to prior work.


Date:                   Monday, 28 August 2023

Time:                   1:00pm - 3:00pm

Venue:                  Room 5501
                        lifts 25/26

Committee Members:      Dr. Qifeng Chen (Supervisor)
                        Prof. Chi-Keung Tang (Chairperson)
                        Dr. Sai-Kit Yeung


**** ALL are Welcome ****