Noise Robust 3D Neural Rendering With Generalized NeRF Models

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "Noise Robust 3D Neural Rendering With Generalized NeRF Models"

By

Mr. Chan Ho PARK


Abstract:

3D reconstruction - inferring the 3D geometry and representation from a
collection of 2D images - has been a long-standing task in computer vision for
its complexity and wide possible applications especially for its capability to
convert 2D images into a more immersive perspective.

From that regard, Neural Radiance Fields (NeRF) propose a neural rendering
paradigm for representing a scene as a continuous function in 3D space and
inspire many new ways of efficiently representing and effectively rendering a
scene. However, most methods are tested on a controlled setting where it has a
clean set of images along with its camera information. In a more general and
practical setting, images captured by the user will contain various types of
noises, including blur, defocus, and background noise. Although NeRF-based
model structures inherently serve as a scene prior for noise handling, there is
no component in NeRF-based models accounting for the potential noise in the
source images. Furthermore, existing noise-aware NeRF models typically design a
rendering method based on the real physical image formulation process of one
specific type of noise. This can limit the robustness of the model against
multiple types of noises and also requires the prior knowledge of the type of
noise existing in the source images.

With these motivations, this work designed a simple and model-agnostic module
for noise-robust neural rendering in Generalizable NeRF (GNeRF) models. For
noise robust rendering, this work devises reconstruction-based modules to
handle multiple types of noise including burst noise and blur noise. The
proposed module shows improvement not only in terms of visual quality but also
in stable and consistent prediction over varying noise levels.


Date:                   Wednesday, 6 December 2023

Time:                   10:00am - 12:00noon

Venue:                  Room 3494
                        lifts 25/26

Committee Members:      Dr. Qifeng Chen (Supervisor)
                        Prof. Pedro Sander (Chairperson)
                        Dr. Dan Xu


**** ALL are Welcome ****