Robust Scene Understanding in Challenging Scenarios

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Robust Scene Understanding in Challenging Scenarios"

By

Mr. Tuan Anh VU


Abstract:

In recent years, computer vision and graphics fields have witnessed significant 
progress with the emergence of novel techniques and architectures to address 
complex challenges. This thesis presents novel methodologies and advancements 
to enhance the accuracy and robustness of current methodologies in scene 
understanding. The study focuses on developing algorithms for 2D transparent 
object segmentation in challenging indoor environments, aiming to significantly 
improve detection capabilities in scenarios with cluttered backgrounds and 
variable lighting conditions.

Additionally, it explores 3D test time augmentation methods for classification 
and segmentation, targeting both static and dynamic objects in indoor and 
outdoor scenes to enhance model resilience and accuracy. By implementing these 
methods, the research aims to provide robust solutions that adapt to variations 
in data, thereby improving the overall performance of 3D models.

Furthermore, the research combines 3D reconstruction with motion flow 
estimation to achieve a comprehensive understanding of dynamic objects, such as 
humans and animals, in indoor settings. This approach aims to accurately track 
and predict object movements, enhancing the analysis of dynamic scenes.

The study also utilizes text-to-image diffusion models for 2D open vocabulary 
camouflage instance segmentation, addressing the detection and segmentation of 
camouflaged objects in outdoor and underwater environments. This method 
leverages advanced diffusion models to identify and segment camouflaged 
instances from diverse vocabularies, improving the detection and analysis of 
such objects in challenging settings.

Lastly, the creation of a comprehensive dataset for camouflaged animals in 
videos aims to improve classification, detection, and segmentation algorithms. 
This dataset includes outdoor and underwater videos of camouflaged animals, 
facilitating the development of robust algorithms capable of accurately 
analyzing these animals in their natural habitats. This effort contributes to 
wildlife conservation and environmental monitoring by providing a valuable 
resource for further research.

In summary, this research pushes the boundaries of scene understanding by 
offering more accurate and robust solutions for interpreting complex scenes. By 
addressing specific challenges associated with different environments and 
object types, it aims to significantly advance the field of computer vision.


Date:                   Tuesday, 13 August 2024

Time:                   10:00am - 12:00noon

Venue:                  Room 5510
                        Lifts 25/26

Chairman:               Prof. Hongbo FU (EMIA)

Committee Members:      Prof. Sai Kit YEUNG (Supervisor)
                        Prof. Huamin QU
                        Dr. Dan XU
                        Dr. Stanley Chun Kwan LAU (OCES)
                        Dr. Di LIN (TJU)