Exploring Efficient and Robust Marine Visual Analysis and Building Marine Foundation Models

PhD Qualifying Examination


Title: "Exploring Efficient and Robust Marine Visual Analysis and Building 
Marine Foundation Models"

by

Mr. Ziqiang ZHENG


Abstract:

The marine ecosystem is the most productive of all underwater ecosystems and 
shares immense ecological, social, and economic value. Performing marine study 
plays a significant role in protecting the marine environment and understanding 
marine science. The marine research involves the study of marine biology, 
oceanography, and environmental science through the lens of visual data, 
enabling scientists and researchers to observe, document, and analyze the vast 
and mysterious creatures beneath the water's surface. Most existing marine 
studies highly depend on describing and analyzing the collected image/video 
data based on in-situ marine/underwater surveying approaches. There are two 
main limitations for existing marine studies: 1) they cannot support a very 
large scale data collection and data scarcity has become one of the important 
factors that hinder the further development of the marine analysis; 2) further 
data analysis procedure still requires many human labors, time costs, and is 
also limited to specific biology users. Recent foundation models have achieved 
great success, driven by a significant scale of training data and powerful 
networks. Such foundation model recipe leads to efficient and flexible models, 
supporting a wide spectrum of downstream visual analysis tasks. However, few 
attempts have been explored in the marine field and we wonder whether this 
recipe is effective in marine research field. In this survey, we performed a 
comprehensive review of the existing marine visual analysis techniques and the 
detailed discussions about the potential future directions. We have reviewed 
the recent research progress of marine visual analysis, including underwater 
image/video enhancement, object detection, dense segmentation and 3D scene 
understanding. Then we discuss the potential of utilizing existing foundation 
models for marine visual analysis and dissect what kind of domain-specific 
modifications were required for building effective and efficient marine 
foundation models. We have also reviewed the existing marine datasets and 
analysis platforms designed for marine analysis. The detailed and hierarchical 
discussions about potential applications of built marine foundation models are 
also included. Finally, we discuss the insightful future directions for 
promoting the marine visual analysis.


Date:                   Wednesday, 10 April 2024

Time:                   2:00pm - 4:00pm

Venue:                  Room 5501
                        Lifts 25/26

Committee Members:      Dr. Sai-Kit Yeung (Supervisor)
                        Prof. Chi-Keung Tang (Chairperson)
                        Dr. Tristan Braud
                        Prof. Pedro Sander