Underwater SLAM for Marine Robot Navigation

The Vision, Graphics and Computational Design (VGD) Group at HKUST led by Prof. Sai-Kit Yeung performs cutting-edge research on 3D reconstruction, understanding, modeling, and AI techniques for design and content generation, human-robot interaction, and enhancing VR and AR applications.

The VGD group is now working very closely with marine biologists, field ecologists, and coral conservationists to develop novel computer vision and graphics techniques and integrative systems to solve a wide range of marine related problems.

The fundamental challenge of applying computer vision algorithms in the marine environment is how dynamic it is: objects like fishes are constantly moving, seaweed and soft coral change their shapes with water current, light becomes everchanging and also attenuates as we go deeper. Depending on the season, location, and even different time in the day, the visibility can vary significantly.

All these violate the invariant assumption of most computer vision algorithms. Solving these challenges for underwater computer vision problems for marine applications can let us better understand mother nature, and we believe that this shall also help us develop robust computer vision algorithms that can also solve problems on the ground. We believe this should also create more opportunities for the computer vision community to tackle more interesting problems.

One of the underwater computer vision projects we are working on is to develop a new real-time visual simultaneous localization and mapping (SLAM) system for marine robot navigation. Without special equipment, positioning in an underwater environment is a very challenging problem because GPS does not work due to strong EM waves attenuation. Commercial underwater robots typically rely on acoustic positioning to locate themselves under the water. However, using acoustics based positioning is expensive, bulky, and not accurate.

Our goal is to enhance the underwater SLAM system with the recent rapid development in computer vision. There are two core research tasks to develop an underwater visual SLAM system. 1) To make the scene as invariant as possible, this research task will also be useful in other applications. 2) To develop a SLAM system that is robust to tracking failure. Below we show some examples on why SLAM is hard in an underwater situation.

Visual SLAM relies on feature extraction and matching to track the agent's pose and mapping under the assumption of a static scene. In this figure, moving objects dominate the scene and most of the matched features lie in the dynamic area. It would significantly reduce the accuracy of pose estimation and lead to tracking failure.

Dynamic light affects feature matching. To achieve real-time performance, feature-based SLAM utilizes ORB feature, an efficient but not illumination invariant feature. As the above pair of images show, although vast features are extracted, there are less matched features.

The new underwater SLAM system will be developed based on our recent Dual-SLAM [1] which is a state-of-the-art SLAM algorithm that is more robust in terms of tracking failure. The preliminary results from Dual-SLAM will give us good guidance to develop an image pre-processing module that makes the scene invariant and hence let us produce better SLAM output. Below shows some comparison and also our preliminary results.

We are hiring! If you are an ocean lover and want to develop new technology to solve different marine related problems, please email to join our research program.

[1] Huajian Huang, Daniel Lin, Siying Liu, Dong Zhang, Sai-Kit Yeung. Dual-SLAM: A framework for robust single camera navigation. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020)

Demo Videos

We compare our preliminary results with ORB-SLAM, a state-of-art indirect SLAM, and DSO, a state-of-art direct SLAM. This video shows that our system is more robust and accurate.