Completion of 3D Point Clouds Derived from 2D Sonar Images using PCTMA-Net

Abstract

This research introduces a method for completing point clouds derived from 2D sonar images utilizing the Point Cloud Transformer with Morphing Atlas-based Point Generation Network (PCTMA-Net), initially developed for noise-free point clouds. The research thus centres on evaluating how effectively this network can adapt to and process the additional complexity introduced by sonar noise. The study evaluates PCTMA-Net's effectiveness in adapting to sonar noise, interpolating missing regions, and enhancing 3D reconstruction accuracy. This is crucial for applications requiring precise 3D models from limited 2D sonar imaging. The study uses a Blender-generated and real sonar image dataset with different noise levels. It compares a model with pre-trained weights and one whose hyperparameters are tuned to noisy point cloud conditions. This setup investigates the network's adaptability to different noise levels and data irregularities typical in underwater imaging. The goal is to explore PCTMA-Net's noise-handling capabilities and potential improvements for sonar-based 3D completion and refining tasks.