2D-3D Registration Method for X-Ray Image Using 3D Reconstruction Based on Deep Neural Network

Abstract

This paper proposes a method for registering X-ray images with its 3D CT model by estimating 3D point clouds from X-ray images and their corresponding points on the image. Many conventional methods generate a simulated X-ray image from a 3D CT model and optimize the pose by using the similarity metrics between the simulated X-ray and the input X-ray image. On the other hand, deep learning approaches that predict pose information need a canonical coordinate system defined manually on the pre-operative CT to properly utilize the estimated pose. Therefore, we devise a fully automatic registration pipeline that is independent of coordinate system, by recovering 3D point clouds from X-ray images, estimating the corresponding points on the images, and aligning them with the given 3D CT model.