Surgical Instrument Segmentation using Estimated Depth from Monocular Laparoscopic Image

Abstract

It is necessary to extract surgical instruments from laparoscopic images in order to improve the safety of laparoscopic surgery using a surgery support system. It is reported that the segmentation accuracy can be improved by using color and depth information. In this paper, we propose a U-Net based image segmentation network using the estimated depth by pix2pix as well as color for improving the accuracy. We conducted experiments using 4-fold cross validation with 1,800 images in the MICCAI challenge dataset, and confirmed that the proposed method achieved the average IoU of 88％ and the average Dice coefficient of 93%. The proposed method reduced the excessive extraction and improved the extraction accuracy by using the estimated depth information as well as color information.