Real-time Endoscopic Image Overlay of the Tip of Surgical Instruments for Control of a Forceps Robot

Abstract

In this research, we aim to establish a method of real-time high accuracy endoscopic image overlay of the tip of surgical instruments for control of a forceps robot. In general, the relationship between the position of an object in the three-dimensional space and its projection onto the camera image can be represented by a single camera model. However, the closer an endoscope gets to surgical instruments, the larger the projection error becomes. The proposed system introduces virtual multi camera models according to the distance between an endoscope and surgical instruments for alleviating the projection error.