Real-time 3D tracking of articulated tools for robotic surgery

Abstract

In robotic surgery, tool tracking is important for providin g safe tool-tissue interaction and facilitating surgical sk ills assessment. De- spite recent advances in tool tracking, existing approache s are faced with major difficulties in real-time tracking of articulated tool s. Most algo- rithms are tailored for offline processing with pre-recorded videos. In this paper, we propose a real-time 3D tracking method for articul ated tools in robotic surgery. The proposed method is based on the CAD mo del of the tools as well as robot kinematics to generate online pa rt-based templates for efficient 2D matching and 3D pose estimation. A r obust verification approach is incorporated to reject outliers in 2D detections, which is then followed by fusing inliers with robot kinemati c readings for 3D pose estimation of the tool. The proposed method has be en val- idated with phantom data, as well as ex vivo and in vivo experiments. The results derived clearly demonstrate the performance ad vantage of the proposed method when compared to the state-of-the-art