Model-based 3-D pose estimation and feature tracking for robot assisted surgery with medical imaging

International audienceIn this paper we address the problem of the pose estimation based on multiple geometrical features for monocular endoscopic vision with laparoscopes and for stereotaxy with CT scanners. Partial and full pose estimation (6 dofs) are considered with applications to minimally invasive surgery. At the University of Strasbourg, we have been developing a set of techniques for assisting surgeons in navigating and manipulating the three-dimensional space within the human body. In order to develop such systems, a variety of challenging visual tracking and registration problems with pre-operative and/or intra-operative images must be solved. This paper integrates several issues where computational vision can play a role. Depth recovery (from the tip of a surgical instrument w.r.t. living tissue), the Pl ¨ucker coordinates (4 dofs) of a markerless cylindrical instrument, the 6 dofs of a needle-holder with an heterogeneous set of features and stereotaxy are the examples we describe. Projective invariants with perspective projection, quadrics of revolution and stereotactic markers are features which are useful to achieve the registration with uncalibrated or calibrated devices. Visual servoing-based tracking methods have been developed for image-guided robotic systems, for assisting surgeons in laparoscopic surgery and in interventional radiology. Real-time endoscopic vision and single-slice stereotactic registration has been proposed to retrieve the out-of-field of view instruments, to position a needle and to compensate small displacements like those due to patient breathing or any small disturbances which may occur during an image-guided surgical procedure