Tool Tracking in a Laparoscopic Virtual Reality Training System

Minimally invasive surgery (MIS) is replacing the open surgical approach to reduce pain, expedite patient recovery, shorten hospital stays and improve cosmetic results for many abdominal procedures. It is widely accepted that the increased technical difficulty of MIS approaches (difficult visualization, tool fulcrum effect, etc.) requires specialized surgical training. The fundamentals of laparoscopic surgery (FLS) program has become a de facto standard to evaluate surgical skills for laparoscopic MIS Simulation-based surgical training has been growing not only as an innovative way to teach surgery [2] but also as a method to assess the skill of a surgeon in performing a procedure, without risk to patients We use cameras to track tool motion [5], a general concept of the design and determination of appropriate workspace visualization were discussed. The fabrication process was explained and the first prototype was presented. In this paper, the computer vision and image processing aspects of the surgical simulator are described; these are used to detect the motion of the tool in the physical simulator and map these movements into a virtual surgical environment (Vizard, WorldViz, Santa Barbara, CA) for visualization. Pilot virtual simulation tasks are also described. Methods 2.1 Image-Based Tracking. An image-based method is used to send grasper motion data into the simulator by using video from a pair of cameras, which are calibrated using toolboxes available with MATLAB software. A pose estimation algorithm A four-colored marker is attached to the end of the grasper as shown in The red-blue-green color space is used to find the position of the markers against a white background as frames are streamed from the cameras. The differently colored markers are arranged to minimize occluded views and detect rotation about the tool shaft. The color threshold module is used to segment the colored dots in the images by using the MATLAB image processing toolbox. The colored dots are extracted by subtracting the channel color of the image from the grayscale of the images and extracting the pixels whose value exceeds the threshold. After calibrating the position of the cameras, the 3D positions of the markers are identified with respect to the 2D color-filtered images by applying a triangulation method. In this method, the nearest point from the projection line to the 3D position of the object is calculated, as illustrated in In the first step, we determine the direction of the 3D rays by calculating the unit vector pointing from the camera center to the object