Implementation of a Visual Servo Control in a Bi-Manual Collaborative Robot

This project presents the application of a visual servo control to an industrial human-like robot, both using a simulation environment and in a real platform. In a visual servo scheme, the control loop is closed by a vision sensor, usually a camera (or more than one, in the stereo approach). The camera acquires the image of a defined target and, a control algorithm calculates the relative pose of robot-target and then, continuously sends commands to the robot in order to position it as required. The pose calculation and controller algorithms have been written in C++. The work has been carried out through a sequence of stages that are presented in this document. The first part goes through the basic theoretical ideas that support the design of the visual servo. It is composed of three main areas: computer vision, which deals mostly with the implementation of the vision sensor; robot kinematics, which allows define the equations that describe time evolution of the robot position, orientation, speed and joints values; and finally, the merge of both areas, the visual servo itself that makes up the control loop. A next section explains how the different tools and frameworks have been used to implement the control loop. Some of these tools are manufacturer proprietary programs, others are open source. There is a detailed description of how the simulation environment is set, the content of each of the blocks in the control loop and a basic explanation of the manufacturers program. The results show how the robot (simulated and real) converges to the relative set point pose and is also able to track changes in the position and orientation of the target