Vision-based manipulation with the humanoid robot Romeo

International audienceThe aim of this paper is to show how visual servoing can help a humanoid robot to realize manipulation tasks with one and two hands, in conjunction with a gaze control. In addition, the use of vision in a closed-loop control scheme allows to accomplish these tasks with high repeatability without an accurate calibration of the robot kinematic model, the intrinsic and extrinsic parameters of the camera. The first application shown is grasping: one arm is controlled in order to reach the desired pose for a successful grasp along with a gaze control that keeps hand and object in the camera field of view. This approach is extended to the grasping of cylindrical objects, ignoring in the control the orientation along the revolution axis, that could cause an unnecessary motion of the arm. Moreover, we show how to control both arms with a master/slave approach for a two-handed manipulation without any force control. The robot is then able to solve a ball-in-maze game in augmented reality manipulating a tray with two hands, just using vision. A secondary task to avoid the joint limits using a large projection operator is added to improve the reliability of the system. Finally, the framework is implemented and evaluated on the humanoid robot Romeo using the ViSP library. The source code of the libraries, demos and examples are completely accessible on GitHub, allowing an easy adaptation of the visual servoing framework to other robots