Movement-Flow-Based Visual Servoing and Force Control Fusion for Manipulation Tasks in Unstructured Environments

Abstract—In this paper, a new approach for fusing visual and force information is shown. First, a new method for tracking trajec-tories, called movement flow-based visual servoing system, which presents the correct behavior in the image and in the three-di-mensional space, is described. The information obtained from this system is fused with that obtained from a force control system in unstructured environments. To do so, a new method of recognizing the contact surface and a system for fusing visual and force infor-mation are described. The latter method employs variable weights for each sensor system, depending on a criteria based on the de-tection of changes in the interaction forces processed by a Kalman filter. Index Terms—Autonomous manipulation, eye-in-hand system, force control, multisensor systems, tracking trajectories, 2-D visual servoing. I