Towards sensor-based manipulation of flexible objects

International audience— This paper presents the FLEXBOT project, a joint LIRMM-QUT effort to develop (in the near future) novel methodologies for robotic manipulation of flexible and deformable objects. To tackle this problem, and based on our past experiences, we propose to merge vision and force for manipulation control, and to rely on Model Predictive Control (MPC) and constrained optimization to program the object future shape. Index Terms— Control for object manipulation, learning from human demonstration, sensor fusion based on tactile, force and vision feedback. I. CONTEXT This abstract does not present experimental results, but aims at giving some preliminary hints on how flexible robot manipulation should be realized in the near future, particularly in the context of the FLEXBOT project, jointly submitted to the PHC FASIC Program 1 by LIRMM and QUT researchers. The objective of FLEXBOT is to solve one of the most challenging open problems in robotics. In fact, we aim at developing novel methodologies enabling robotic manipulation of flexible and deformable objects. The motivation comes from numerous applications, including the domestic, industrial, and medical examples 2 shown in Fig. 1. Many difficulties emerge when dealing with flexible manipulation. In the first place, the object deformation model (involving elasticity or plasticity) must be known, to derive the robot control inputs required for reconfiguring its shape. Ideally, this model should be derived online, while manipulating , with a simultaneous estimation and control approach, as commonly done in active perception and visual servoing. Hence perception, particularly from vision and force, will be indispensable. This leads to a second major difficulty: deformable object visual tracking. In fact, most current visual object tracking algorithms rely on rigidity, an assumption that is not valid here. A third challenge will consist in generating control inputs that comply with the shape the object is expected to have in the near future. In the next section, we provide a brief survey of the state of art on flexible object manipulation. We then conclude by proposing some novel methodologies for addressing the problem