Integration of robotic vision and tactile sensing for wire-terminal insertion tasks

This paper reports the development of a manipulation system for electric wires, implemented by means of a commercial gripper installed on an industrial manipulator and equipped with cameras and suitably designed tactile sensors. The purpose of this system is the execution of wire insertion on commercial electromechanical components. The synergy between computer vision and tactile sensing is necessary because, in a real environment, the tight spaces very often prevent the possibility to use the vision system, also when the same task is performed by a human being. A novel technique to speed up the generation of training data sets for convolutional neural networks (CNNs) is proposed. Therefore, this technique is used to train a CNN in order to detect small objects (such as wire terminals). Moreover, aiming to prevent faults during the task and to interact with the environment safely, several machine learning approaches are used to produce an affordable output from the tactile sensor. The proposed approach shows how a cheap sensor embedded with suitable intelligence can provide information comparable to a more expensive force sensor. Note to Practitioners - This paper was motivated by the lack of commercial solution for the automatic cabling of switchgears. Existing approaches to this problem are in some way limited to specific large-scale products or simple layouts. This paper investigated a robust and flexible solution, based on the exploitation of multiple sensors and machine learning algorithms, for wire detection, grasping, and connection. The proposed approach is characterized by simple design and self-tuning capabilities, and it can be easily employed on a wide range of switchgear layouts thanks to the large workspace of the manipulator. Experimental results show that the proposed system is able to achieve a 95% success rate within a realistic admissible region. In the future research, we will integrate the proposed solution with an electromechanical component localization module and a terminal fastening system to evaluate the performance on the real production line