Automated Defect Analysis of Mechanical Components using Deep Learning-Based Computer Vision

The introduction of robots and automation in industrial processes brought many benefits to manufacturing industries, by improving both quality and quantity of production. Automation-assisted production is however mostly open-loop and requires checkpoints to perform product quality analysis. Early vision-based systems were therefore developed since the nineties. This approach worked very well for those activities in which the critical issues can be formally expressed by means of geometrical properties (e.g. measures) or the presence/absence of well-known features on the inspected objects. However, to date, there are still many quality-control activities that cannot be performed by automated vision inspection machines. Their use is limited by the need to express the checks to be performed as a predefined sequence of actions in which each of them must be carefully designed to fit the specific production requirements. Nevertheless, many of the checks which can be easily learned by humans, are extremely difficult to be formally described using a set of static rules. The major difference relies on the fact that humans can exploit their experience as a relevant evaluation criterion while assessing the quality of a product. On the other hand, implementing such feature on an automated vision inspection machine requires the development of novel algorithms which can be trained and improved with time and experience. Early attempts using traditional Artificial Neural Networks (ANNs) and other learning tools dramatically failed due to the complexity of modeling in rigid mathematical structures the vast set of rules required to model the expertise. In the last few years, the research on Deep Neural Network (DNN) brought new interest in the field of machine learning based on ANNs. In particular, it has been shown that, given enough data, a DNN can be successfully used to perform complex computer vision tasks such as object classification, object detection and image segmentation. Those networks showed several interesting features, such as experience-based learning, scalability, and performances similar (and in some cases even superior) to human beings. Moreover, with respect to previous ANN approaches, these networks have the advantage that they do not require anymore explicit efforts to describe optimized features extraction algorithms. This Thesis work has the aim to map the human learning ability from experience into an equivalent learning-from-data capability of a DNN. The effectiveness of this approach has been proved in a real application for the inspection of welding defects on the assembly line of fuel injectors. We designed and assessed an automated defect analysis system based on deep learning. The Thesis covered all the development steps from early concept design, specification design, development of a prototype, software design and finally the development of a complete system which will be integrated into the assembly line. The hardware and software have been designed for all the system components, from the acquisition of the images to the communication of the analysis results to the enterprise PLC. Most of the efforts have been put in the design of the network architecture and on the definition of the training procedure. Starting from state-of-the-art deep architectures and using the transfer learning technique, it has been possible to train a network with about 7 millions parameters using a reduced number of injectors images, obtaining an accuracy of 97.22%. The realization of this Thesis has been made possible thanks to the collaboration with Continental Automotive Italy S.p.A and their interest in the Industry 4.0 research topics to bring intelligence in the manufacturing processes. So far, the system described in this work has been successfully tested on the assembly line and will enter into production starting from June 2018. We already provided intelligence in the system so that during the early phases of operation, it will collect new images to extend the existing dataset and to improve further its performance. With this Thesis work, we showed that deep neural networks can successfully perform quality inspection tasks which are actually done by humans. The approach, the system and the methodology described here can be also easily extended and applied to other applications