Developing Learning Algorithms for Enhancing Industrial Machine Vision Systems and Improving Task Accuracy of Robotic Manipulators

Vision based learning techniques have become increasingly important in recent years for the development of highly accurate and robust algorithms in various fields of industry. Some of the most important applications of machine vision and learning in industry are structural health monitoring (SHM) and industrial robotics. SHM has been a critical technology in monitoring the structural integrity of composite materials used in aerospace industry. Since airframes operate under continuous external loads, they are exposed to large deformations that may adversely affect their structural integrity. Therefore, critical components such as wings should be continuously monitored to ensure a long service life. In this thesis, a realtime SHM system is developed for airframe structures to localize and estimate the magnitude of the loads causing deflections to the wings. To this end, a framework based on artificial neural networks (ANN) is developed where features extracted from a depth camera are utilized. The localization of the load is treated as a multinomial logistic classification problem and the load magnitude estimation as a logistic regression problem. The neural networks trained for classification and regression are preceded with an autoencoder (AE), through which data at a much smaller scale are extracted from the depth images. The effectiveness of the proposed method is validated by an experimental study performed on a composite UAV wing subject to concentrated and distributed loads, and the results obtained by the proposed method are superior when compared with a method based on Castigliano’s theorem. As for industrial robots, they are poised to replace CNC machines in the near future due to their lower price, high degree of automation and larger working space. However, their relatively low accuracy is a hindrance in their wide deployment in the manufacturing industry. Laser trackers are known to significantly increase their accuracy for manipulation tasks, however their high cost is a major problem for their usage. Therefore, more affordable solutions such as machine vision systems can become a valuable addition to the robotics industry. In this thesis an eye to hand camera based pose estimation system is developed for robotic machining and the accuracy of the estimated pose obtained through the Levenberg-Marquardt (LM) algorithm is improved using three supervised learning approaches. These approaches can enhance the estimated pose during both no-load trajectory tracking and machining process. The first proposed method is based on a Long Short Term Memory (LSTM) neural network and the other two are based on sparse regression and they are named as Sparse Identification of Nonlinear Statics (SINS) and Sparse Nonlinear Finite Impulse Response (SNFIR). Both of the LSTM and SNFIR algorithms can take the dynamics into account during robotic machining through utilization of the torque information available from the sensors at each joint to improve the estimated pose. The SINS algorithm can be used to improve the estimated pose through utilization of nonlinear static functions during no-load trajectory tracking. The proposed methods are validated by an experimental study performed using a KUKA KR240 R2900 ultra robot while following sixteen distinct trajectories based on ISO 9283 in addition to two distinct machining processes. The machining was performed while milling a NAS 979 part during which the orientation of the cutting tool was fixed, and free form milling, during which the orientation of the cutting tool continuously changed. Additionally, a target object to be tracked by the camera was designed with fiducial markers to guarantee trackability with ± 90° in all directions. The design of these fiducial markers guarantee the detection of at least two distinct non-parallel markers from any view, thus preventing pose estimation ambiguities. Moreover, in order to reduce the human errors due to the construction of the camera target and placement of the markers on it, this work proposes a method for optimizing the positions of the corners of the fiducial markers in the object frame using a laser tracker. The proposed methods were compared with an Extended Kalman Filter (EKF) and the experimental results show that the proposed approaches significantly improve the pose estimation accuracy and precision of the vision based system during robotic machining while proving much more effective than the EKF approach. Moreover, the proposed methods based on sparse regression provide parsimonious models and better results when compared with the proposed LSTM based approach