Simulation scan comparison for process monitoring using 3D scanning in manufacturing environments

In order for robotics to be used in automated production facilities, the manufacturing systems must be highly constrained. Despite the efforts of designers to generate accurate simulations, differences often occur in the real implementation. These differences are especially common in flexible manufacturing where people are required to operate in the same space as automated machines. Objects located in unexpected positions or orientations, additional objects in the area or missing objects can cause a robot to perform the designated operations incorrectly leading to wastage and loss of production. Typically, robotic systems have no information about the real environment around it and operate blind. In order to provide more information to the robot about its environment, information such as 3D scans can be provided to the system. This research compares a 3D scan of what is present in the environment to a 3D simulation of what is expected in the environment. The simulation scan comparison (SSC) process detects missing, unknown and incorrectly located objects. The results from the comparison process can be used to make decisions about how operations should be performed so that the outcome is satisfactory. For example, if an unknown object is in the path of a robot, the robot can be rerouted around the object. The surfaces of both simulation objects and 3D scan data are described using triangles, allowing triangle neighbour data to be used for more efficient identification of unknown objects, and triangle normal data to detect missing objects. In addition, occupancy grids have been employed for both the path planning and surface-matching processes further increasing the computational efficiency of the system. The SSC process was used to detect missing, unknown and incorrectly located objects in a typical robot cell to an accuracy of 98 %, thus demonstrating that the SSC process is capable of identifying unknown objects or unexpected situations and using these detections to update the original simulation or alter the manufacturing proces