Computer Vision-Based Inspection System for Worker Training in Build and Construction Industry

Recently computer vision has been applied in various fields of engineering successfully ranging from manufacturing to autonomous cars. A key player in this development is the achievements of the latest object detection and classification architectures. In this study, we utilized computer vision and the latest object detection techniques for an automated assessment system. It is developed to reduce the person-hours involved in worker training assessment. In our local building and construction industry, workers are required to be certificated for their technical skills in order to qualify working in this industry. For the qualification, they are required to go through a training and assessment process. During the assessment, trainees implement an assembly such as electrical wiring and wall-trunking by referring to technical drawings provided. Trainees’ work quality and correctness are then examined by a team of experts manually and visually, which is a time-consuming process. The system described in this paper aims to automate the assessment process to reduce the significant person-hours required during the assessment. We employed computer vision techniques to measure the dimensions, orientation, and position of the wall assembly produced hence speeding up the assessment process. A number of key parts and components are analyzed and their discrepancies from the technical drawing are reported as the assessment result. The performance of the developed system depends on the accurate detection of the wall assembly objects and their corner points. Corner points are used as reference points for the measurements, considering the shape of objects, in this particular application. However, conventional corner detection algorithms are founded upon pixel-based operations and they return many redundant or false corner points. In this study, we employed a hybrid approach using deep learning and conventional corner detection algorithms. Deep learning is employed to detect the whereabouts of objects as well as their reference corner points in the image. We then perform a search within these locations for potential corner points returned from the conventional corner detector algorithm. This approach resulted in highly accurate detection of reference points for measurements and evaluation of the assembly