An Artificial Intelligence-based Analysis on Human Behaviors for Digital Twin Development in Manufacturing System with Workers

Recent improvements of manufacturing processes and technologies have led to a work-space where industrial robots and engineers cooperate to execute manufacturing tasks. Consequently, the role of human workers has become more crucial, and clear understandings of their behav- iors are required. Yet, related studies are lacking, especially in the term of building digital twins. This paper presents an innovative method to investigate human workers in manufacturing sys- tems by collecting data through a human-in-the-loop (HITL) experiment based on virtual reality (VR) technology and analyzing using artificial intelligence (AI) techniques. First, a HITL exper- iment was designed and performed in a VR environment with a simple manufacturing task to collect behavior data of participants: hand behaviors, time consumed, and failures. To see if data collected could discriminate participants, multiple machine learning (ML) models were trained using the hand behaviors??? data. Most ML models showed an accuracy over 90%. In addition, an agent is modeled using the remaining data as inputs. The overall relationship between the physi- cal capabilities and the system was confirmed from the simulation results. The findings show the feasibility of applying AI and VR based HITL to examine human workers and extending it to develop realistic digital twins of the manufacturing systems