DIGITAL TWIN FRAMEWORK FOR HUMANROBOT INTERACTION BY MEANS OF INDUSTRY 4.0 ENABLING TECHNOLOGIES

The introduction of Industry 4.0 technologies has shaped the old form of manufactures. Despite the enormous existence of technologies such as IoT, CPS, AI or collaborative and autonomous robots in the industrial environment and while the main objective of Industry 4.0 is to implement a better connected, flexible and smarter industrial environment, some aspects still lack to be better integrated and implemented. Among these aspects, the human-robot interaction, collaborative robot programming and simulation which still need many improvements in order to fit in the new smart environments where cobots and humans work together in hybrid teams. This research envisions the future of robot programming and robot simulation in industrial environment where humans and robots work side by side in hybrid teams. The main objective of this work was to build and demonstrate a new digital twin-based framework that is designed to enhance the human-robot interaction, robot programming and realtime in-real-environment simulation. The proposed approach required to afford a flexible real-time service-based framework for both vertical and horizontal integration. It also needed to provide an intuitive and human-friendly usage for any unskilled worker. This dissertation introduces the main six steps of the digital twin for human-robot interaction proposed framework which was adapted and modified from the common 5-C architectural design of CPSs. Its flexible architecture grants a robust integration of new devices, systems or APIs. Since this framework was initially designed for human-robot interaction, its capabilities was demonstrated through a use case study and implementation. The first three-C steps of the method (Connect, Collect and Combine) should be initiated at the beginning but executed only one time during each process life-cycle. Connection establishment between physical and digital worlds is guaranteed in step one. Data Collection from physical devices was done in step two. Combining both worlds in one scene and synchronization between twin models was accomplished during step three. Data analysis, algorithms generation and motion planning are processed in step four. Then, a simulation of digital model generated motions was visualized through mixed reality interfaces and while enabling user interaction was executed during step five. At t he e nd, after approval, robot movements are generated and actions are made by the physical twin. All-along the six steps, an horizontal technological architecture was used. First, an IoT Gateway infrastructure was established to maintain the real-time data exchange between the system’s different components. Then, a MR-based immersive interface was developed through many phases to enable digital world set-up, visualization, simulation and interaction using human gestures. At the meanwhile, a broker was implemented to handle diverse tasks mainly citing the motion planning and the AI-based object pose estimation defining. The broker is also responsible on new elements integration. At the end, implemented system approved the main objectives of the proposed research methodology which are: • Intuitive robot programming: any unskilled worker can program the robot thanks to the human-friendly interface and the autonomous assistance capabilities of the robot while estimating position and planning motions. • Realistic simulation: a simulation done in real environment with unpredicted real conditions and objects. • Flexible system integration: it is easy to integrate new devices and features thanks to the broker master interface that connects all separated elements with all their diverse interfaces and platforms