Enabling Human-Robot-Interaction via Virtual and Augmented Reality in Distributed Control Systems

Production and assembly lines are nowadays transforming into flexible and interconnected cells due to rapidly changing production demands. Changes are, for example, varying locations and poses for the processed work pieces and tools as well as the involved machinery like industrial robots. Even a variation in the combination or sequence of different production steps is possible. In case of older involved machines the task of reconfiguration and calibration can be time consuming. This may lead, in addition with the expected upcoming shortage of highly skilled workers, to future challenges, especially for small and medium sized enterprises. One possibility to address these challenges is to use distributed or cloud-based control for the participating machines. These approaches allow the use of more sophisticated and therefore in most cases computationally heavier algorithms than offered by classic monolithic controls. Those algorithms range from simple visual servoing applications to more complex scenarios, like sampling-based path planning in a previously 3D-reconstructed robot cell. Moving the computation of the machine’s reactions physically and logically away from the machine control complicates the supervision and verification of the computed robot paths and trajectories. This poses a potential threat to the machine’s operator since he/she is hardly capable of predicting or controlling the robot’s movements. To overcome this drawback, this paper presents a system which allows the user to interact with industrial robot and other cyber physical systems via augmented and virtual reality. Captured topics in this paper are the architecture and concept for the distributed system, first implementation details and promising results obtained by using a Microsoft HoloLens and other visualization devices