The effects of situated visualized telepresence on task performance in gesture-based teleoperation

This paper introduces a method for gesture-based teleoperation where the human operator can virtually exist within the remote environment using mixed reality. The proposed method visually mixes the operator’s body with their perception of the remote environment, so that the operator is virtually present at the remote site. The idea is the that operator can interact with a remote task directly using their own body while relying on the motion-mimicking of the telemanipulator, rather than having to control the telemanipulator based on visual feedback to execute the task indirectly. This study aims to find out whether the direct coupling of the situated condition has a positive effect on task performance compared to a control condition where the operator perceives themselves and the remote environment separately. A prototype of a gesture-based master-slave teleoperation system was developed[1] using a two-dimensional camera feed of the remote environment¬¬. A human subject study was carried out where 26 novice participants executed a self-paced reactionary task in two dimensions, where they had to balance completion time with error rate. The task consisted of hitting a randomly spawning target 90 times by reaching out and coming back to the starting position. During the experiment, motion input was recorded along with timestamps of the spawns and the successful hits. The results showed an 8.4% faster completion time, a 16.7 % higher maximum input velocity, and a 44.2% lower error rate for the situated condition versus the separated condition. The participants also filled in a questionnaire, which showed that the majority subjectively preferred using the situated condition. For future research, this study recommends focusing on experimenting with different types of tasks, such as precision tasks and tasks of increased difficulty and complexity, as well as expanding the system to three-dimensional visual perception. [1] https://github.com/Thomahawkuru/panda_experiment https://www.youtube.com/watch?v=W2bA0NZ4GY0 Performing a set of 30 spawns using the separated condition https://www.youtube.com/watch?v=7zLhqgICGTE&feature=youtu.be performing a set of 30 spawns using the situated conditionMechanical Engineerin