Technologies and paradigms for natural human-robot collaboration

As humans, we are good at conveying intentions, working on shared goals. We coordinate actions, and interpret contextual information from behavioural cues. We can perform all of the above entirely while performing a collaborative task in a team. The research in human-human interaction (HHI) has set a precedent for human-robot interaction (HRI) to develop frameworks facilitating teamwork among humans and robots. To design frameworks where collaborative robots can work alongside humans, it is essential to endow the robots with communication abilities. With the advancements in the field of computer vision and robotics, it is now possible to integrate natural-ways of human communication in the HRI frameworks. In the context of human-robot collaboration, this dissertation presents technologies which are developed to facilitate a natural, human-like collaboration with the robots. The technologies allow naive users to work with the collaborative robot in close proximity.During a task, the user instructs the robot using static hand gestures, to which it responds with a gaze movement indicating the action it is going to perform. Once the user approves of the indicated action, the robot proceeds to execute it. The dissertation also presents an analysis of human behaviour towards a change in the behaviour of the robot. The work presented in the dissertation focuses on three central topics related to HRI, (1) a vocabulary of the static hand gestures performed by humans in a collaborative task, (2) teaching semantics of the gestures to the robot during the task, and (3) understanding human behaviour in response to the ability of the robot to execute the task.Concerning the first topic, a gesture detection and pose estimation method was developed to detect the static hand gestures independent of the upper/full-body pose. The method was evaluated on two datasets, Innsbruck Pointing at Objects (IPO) dataset and Innsbruck Multi-view Hand Gesture (IMHG) dataset, consisting of the gestures with the different shape, size, and colour of hands. Further analysis of the datasets led to some of the key findings associated with deictic gestures. The pointing of the user was certainly affected because of the effect of parallax caused due to the offset between head and hand.A systematic bias was found when the users pointed at targets away from themselves compared to the targets near them. Similarly, a bias was observed while pointing at targets on the ipsilateral side as compared to the contralateral side.To address the second topic, a HRI framework namely, Proactive Incremental Learning (PIL), was developed. The PIL framework is a probabilistic, statistically-driven approach. To best represent the intentions of the users, the PIL framework gave them the freedom to teach semantics of the gestures to the robot at will. The users develop associations between the hand gestures and the robot actions on the fly, i.e. during the task. The PIL framework provides the robot with gesture predicting capabilities once it has consolidated a few gesture-action associations. Consequently, the robot is able to perform manipulation actions proactively without an instruction from the user. Furthermore, based on the interaction history the robot is capable of anticipating its next action. To best evaluate the PIL framework, it was tested on a table assembly task in a simulated as well as a real-robot environment. In the simulation, various detection rates were simulated to observe the effect of an accurate gesture detection on completion of the task. While in the real environment, the PIL framework was tested based on the evaluation metrics, such as interaction effort, neglect tolerance, and robot attention demand, which are used for comparing HRI frameworks.Lastly, in regard to the third topic, a user study was conducted with non-roboticists -- people with no experience in working with a robot -- to compare proactive and reactive behaviours of the robot. The proactive robot used the PIL framework to operate, whereas the reactive robot executed the actions only after an instruction was given. The reactive robot lacked gesture prediction and action anticipation capabilities. Various behavioural cues such as gaze, gesture, and speech, of the participants, were analysed from the video recordings of the study. The user study showed further insights in balancing the two aspects of HRI, user engagement and task efficiency.While the reactive robot engaged the user more as compared to the proactive robot, it was slower in executing the task. The proactive robot was appreciated for its quick task learning and execution abilities.Dadhichi ShuklaKumulative Dissertation aus fünf ArtikelnUniversität Innsbruck, Dissertation, 2018OeBB(VLID)275851