Early Turn-Taking Prediction for Human Robot Collaboration

To enable natural and fluent human robot collaboration, it is critical for a robot to comprehend their human partners’ on-going actions, predict their behaviors in the near future, and plan its actions accordingly. Specifically, the capability of making early predictions can allow robots to determine the precise timing of turn-taking events and start planning and executing preparative tasks to take the turn. Such proactive behavior would save waiting time and increase efficiency and naturalness in collaborative tasks. To that end, this dissertation presents the design and implementation of an early turn-taking prediction framework, centered around physical human robot collaboration tasks. The prediction framework leverages multimodal communication cues (both explicit and implicit cues) to reason about human’s incoming turn-taking intentions. After such intent is recognized, the robot would proactively engage interaction with the human to accelerate the turn switch process, aiming to increase collaboration fluency. The developed framework was evaluated in two important scenarios, the first one is healthcare where a robotic scrub nurse delivers surgical instruments to surgeons in the operating room. The second one is manufacturing where a robotic assembly assistant delivers assembly parts and tools to the human worker on the manufacturing floor. Throughout the comprehensive evaluation, it was found that the proposed turn-taking prediction framework outperformed the state-of-the-art computational alternatives in its accuracy and earliness of spotting out the correct human turn-taking intention. When compared to homogeneous human teams’ performance, the proposed algorithm is found to yield better prediction accuracies when partial temporal information is available. Such behavior indicates the proposed algorithm’s advantage in recognizing an underlying human intention that is not fully revealed yet, thus featuring its “early” capability. The robotic assistants equipped with turn-taking intelligence has been found to generate higher collaboration fluencies, shorter task completion times, more proactive behavior, and higher level of trust with robot partner, compared to the alternatives without such capability