Activity Recognition and Stress Detection for Manual and Automated Driving

This work aims to explore methods to model physical and mental user states in an automated driving scenario. For this aim, a driving simulator study is conducted in which subjects are tasked to drive themselves or work on a secondary tasks while the car was driving automatically. On the basis of the findings from this study, a statistical classifier is constructed which aims to capture the passengers action state and level of stress. In this work, we use the full body model obtained from using openpose on our visual data as a baseline to asses the drivers’ joint locations. We further use a hierarchical approach to first derive semantically motivated primitive features, namely the position of the subjects’ hands and the subjects’ head rotation. These primitives are then used to classify poses yielding information on the passengers action state. Secondly, we explore strategies to model the subjects’ stress level using heart rate data. We present both a baseline approach depending solely on the passengers’ age and gender and a second approach, where we train a mixture model on previously gathered subject data. We find that using a hierarchical approach, we are able to reach classification accuracy of on average 75%-85% depending on the classifier used. We find that both heart rate approaches yield the same, explainable pattern. The topic of this work is part of a bigger project dealing with improving human-machine understanding and communication in automated driving. It aims at creating a framework for detecting and quantifying both the passengers physical and mental state. When possible and needed, adaptation strategies are then suggested based on the user state, the person’s user profile and the current state of the world, in order to provide a positive impact on the person’s current state