Towards Less Supervision for Scalable Recognition of Daily Activities

This thesis is concerned with scalable recognition of human activities in real-world settings. Research towards this aim has addressed the automated detection of Activities of Daily Living, such as personal hygiene, eating, meal preparation, or housekeeping, as a particularly fruitful endeavor for elderly health care. The focus of this thesis lies on two challenges within these efforts: characterization of daily activities in sensor readings, and practical methods to label these data. We address these challenges by investigating several research directions for unobtrusive activity recognition that require only a limited number of sensors and minimal annotation overhead. We utilize a multi-sensor approach to characterize two important aspects of activities. We use wearable acceleration sensors to infer characteristic body movements and RFID tags in combination with RFID readers to recognize object usage during execution of activities. The benefit of the proposed approach is that it is able to attain high recognition performance even when the number of sensors is significantly decreased to a single wrist-worn sensor and just a few tagged objects. This is achieved by augmenting the learning process with additional information from complementary sensors. We also explore the combination of two types of sensors, namely accelerometers for body-motion and infra-red sensors for detecting indoor location where the activities are performed. The goal of this study is to investigate the applicability of two different techniques to significantly reduce the need for labeled training data. The first technique combines small amounts of labeled activity data with easily obtainable unlabeled data in a semi-supervised learning process. The second technique aims at focusing labeling efforts on the most profitable instances by utilizing active learning. The experimental results indicate that we can achieve comparable and sometimes even better performance than the fully supervised approaches. In order to further enhance the applicability of activity recognition in real-world settings, we propose a novel multi-instance learning method that is able to learn from sparsely labeled data. Instead of requiring labels for each individual training sample, we group sensor data into bags-of-activities and provide the labels only on the bag level. We propose several novel algorithmic extensions of multi-instance learning that support new labeling scenarios allowing less constrained ways of annotating activity data. We systematically analyze the trade-off between the labeling efforts and recognition performance. Lastly, we introduce several graph-based label propagation strategies for enabling long-term activity recordings without the need for detailed continuous activity annotations. We propose two different ways of combining multiple graphs based on data similarity in feature space and time. We carry out a comparative evaluation of this approach and the multi-instance learning approach. We show that the graph-based approach outperforms multi-instance learning. Overall, this thesis demonstrates the feasibility of using unlabeled data for learning more expressive activity classifiers and the potential of multi-sensor approaches to facilitate scalable activity recognition