Accurate and efficient human activity recognition

© 2018 Dr. Weihao ChengHuman Activity Recognition (HAR) is a promising technology which enables artificial intelligence systems to identify user's physical activities such as walking, running, and cycling. Recently, the demand for HAR is continuously increasing in pace with the rapid development of ubiquitous computing techniques. Major applications of HAR including fitness tracking, safety monitoring, and contextual recommendation have been widely applied in people's daily lives. For example, a music App on smartphones can use HAR to detect the current activity of the user and recommend activity-related songs. State-of-the-art HAR methods are based on the machine learning technique, where a classification model is trained on a dataset to infer a number of predefined activities. The data for HAR is usually in the form of time series, which can be collected by sensors such as accelerometers, microphones, and cameras. In this thesis, we mainly focus on HAR using the data from inertial sensors, such as accelerations from accelerometers. A large number of existing studies on HAR aim to obtain high recognition accuracy. However, efficiency is also an important aspect of HAR. In this thesis, we attempt to improve HAR methods for both accuracy and efficiency. Toward this goal, we first devise accurate HAR methods, and then improve the efficiency of HAR while maintaining the accuracy. More specifically, we tackle three problems. The first problem is to accurately recognize the current activity during activity transitions. Existing HAR methods train classification models based on tailored time series containing single activity. However, in practical scenarios, a piece of time series data could capture multiple interleaving activities causing activity transitions. Thus, recognition of the current activity, i.e., the most recent one, is a critical problem to investigate. The second problem is to accurately predict complex activities from ongoing observations. Many time-critical applications, such as safety monitoring, require early recognition of complex activities which are performed over a long period of time. However, without being fully observed, complex activities are hard to be recognized due to their complicated patterns. Therefore, predicting complex activities from ongoing observations is an important task to study. The third problem is to improve energy-efficiency of HAR on mobile devices while maintaining high accuracy. Many applications of HAR are based on mobile devices. However, due to the limited battery capacity, real-time HAR requires minimization of energy cost to extend the operating spans of the devices. Generally, the cost can be cut down by reducing algorithmic computations and sensing frequencies. Yet it is worth to find a maximal cost reduction while preserving a high recognition accuracy. In this thesis, we present a set of algorithms to address the proposed problems. The key contributions of the thesis can be summarized as follows: 1. We propose a method to accurately recognize the current activity in the presence of multiple activities with transitions. The method partitions a time series matching the occurring activities, where the maximum classification error of the activities is minimized. 2. We propose a method to accurately predict complex activities over time from ongoing multivariate time series. The method utilizes an action sequence model and a complex activity model, which make predictions alternately based on each other as the observed data increases. 3. We propose a method to minimize the computational cost of HAR while maintaining high recognition accuracy. The method uses a Markov Decision Process (MDP) to select an optimal subset of feature representations for ensemble classification that minimizes redundant computations. 4. We propose a method to minimize a combined measurement of sensing cost and classification error of HAR. The method uses MDP to select appropriate sensing rate to sample the incoming data points, where the sparsity of the outcome time series is ensured to preserve the recognition accuracy