HuMAn: Complex Activity Recognition with Multi-Modal Multi-Positional Body Sensing

Current state-of-the-art systems in literature using wearables are not capable of distinguishing large number of fine-grained and/or complex human activities, which may appear similar but with vital differences in context, such as lying on floor versus lying on bed versus lying on sofa. This paper fills the gap by proposing a novel system, called HuMAn, that recognizes and classifies complex at-home activities of humans with wearable sensing. Specifically, HuMAn makes such classification feasible by leveraging selective multi-modal sensor suites from wearable devices, and enhances the richness of sensed information for activity classification by further leveraging placement of wearables across multiple positions on human body. The HuMAn system consists of the following components: (a) a practical feature set extraction method from specific multi-modal sensor suites; (b) a novel two-level structured classification algorithm that improves accuracy by leveraging sensors in multiple body positions; and (c) improved refinement in classification of complex activities with minimal external infrastructure support (e.g., only a few Bluetooth beacons used for location context). The proposed system is evaluated with 10 users in real home environments. Experimental results demonstrate that the HuMAn system can detect 21 complex at-home activities with high degree of accuracy. For same-user evaluation strategy, the average activity classification accuracy is as high as 95 percent over all of the 21 activities. For the case of 10-fold cross-validation evaluation strategy, the average classification accuracy is 92 percent, and for the case of leave-one-out cross-validation strategy, the average classification accuracy is 75 percent