Improving motion-based activity recognition with ego-centric vision

Human activity recognition using wearable computers is an active area of research in pervasive computing. Existing works mainly focus on the recognition of physical activities or so called activities of daily living by relying on inertial or interaction sensors. A main issue of those studies is that they often focus on critical applications like health care but without any evidence that the monitored activities really took place. In our work, we aim to overcome this limitation and present a multi-modal egocentric-based activity recognition approach which is able to recognize the critical objects. As it is unfeasible to expect always a high quality camera view, we enrich the vision features with inertial sensor data that represents the users' arm movement. This enables us to compensate the weaknesses of the respective sensors. We present first results of our ongoing work on this topic