Improving Action Classifiers with Depth Semester Thesis

The detection and classification of human activities and gestures in video clips has been a popular research topic during the last years. Lately, 3D data has proven to facilitate this task. In addition, the recent availabil-ity of depth sensors in normal products has produced many interesting applications for that research. In this thesis, we use depth and video information from a Microsoft Kinect device in a Hough-voting framework designed for action recognition. The depth information appears to be an invaluable addition to the set of conventional image features, if available for training and testing. However, the goal of this thesis is to use 3D data during training of the classifier, but test it on 2D data only. For the purpose of experimentation and evaluation, different datasets have been recorded with the Kinect. We present several modifications to the original Hough framework. In a first scenario, an attempt to imitate the depth information is made. The results of two different approaches show that depth imitation fails to considerably improve the average classification performance. In the second scenario, a more refined approach to regularize the training process of random decision trees using depth is adopted. The effects of this balanced use of depth data and normal image data look quite promising. Further, the use of depth as a means of image segmentation is examined. Aside from that, reconstruction of depth images from 2D data i