Recognising Human-Object Interaction via Exemplar based Modelling

Human action can be recognised from a single still im-age by modelling Human-object interaction (HOI), which infers the mutual spatial structure information between hu-man and object as well as their appearance. Existing ap-proaches rely heavily on accurate detection of human and object, and estimation of human pose. They are thus sensi-tive to large variations of human poses, occlusion and un-satisfactory detection of small size objects. To overcome this limitation, a novel exemplar based approach is pro-posed in this work. Our approach learns a set of spatial pose-object interaction exemplars, which are density func-tions describing how a person is interacting with a manip-ulated object for different activities spatially in a proba-bilistic way. A representation based on our HOI exemplar thus has great potential for being robust to the errors in human/object detection and pose estimation. A new frame-work consists of a proposed exemplar based HOI descrip-tor and an activity specific matching model that learns the parameters is formulated for robust human activity recog-nition. Experiments on two benchmark activity datasets demonstrate that the proposed approach obtains state-of-the-art performance. 1