Occlusion reasoning for tracking multiple people

Abstract—Occlusion reasoning is one of the most challenging issues in visual surveillance. In this letter, we propose a new ap-proach for reasoning about occlusions between multiple people. In our approach, occlusion relationships between people are explicitly defined and deduction of the occlusion relationships is integrated into the whole tracking framework. The prior knowledge is sup-plied by a set of models which include a 2-D elliptical shape model, a spatial-color mixture of Gaussians appearance model, and a mo-tion model with constant velocity. An observation likelihood func-tion is constructed based on the similarity between the observations and the object appearance models with given states. The occlusion relationships are deduced from the current states of the objects and the current observations, using the observation likelihood function. The previous occlusion relationships are not required for deducing the current occlusion relationships. The problem of tracking and occlusion reasoning for more than two people is formulated math-ematically, and a solution is proposed based on particle filtering. Experimental results on several real video sequences from indoor and outdoor scenes show the effectiveness of our approach. Index Terms—Occlusion reasoning, particle filtering, tracking multiple people, visual surveillance. I