Learning to track multiple people in omnidirectional video

Abstract — Meetings are a very important part of everyday life for professionals working in universities, companies or governmental institutions. We have designed a physical aware-ness system called CAMEO (Camera Assisted Meeting Event Observer), a hardware/software system to record and monitor people’s activities in meetings. CAMEO captures a high resolution omnidirectional view of the meeting by stitching images coming from almost concentric cameras. Besides recording capability, CAMEO automatically detects people and learns a person-specific facial appearance model (PS-FAM) for each of the participants. The PSFAMs allow more robust/reliable tracking and identification. In this paper, we describe the video-capturing device, photometric/geometric autocalibration process, and the multiple people tracking system. The effectiveness and robustness of the proposed system is demonstrated over several real-time experiments and a large data set of videos. Index Terms — Omnidirectional-video capturing, Multiple people tracking, Subspace methods, Meeting understanding