‡Department of Psychological and Brain Sciences

Visual attention is a crucial indicator of the relative im-portance of objects in visual scenes to human viewers. In this paper, we propose an algorithm to extract objects which attract visual attention from videos. As human attention is naturally biased towards high level semantic objects in visual scenes, this information can be valuable to extract salient objects. The proposed algorithm extracts dominant visual tracks using eye tracking data from multiple sub-jects on a video sequence by a combination of mean-shift clustering and Hungarian algorithm. These visual tracks guide a generic object search algorithm to get candidate ob-ject locations and extents in every frame. Further, we pro-pose a novel multiple object extraction algorithm by con-structing a spatio-temporal mixed graph over object can-didates. Bounding box based object extraction inference is performed using binary linear integer programming on a cost function defined over the graph. Finally, the object boundaries are refined using grabcut segmentation. The proposed technique outperforms state-of-the-art video seg-mentation using eye tracking prior and obtains favorable object extraction over algorithms which do not utilize eye tracking data. 1