XU, SONG, HOOGS: AN EFFICIENT ONLINE HIERARCHICAL SUPERVOXEL 1 An Efficient Online Hierarchical Supervoxel Segmentation Algorithm for Time-critical Applications

Video segmentation has been used in a variety of computer vision algorithms as a pre-processing step. Despite its wide application, many existing algorithms require pre-loading all or part of the video and batch processing the frames, which introduces tempo-ral latency and significantly increases memory and computational cost. Other algorithms rely on human specification for segmentation granularity control. In this paper, we pro-pose an online, hierarchical video segmentation algorithm with no latency. The new algorithm leverages a graph-based image segmentation technique and recent advances in dense optical flow. Our contributions include: 1) an efficient, yet effective proba-bilistic segment label propagation across consecutive frames; 2) a new method for label initialization for the incoming frame; and 3) a temporally consistent hierarchical label merging scheme. We conduct a thorough experimental analysis of our algorithm on a benchmark dataset and compare it with state-of-the-art algorithms. The results indicate that our algorithm achieves comparable or better segmentation accuracy than state-of-the-art batch-processing algorithms, and outperforms streaming algorithms despite a sig-nificantly lower computation cost, which is required for time-critical applications.