Semi-automatic video object segmentation basing on hierarchy optical flow

Abstract In the new MPEG-4 video coding standard, the semi-automatic video segmentation plays a key role in supporting object-oriented coding and enabling content-based functionalities. A novel hierarchy optical flow based semi-automatic video segmentation method is presented in this paper. The proposed segmentation method contains spatial and temporal segmentation. For the spatial segmentation, a point-based graphic user interface (PBGUI) is presented, with which the user can input easily, and then active contour model and tracking bug algorithm are applied to precisely define the video object of interest to be segmented. With the result of spatial segmentation, the temporal segmentation involves non-rigid object contour tracking and rigid object whole-tracking by hierarchy optical flow algorithm based on the Lucas-Kanade algorithm. And the tracking point selection algorithm is proposed to greatly improve the tracking performance in the rigid object whole-tracking. The experimental results show that the proposed algorithm can precisely segment video objects from video streams. Keywords: MPEG-4, optical flow, video segmentation, object-oriented coding 1