Image segmentation in video sequences: A probabilistic approach

“Background subtraction ” is an old technique for finding moving objects in a video sequence—for example, cars driving on a freeway. The idea is that subtracting the current image from a time-averaged background image will leave only non-stationary objects. It is, however, a crude ap-proximation to the task of classifying each pixel of the current image; it fails with slow-moving objects and does not distinguish shadows from moving objects. The basic idea of this paper is that we can classify each pixel using a model of how that pixel looks when it is part of different classes. We learn a mixture-of-Gaussians classi-fication model for each pixel using an unsuper-vised technique—an efficient, incremental ver-sion of EM. Unlike the standard image-averaging approach, this automatically updates the mixture component for each class according to likelihood of membership; hence slow-moving objects are handled perfectly. Our approach also identifies and eliminates shadows much more effectively than other techniques such as thresholding. Ap-plication of this method as part of the Roadwatch traffic surveillance project is expected to result in significant improvements in vehicle identification and tracking.