Real-time direct tracking of color images in the presence of illumination variation

Abstract — This paper introduces a novel color tracking model for image registration that exploits directly the color information provided by standard color cameras. Furthermore, unlike previous approaches, the color tracking model is de-signed to handle both global and local illumination changes within a robust framework that also rejects outliers such as occluding objects, shadows, etc. In order to demonstrate the proposed approach a planar template tracking algorithm is used, however, the approach is also valid for a general class of direct tracking algorithms. In particular, the objective function is defined to be minimized directly in the CFA (color filter array) space instead of the common RGB space. It will be shown that this not only takes advantage of the discernibility of color measurements but also drastically improves the efficiency of the vision processing pipeline and ultimately improves real-time performance. A robust global illumination model is then combined with a robust M-estimation technique that is shown to handle global and local illumination changes without requiring enlarging the state space of the estimator towards non-real time proportions in order to model every local illumination variation. Results from synthetic and real sequences are presented to demonstrate the proposed concept. I