Simultaneously Modelling and Tracking using Mutual Information

The aim of this work is to track non-rigid objects using appearance in real-time without a pre-learned model. The tracker should be robust to noise, occlusions, changes in lighting conditions, and pose variations. Recovery from tracking failure should be possible. The approach taken is to localise small image patches within each frame of a video sequence using function optimisation. Mutual Information (MI) is chosen as the similarity metric, due to its robustness to outlier pixel values and lack of assumptions about linear intensity relationships. The choice of similarity metric and descriptor strongly influence performance, whatever the application, hence the contributions made here have wide applicability. Despite the wide use of MI for registration, many open problems still exist. MI suffers from artefacts in its function surface, which can prevent convergence. Numerous different MI methods have been proposed to overcome artefacts, but few methods have published analytic derivatives, precluding their use in fas