Classification of Moving Objects and Recognition of Human Activity Using Infrared Surveillance Sensors

Linear pyroelectric array sensors have enabled useful classifications of objects such as humans and animals to be performed with relatively low cost hardware in border and perimeter security applications. Ongoing research has sought to improve the performance of these sensors through signal processing algorithms. In this thesis, we introduce the use of Hidden Markov Tree (HMT) models for object recognition in images generated by linear pyroelectric sensors. HMTs are trained to statistically model the wavelet features of individual objects through an expectation-maximization (EM) learning process. Human versus animal classification for a test object is made by evaluating its wavelet features against the trained HMTs using the maximum-likelihood (ML) criterion. The classification performance of this approach is compared to two other techniques; a texture, shape, and spectral component feature (TSSF) based classifier and a speeded up robust feature (SURF) based classifier. The evaluation indicates that among the three techniques, the wavelet based HMT model works well, is robust, and has improved classification performance compared to a SURF features based algorithm in equivalent computation time. When compared to the TSSF based classifier, the HMT model has slightly degraded performance but almost an order of magnitude improvement in computation time enabling real time implementation. A second goal of this research is to classify the activity of objects identified as human. If the linear pyroelectric array sensor identifies the object as human, this then triggers a thermal video camera to capture video. From the video then, the goal is to recognize the activity as either suspicious or not based on a stored activity database. Recognition of human activity is crucial for surveillance and monitoring systems. In this thesis, we investigate the recognition of motion based activity in thermal infrared video. The segmentation of human poses or motions from known or unknown backgrounds is always a challenging task due to the lighting conditions and the colors of clothing and surfaces. ViBe: A universal background segmentation technique has been employed to improve the pose segmentation from the background. We have proposed a contrast based spatio-temporal template named temporal contrast image (TCI) which can capture small motion and is useful for repetitive and non-repetitive activity recognition. Hu\u27s moment invariant feature descriptor and Naive Bayesian classifier are used for activity recognition. We have also combined our approach with existing spatio-temporal image formation techniques such as the gait energy image (GEI), motion energy image (MEI) and motion history image (MHI) for performance comparison. Experimental results on a limited set of activities demonstrate the effectiveness of our proposed approach. The method proposed in this work outperforms the statistical method for non-repetitive activity recognition. The overall goal of this research is to create a simple surveillance system that has real time moving object detection together with classification and human activity recognition. The profiling sensors described in this dissertation are relatively simple devices when compared to typical imaging cameras