Pedestrian Detection Using Motion Saliency Aided Convoltional Neural Network

Pedestrian and crowd analysis is one of the oldest problems in the area of computer vision and image processing. Researchers have proposed several algorithms that analyze crowds for different purposes aiming to improve their security and safety. The recent advancements in the areas of machine learning and computer devices resulted in a revolution in the proposed algorithms for such problem. The use of Convolutional Neural Networks in the proposed algorithms boosted the performance of the state-of-the-art algorithms. This research project studies the impact of integrating of motion saliency along with the CNN based pedestrian detector. A detailed literature review was conducted to draw the pathway of the project, discussing different approaches of motion flow algorithms and CNN based detectors. Background subtraction based on Gaussian Mixture Model motion flow algorithm was used to extract the motion saliency information. Faster R-CNN based on AlexNet was considered and integrated with the motion flow algorithm. Two different approaches of integration were proposed and tested: (1) Motion Masked Faster R-CNN method, where the motion information is used to generate a mask for the scene, keeping the regions of interest where motion is found and people may be available; (2)Motion Corrected Faster R-CNN method, where the motion information is utilized after the Faster R-CNN detector locates all pedestrian in the frame to correct the false detections. The report discusses the hardware and software setups used in this thesis project to develop and implement the proposed detectors. PETS2009 dataset was used to test the proposed detectors, which are trained on Caltech pedestrian dataset. It was found that the integration of motion information improves the precision of the Faster R-CNN detector by 7% - 30%, where the error rate dropped by 6% - 34%. A detailed discussion of further improvement pathways is provided in the report. Limitations of current implementation of the proposed detectors are discussed. The evaluation of the proposed detectors is compared to state-of-the-art detectors in the literature. Models’ performance was discussed, noting the challenges that degraded their performance and how to overcome them