Embedded computing techniques for remote mobile video surveillance systems

Unmanned aerial vehicles (UAVs) equipped with cameras are increasingly being deployed for performing vision-based wide area surveillance with minimal human intervention. Existing techniques employing global motion estimation (GME) for automatic surveillance are typically complex and compute-intensive. In this thesis, low-complexity techniques for the key functional blocks of the GME, namely, corner detection, feature tracking and robust estimation have been proposed. The proposed techniques are capable of adapting to varying image content, camera motions and moving targets, thereby making them suitable for real-time processing of aerial videos on resource-limited UAVs. A novel compute-efficient pruning technique (called PP-ER) for corner detection is proposed using simple approximations of the Shi-Tomasi and Harris corner measures for rapid and efficient extraction of high-quality corner candidates. This allows for restricting the complex corner measure computations to only a small pool of corner candidates. Evaluations on a Nios-II platform, without floating point unit (FPU) show a speedup in execution time of 48-82% in Shi-Tomasi and 45-81% in Harris corner detection when detecting 300 corners, at the same time achieving comparable accuracy as the conventional Shi-Tomasi/Harris detectors, when applied to the Oxford repeatability dataset. In order to eliminate the need for manual setting of optimal threshold, to guarantee the required number of corners for a wide range of image content, an automated thresholding method based on iterative thresholding is proposed, in this thesis. This has led to notable reduction in the number of trials needed to release the required corner candidates. In order to further enhance compute efficiency, a mask-based non-maximal suppression scheme is employed to turn off neighbours of already selected corners. Evaluations show that with automated thresholding, an average of only 1.8% and 4.3% of the corner candidates are released, when compared with fixed threshold based detection of 1000 corners on Shi-Tomasi and Harris detectors respectively. The low complexity pruning and automated thresholding were integrated into the corner detection process, to reduce the computational complexity and to eliminate the need for manual intervention for threshold selection, respectively. Evaluations on the Nios-II platform with a floating point unit, show an average speed-up in execution time of 67% for Shi-Tomasi and 51% in Harris corner detectors without compromising on the quality of the corners reported. Next, a low-complexity GME method (called sparse-GME) is proposed by employing a minimum number of well-distributed sparse corner features. A selective and systematic re-population strategy has also been introduced to improve the accuracy prior to necessitating a uniform increase in the overall density of features. Methods for rapid evaluation of GM estimations were introduced to facilitate this iterative process. Evaluations on aerial video datasets show that for 95% of the frames, GME with the first pass sparse estimation is performed, while achieving a similar accuracy as the dense set of features for 97% of the cases. Results with simulated data show that the proposed method is able to deterministically ramp up the features when the number of moving objects is increased. To cope with significant distortions, a novel adaptive windowing method was introduced within the Kanade-Lucas-Tomasi (KLT) feature tracker. This has eliminated unnecessary computations and enhanced the robustness of GME during fast rotations and scale changes in the camera motion. Evaluations with a benchmark tracking dataset show that the proposed adaptive windowing method outperforms the conventional fixed-window KLT in terms of robustness. In addition, compared to the well-known affine KLT, the proposed method achieves comparable robustness at an average runtime speedup of 7x. On simulated frames with global motions of in-plane rotations and scale changes, applying the robust adaptive windowing for KLT leads to 70% reduction in the GME error for the proposed sparse-GME. Finally, a unified computation framework is proposed to show how the proposed techniques for the individual modules of GME, can be fully integrated to realize an adaptive and low-complexity GME for deployment on low-resource platforms in aerial video surveillance systems.Doctor of Philosophy (SCE