SmartSAM: A Multisensor Network Based Framework for Video Surveillance and Monitoring

A network of multi modal sensors with distributed and embedded computations is con-sidered for a video surveillance and monitoring application. Practical factors limiting the video surveillance of large areas are highlighted. A network of line-of-sight sensors and mobile-agents based computations are proposed to increase the effectiveness. CMOS digi-tal cameras in which both sampling and quantization occur on the sensor focal plane are more suitable for this application. These cameras operate at very high video frame rates and are easily synchronized to acquire images synaptically across the entire network. Also, they feature highly localized short term memories and include some SIMD parallel computa-tions as an integral part of the image acquisition. This new framework enables distributed computation for piecewise stereovision across the camera network, enhanced spatio-temporal fusion, and super resolution imaging of steadily moving subjects. A top level description of the monitor, locate and track model of a surveillance and monitoring task is presented. A qualitative assessment of several key elements of the mobile agents based computation for tracking persistent tokens moving across the entire area is outlined. The idea is to have as many agents as the number of persons in the field of view, and perform the computations in a distributed fashion without introducing serious bottlenecks. The overall performance is promising when compared against that of a small network of cameras monitoring large corridors with a human operator in the loop