Development of an intelligent system for deep learning-based video object detection and tracking

Object detection and tracking are Computer Vision techniques that allow systems to detect any object and track their movements within images or video streams. Systems implementing such techniques find broad applications in many scenarios, especially regarding surveillance, so as to automatize works that need visual cognition. The challenging part in surveillance system applications is the real-timing constraint: object detection algorithms are hardware demanding in order to work in real time, especially with the usage of convolutional neural networks. Therefore, a trade-off between accuracy in results and efficiency in terms of computational time has to be considered. In this work, a study regarding the computational time performance analysis of an application for the detection and tracking of vehicles and persons from a real-time stream is proposed. The stream is provided by two surveillance cameras. The system detects objects by using the state-of-the-art Yolov3 object detection with a CNN trained on COCO dataset. Darknet and Tensorflow frameworks for neural network training and inference are used and compared, proposing several solutions to increase the computational time expressed in FPS. Two new networks have also been trained with two different versions of Yolov3 on a subset of the dataset in order to understand the impact of the number of classes on the FPS. The system will track objects across frames with background subtraction method and centroid algorithm