Early Exiting-Enabled Siamese Tracking for Edge Intelligence Applications

Visual object trackers based on deep neural networks have attained state-of-the-art performance in recent years. Despite the outstanding accuracy gained by deep layers, however, they also demand high computational cost and energy consumption in order to operate in real-time, making them inadequate for edge and latency-sensitive applications. In this paper, we propose an edge computing-friendly Siamese-based visual object tracker. This work concentrates on increasing the tracking speed by reducing computations through integration of side exit branches into the network, as well as skipping the multi-scale search for some frames. By employing exit branches, the tracker is capable of obtaining the result of easy samples from early layers once the criteria are satisfied. The network is trained offline to optimize a joint function that is composed of the weighted loss functions of all exit branches. During inference, the score map is derived from the network and determines the new object location, whereas multi-scale testing can identify scale updates which is only applied under specific conditions. Our proposed tracker deploys an adaptive scale search over two scales that runs at 247.5 FPS on GPU and 37.1 FPS on CPU providing a 2.5x faster rate of processing speed compared to SiamFC, with an acceptable amount of accuracy loss, especially when compared to the significant speed gain and gains in computational efficiency