A Computer Vision Sensor for Efficient Object Detection Under Varying Lighting Conditions

Convolutional neural networks (CNNs) have attracted much attention in recent years due to their outstanding performance in image classification. However, changes in lighting conditions can corrupt image segmentation conducted by CNN, leading to false object detection. Even though this problem can be mitigated using a more extensive CNN training set, the immense computational and energy resources required to continuously run CNNs during always‐on applications, such as surveillance or self‐navigation, pose a serious challenge for battery‐reliant mobile systems. To tackle this longstanding problem, a vision sensor capable of autonomously correcting for sudden variations in light exposure, without invoking any complex object detection software, is proposed. Such video preprocessing is efficiently achieved using photovoltaic pixels tailored to be insensitive to specific ranges of light intensity alterations. In this way, the pixels behave similarly to neurons, wherein the execution of object detection software is only triggered when light intensities shift above a certain threshold value. This proof‐of‐concept device allows for efficient fault‐tolerant object detection to be implemented with reduced training data as well as minimal energy and computational costs and demonstrates how hardware engineering can complement software algorithms to improve the overall energy efficiency of computer vision