Eye-LRCN: A long-term recurrent convolutional network for eye blink completeness detection

Computer vision syndrome causes vision problems and discomfort mainly due to dry eye. Several studies show that dry eye in computer users is caused by a reduction in the blink rate and an increase in the prevalence of incomplete blinks. In this context, this article introduces Eye-LRCN, a new eye blink detection method that also evaluates the completeness of the blink. The method is based on a long-term recurrent convolutional network (LRCN), which combines a convolutional neural network (CNN) for feature extraction with a bidirectional recurrent neural network that performs sequence learning and classifies the blinks. A Siamese architecture is used during CNN training to overcome the high-class imbalance present in blink detection and the limited amount of data available to train blink detection models. The method was evaluated on three different tasks: blink detection, blink completeness detection, and eye state detection. We report superior performance to the state-of-the-art methods in blink detection and blink completeness detection, and remarkable results in eye state detection.This work was supported in part by the Portuguese Foundation for Science and Technology (FCT) through the framework of the Strategic Funding under Grant UIDB/04650/2020. The work of Beatriz Remeseiro and Oscar Luaces was supported in part by the Ministry of Science and Innovation, Spain, under Grant PID2019-109238GB-C21