Impact of facial tattoos and paintings on face recognition systems

Abstract In recent years, face recognition technologies have shown impressive recognition performance, mainly due to recent developments in deep convolutional neural networks. Notwithstanding those improvements, several challenges which affect the performance of face recognition systems remain. The authors investigate the impact that facial tattoos and paintings have on current face recognition systems. To this end, they first collected an appropriate database containing image‐pairs of individuals with and without facial tattoos or paintings. The assembled database was used to evaluate how facial tattoos and paintings affect the detection, quality estimation, as well as the feature extraction and comparison modules of a face recognition system. The impact on these modules was evaluated using state‐of‐the‐art open‐source and commercial systems. The obtained results show that facial tattoos and paintings affect all the tested modules, especially for images where a large area of the face is covered with tattoos or paintings. The authors’ work is an initial case study and indicates a need to design algorithms which are robust to the visual changes caused by facial tattoos and paintings