Multimodal fingerprint and face biometrics with fragile watermarking and convolutional neural network

The rapidly growing use and storage of private, sensitive, and personal information across different applications have given rise to the need to restrict access to such information; thus, leading to the development of biometric authentication. Multimodal biometric authentication has improved system accuracy, but it has not been able to overcome all the vulnerabilities of biometric authentication. To reduce the amount of data that is stored or communicated during the authentication process and to ensure the authenticity of the biometric templates, image watermarking techniques have been used to embed the information in one template over the other. These techniques are either robust or fragile. The robust method can compress the watermarked images but has a limited ability to detect tampering, whereas fragile methods can detect tampering but does not allow watermarked images to be compressed. In this thesis, a new watermarking method is proposed, based on the Discrete Cosine Transform (DCT) method and the Least Significant Bit (LSB) method. The LSBs of the quantized DCT coefficients of a face image are manipulated according to the values of a binarized fingerprint image. This combination was used to allow the storing and communication of the watermarked images using the popular JPEG format. Since the binarized fingerprint image passes only through lossless compression, i.e. Huffman encoding, the results showed that the fingerprint information before watermarking and that extracted from the watermarked image are identical. Moreover, because all frequency ranges were used in the DCT format of the face image, the results showed that the proposed method had not significantly affected the image, i.e. the cover image, unlike other existing methods. As the watermark information is not hand-crafted, tamper detection could not be achieved by comparing a static image to the extracted watermark. Thus, a Machine- Learning (ML)-based method was implemented to detect the existence of fingerprint patterns in the watermark. However, as the proposed system used a Convolutional Neural Network (CNN) to measure the similarity between the templates collected from the user and those stored in the model database, tamper detection was already embedded in the same neural network. Accordingly, this neural network output an authentication measure that represents the probability that the collected templates are authentic. A high authenticity measure indicates that the collected templates match the model’s templates and that there was no tampering of the received templates. Experiments were conducted to evaluate the performance of the proposed system. The results show a 98.96% average accuracy, where each prediction took an average processing time of 139.06 ms. The results also showed that the accuracy of tampering detection was 100%. Besides, the size of the files on the disk (or the bandwidth required to communicate the files) was reduced to less than 50% of their original size using the proposed fragile multibiometric watermarking technique. Hence, the proposed methodology was able to yield outstanding performance, compared to existing state-of-the-art methods, while achieving the objectives of the study, namely, to reduce the file size and the time required to authenticate legitimate users while retaining the ability to detect tampering