Detection of background forgery using a two-stream convolutional neural network architecture

In this paper, a novel two-stream convolutional neural network (CNN) method is proposed to determine whether or not the background of an image is replaced with that of another image. Although removing or changing the background of an image is one of the most common image forgery techniques, to the best of our knowledge, this is the first passive image forgery detection (IFD) method in the literature that directly addresses such forgery. The proposed background forgery detection network (BFDNET) consists of two identical DenseNet-based feature extractors and a CNN-based classifier. The method first divides the given image into the foreground containing people in the image and background regions using the Mask R-CNN model in the preprocessing phase. Then, the features from the foreground and background regions are extracted using feature extractors. At last, the feature pair of the regions are combined and classified using a CNN-based classification network to determine if the background of the input image is changed. While conducting this study, the findings of some ablation experiments are used to determine the structure of the model and some important strategies. In addition, the proposed method is compared with nine state-of-the-art image splicing detection (ISD) methods using a novel dataset in the experiments. Experiments demonstrate that the proposed method achieves an accuracy of 95.5% in detecting background forgeries. The proposed method can improve the accuracy by over 14% compared to prior works