Region-based convolutional neural network using group sparse regularization for image sentiment classification

Abstract As an information carrier with rich semantics, images contain more sentiment than texts and audios. So, images are increasingly used by people to express their opinions and sentiments in social network. The sentiments of the images are overall and should come from different regions. So, the recognition of the sentiment regions will help to concentrate on important factors the affect the sentiments. Meanwhile, deep learning method for image sentiment classification needs simple and efficient approach for simultaneously carrying out pruning and feature selection whilst optimizing the weights. Motivated by these observations, we design a region-based convolutional neural network using group sparse regularization for image sentiment classification: R-CNNGSR. The method obtains the initial sentiment prediction model through CNN using group sparse regularization to get compact neural network, and then automatically detect the sentiment regions by combining the underlying features and sentimental features. Finally, the whole image and the sentiment region are fused to predict the overall sentiment of the images. Experiment results demonstrate that our proposed R-CNNGSR significantly outperforms the state-of-the-art methods in image sentiment classification