Automatically recognizing humor in memes

Memes are progressively becoming popular and they have been spreading in the internet through social media platforms. Usually in the form of an image, video and/or text, people share memes primarily due to its humorous nature that makes it viral. Memes are commonly perceived humorous and some businesses take advantage of them and use them as an advertising or marketing strategy to promote their product or service; however, not all memes are perceived to be humorous. Though it is natural for most humans to recognize humor, making computers understand and recognize humor still remains a challenge. As such, this study compared different models in recognizing humor in memes, using textual and image features to represent humor theories on incongruity, superiority and relief. Data was collected from Twitter and 9GAG, and manually labeled humorous or non-humorous through crowdsourcing. Different models were trained using the Twitter dataset, the 9GAG dataset and the combination of these two. However, the best performing model for the three datasets were different but used the same configuration of separate feature extraction method using only textual features - Twitter: RF, 9GAG: SVM, Merged: CNN. The text within the image and the post were processed separately. Results show that using the actual image as features did not increase the performance of the models. Feature selection showed that features based on incongruity, text similarity, and structure were the most important sets for all the datasets; however, joke words were also relevant for Twitter. Comparing the performance of using only structure features with the other features, both models performed at par. While the performance of the models from the merged Twitter and 9gag dataset were not significantly worse, the fact that the features used by the Twitter and 9GAG models were different suggest that different models should be made for the two meme sources