Multimodal Sentiment Analysis of Spanish Online Videos

Abstract—The number of videos available online and elsewhere is continuously growing, and with this the need for effective methods to process the vast amount of multimodal information shared through this media. This paper addresses the task of multimodal sentiment analysis, and presents a method that integrates linguistic, audio, and visual features for the purpose of identifying sentiment in online videos. We focus our experiments on a new dataset consisting of Spanish videos collected from the social media website YouTube and annotated for sentiment polarity. Through comparative experiments, we show that the joint use of visual, audio, and textual features greatly improves over the use of only one modality at a time. Moreover, we also test the portability of our multimodal method, and run evaluations on a second dataset of English videos. Index Terms—sentiment analysis, multimodal natural language processing