Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-Level Multimodal Sentiment Analysis

www.gelbukh.com We present a novel way of extracting fea-tures from short texts, based on the acti-vation values of an inner layer of a deep convolutional neural network. We use the extracted features in multimodal senti-ment analysis of short video clips repre-senting one sentence each. We use the combined feature vectors of textual, vis-ual, and audio modalities to train a classi-fier based on multiple kernel learning, which is known to be good at heteroge-neous data. We obtain 14 % performance improvement over the state of the art and present a parallelizable decision-level da-ta fusion method, which is much faster, though slightly less accurate.