Visual Persuasion in Mass Media: A Computational Framework for Understanding Visual Communication

Visuals play a vital role in human communication in the modern media landscape, but there have been little progress on a systematic analysis on massive visual data. In this dissertation, I introduce a novel computational paradigm that brings advanced techniques in computer vision and machine learning to bear on research questions in media studies and social science. I first examine how the news media leverage photographs to visually persuade their audiences. I developed a computational model that can predict the viewers' emotional perceptions from politicians' photographs in the mass media by inferring hidden messages and communicative intents embodied in images. I applied the model to track the visual favorability of the U.S. President in the mass media, revealing strong correlations between the visuals from the mass media and the public opinion. Secondly, I investigate the role of facial appearance in social perception and trait inference and its impacts to real-world social events such as elections. The result shows that the automated trait recognition from the facial appearance of politicians predicts the major election outcomes in the U.S. up to 70% accuracy. These approaches extend existing theories and behavioral studies to large-scale analysis, demonstrate the utility of computer vision and machine learning as novel research tools in computational communication science, and suggest a new area of research for computer vision, which goes beyond traditional syntactical scene understanding