Linguistic and Gestural Adaptation

Human conversants in dialog adjust their behavior to their conversational partner in many ways. In terms of language, they adapt to their partners both lexically and syntactically, by using the same referring expressions or sentence structure. In terms of gesture, they mimic their partners' gestural form, frequency, expanse and speed. However, adaptation is not about simply copying dialog partners' words, syntax, and gestures. The process of adaptation to the partner takes place under other special constraints, e.g. providing coherent and informative turns in conversation, expressing one's own personality, or achieving other social and interpersonal goals. How do speakers adapt to one another and at the same time achieve their own conversational goals?In this thesis, we first carry out an exploratory study to show that adapting to different linguistic features results in different style perceptions: adapting to hedges increase perceptions of friendliness, while adapting to syntactic structures increases perceptions of naturalness. On the basis of these results, we propose an adaptation measure that allows us to capture and model adaptation behaviors that may orient to different levels or types of linguistic representations, such as lexical, syntactical, or stylistic variations in the way speakers talk. We build and compare linguistic adaptation models using our measure with four human dialog corpora, and with different feature sets to represent different levels of linguistic representations.We also explore gestural adaptation on both particular gesture forms and gesture style. We set up our experiment in the form of two virtual gents co-telling a story. We first verify that we can express the personality of virtual agents through varying gesture parameters such as speed, height, and expanse. We then show that human subjects prefer adaptive to nonadaptive virtual agents, where the adaptive virtual agent adapts the gesture form and personality of their dialog partner