Modeling nonverbal communicative signaling in predictive social motorics

Kahl S, Kopp S. Modeling nonverbal communicative signaling in predictive social motorics. In: Rothkopf C, Balfanz D, Galuske R, et al., eds. KOGWIS2018: Computational Approaches to Cognitive Science. Darmstadt; 2018: 24.In successful social interaction, people dynamically understand, predict, and influence mental states and actions so as to enable efficient and interactive grounding of shared meaning. We effectively query our communication partners and depend on their reciprocity to grasp their state of knowledge. Communication errors are common, informative and important. Through them we get bits of information that can guide our communicative acts towards our communicative goal. Our computational model of the dynamics within the social brain and between social agents is based on hierarchical predictive processing. We want to describe how our previous modeling approach is advanced by modeling nonverbal communicative signaling, i.e., the strategic alteration of one’s own action kinematics to better achieve a communicative goal [1]. In earlier computational models we showed how the interactive grounding process between two virtual agents can work through reciprocity alone. This process was not efficient, resulting in many repetitions. To increase communication efficiency we focused on the process of self-other distinction in lower levels of the sensorimotor processing hierarchy, i.e., had an action been caused by oneself or another agent. Modeling this distinction already low in the sensorimotor hierarchy resulted not only in the ability to decide who performed perceived actions but more importantly how [2]. Our model of communicative signaling works with knowledge gathered during the interaction by matching perceived actions with our predictive processing hierarchy’s representations of action schemas. Each action schema can be produced by many similar action sequences. Especially informative are (1) the beliefs held about the communication partner and (2) the specific knowledge which sequence of actions led to a communication error, i.e., how the action schema was produced. Based on this information the communicative signaling model selects an action sequence for the next communicative act that will be most distinguishable from the sequence that previously led to a communication error. We propose a model of communicative signaling in a predictive sensorimotor system using information from reciprocity and self-other distinction. This model may help to shed light on how we communicate efficiently strategically selecting easier to disambiguate communicative acts. Our modeling approach focuses on iconic gestures but we speculate that this approach of strategic action selection may underlie many different signaling modalities