Video-analysis based methodologies for psychosocial research

Non-verbal facial synchrony is a widely researched topic in the field of psychosocial research. During this time of the pandemic, researchers are trying to conduct remote human subject studies for psychosocial research. The analysis of human behavior and how they synchronize in a dyadic interaction requires collecting behavioral and physiological data through human subject studies. Most of the studies are highly dependent on physical hardware like infrared gaze detection or facial electromyography to collect behavioral and physiological data. However, the dependency on hardware is a major obstacle while conducting the human subject study remotely via computer-mediated communication (CMC) like Zoom or Webex, as it is not feasible to send hardware to the location of subjects for data collection. This work presented a video-analysis method for collecting and analyzing the facial behavioral data and quantify interpersonal non-verbal facial synchrony. The video-analysis method will support the remote human subject study condition by eliminating the dependency on hardware and help researchers to conduct remote human subject studies. An experiment was designed to compare the video-analysis-based method against the traditional facial electromyography to detect facial expressions, the results are compared as proof of concept to validate the method. The results showed that the video-analysis method produces similar excitation trends for facial expressions like Happy and Frown as facial electromyography, the results also correlate with the manual hand-coded events for expressions. A remote human subject study was designed for the quantification of interpersonal non-verbal facial synchrony, the study was conducted via Zoom. Two subjects were asked to perform specific facial expressions in six different tasks while the principal investigator responded to their expressions to create an interaction with synthetic synchrony events. The non-verbal facial synchrony was quantified and compared based on the global synchrony score metric of each interaction. The video-analysis method generated a standard global synchrony score for each interaction and provided detailed information about the interaction, such as who was leading and following the emotion and the valence of the synchrony. The global synchrony score metric is further used to analyze the secondary data from the SCOTTIE project, demonstrating the application of the video-analysis method on naturally occurring synchrony events in dyadic interaction