Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke

[EN] Social cognition is the innate human ability to interpret the emotional state of others from contextual verbal and non-verbal information, and to self-regulate accordingly. Facial expressions are one of the most relevant sources of non-verbal communication, and their interpretation has been extensively investigated in the literature, using both behavioral and physiological measures, such as those derived from visual activity and visual responses. The decoding of facial expressions of emotion is performed by conscious and unconscious cognitive processes that involve a complex brain network that can be damaged after cerebrovascular accidents. A diminished ability to identify facial expressions of emotion has been reported after stroke, which has traditionally been attributed to impaired emotional processing. While this can be true, an alteration in visual behavior after brain injury could also negatively contribute to this ability. This study investigated the accuracy, distribution of responses, visual behavior, and pupil dilation of individuals with stroke while identifying emotional facial expressions. Our results corroborated impaired performance after stroke and exhibited decreased attention to the eyes, evidenced by a diminished time and number of fixations made in this area in comparison to healthy subjects and comparable pupil dilation. The differences in visual behavior reached statistical significance in some emotions when comparing individuals with stroke with impaired performance with healthy subjects, but not when individuals post-stroke with comparable performance were considered. The performance dependence of visual behavior, although not determinant, might indicate that altered visual behavior could be a negatively contributing factor for emotion recognition from facial expressions.This study was funded by Conselleria de Educacion, Cultura y Deporte of Generalitat Valenciana of Spain (Project SEJI/2019/017), and Universitat Politecnica de Valencia (Grant PAID-10-18).Maza, A.; Moliner, B.; Ferri, J.; Llorens Rodríguez, R. (2020). Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke. Frontiers in Neurology. 10:1-12. https://doi.org/10.3389/fneur.2019.0141511210Nijsse, B., Spikman, J. M., Visser-Meily, J. M. A., de Kort, P. L. M., & van Heugten, C. M. (2019). Social cognition impairments are associated with behavioural changes in the long term after stroke. PLOS ONE, 14(3), e0213725. doi:10.1371/journal.pone.0213725Feldman, R. S., White, J. B., & Lobato, D. (1982). Social Skills and Nonverbal Behavior. Development of Nonverbal Behavior in Children, 259-277. doi:10.1007/978-1-4757-1761-7_10Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45(1), 174-194. doi:10.1016/j.neuropsychologia.2006.06.003Frith, C. D., & Frith, U. (2006). How we predict what other people are going to do. Brain Research, 1079(1), 36-46. doi:10.1016/j.brainres.2005.12.126Zinchenko, O., Yaple, Z. A., & Arsalidou, M. (2018). Brain Responses to Dynamic Facial Expressions: A Normative Meta-Analysis. Frontiers in Human Neuroscience, 12. doi:10.3389/fnhum.2018.00227Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384-392. doi:10.1037/0003-066x.48.4.384Srinivasan, R., & Martinez, A. M. (2021). Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild. IEEE Transactions on Affective Computing, 12(3), 707-721. doi:10.1109/taffc.2018.2887267Smith, M. L., Grühn, D., Bevitt, A., Ellis, M., Ciripan, O., Scrimgeour, S., … Ewing, L. (2018). Transmitting and decoding facial expressions of emotion during healthy aging: More similarities than differences. Journal of Vision, 18(9), 10. doi:10.1167/18.9.10Thompson, A. E., & Voyer, D. (2014). Sex differences in the ability to recognise non-verbal displays of emotion: A meta-analysis. Cognition and Emotion, 28(7), 1164-1195. doi:10.1080/02699931.2013.875889Doležal, J., & Fabian, V. (2015). 41. Application of eye tracking in neuroscience. Clinical Neurophysiology, 126(3), e44. doi:10.1016/j.clinph.2014.10.200Guo, K. (2012). Holistic Gaze Strategy to Categorize Facial Expression of Varying Intensities. PLoS ONE, 7(8), e42585. doi:10.1371/journal.pone.0042585Guo, K., Soornack, Y., & Settle, R. (2019). Expression-dependent susceptibility to face distortions in processing of facial expressions of emotion. Vision Research, 157, 112-122. doi:10.1016/j.visres.2018.02.001Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11(4), 860-865. doi:10.1037/a0022758Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J. Y., & Franconeri, S. L. (2014). Eye movements during emotion recognition in faces. Journal of Vision, 14(13), 14-14. doi:10.1167/14.13.14Guo, K., & Shaw, H. (2015). Face in profile view reduces perceived facial expression intensity: An eye-tracking study. Acta Psychologica, 155, 19-28. doi:10.1016/j.actpsy.2014.12.001Guo, K. (2013). Size-Invariant Facial Expression Categorization and Associated Gaze Allocation within Social Interaction Space. Perception, 42(10), 1027-1042. doi:10.1068/p7552Sirois, S., & Brisson, J. (2014). Pupillometry. WIREs Cognitive Science, 5(6), 679-692. doi:10.1002/wcs.1323Eckstein, M. K., Guerra-Carrillo, B., Miller Singley, A. T., & Bunge, S. A. (2017). Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development? Developmental Cognitive Neuroscience, 25, 69-91. doi:10.1016/j.dcn.2016.11.001Ariel, R., & Castel, A. D. (2013). Eyes wide open: enhanced pupil dilation when selectively studying important information. Experimental Brain Research, 232(1), 337-344. doi:10.1007/s00221-013-3744-5Zekveld, A. A., & Kramer, S. E. (2014). Cognitive processing load across a wide range of listening conditions: Insights from pupillometry. Psychophysiology, 51(3), 277-284. doi:10.1111/psyp.12151De Gee, J. W., Knapen, T., & Donner, T. H. (2014). Decision-related pupil dilation reflects upcoming choice and individual bias. Proceedings of the National Academy of Sciences, 111(5), E618-E625. doi:10.1073/pnas.1317557111Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45(4), 602-607. doi:10.1111/j.1469-8986.2008.00654.xDuque, A., Sanchez, A., & Vazquez, C. (2014). Gaze-fixation and pupil dilation in the processing of emotional faces: The role of rumination. Cognition and Emotion, 28(8), 1347-1366. doi:10.1080/02699931.2014.881327Lanata, A., Armato, A., Valenza, G., & Scilingo, E. P. (2011). Eye tracking and pupil size variation as response to affective stimuli: a preliminary study. Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare. doi:10.4108/icst.pervasivehealth.2011.246056Peinkhofer, C., Knudsen, G. M., Moretti, R., & Kondziella, D. (2019). Cortical modulation of pupillary function: systematic review. PeerJ, 7, e6882. doi:10.7717/peerj.6882Grill-Spector, K., Knouf, N., & Kanwisher, N. (2004). The fusiform face area subserves face perception, not generic within-category identification. Nature Neuroscience, 7(5), 555-562. doi:10.1038/nn1224Ferretti, V., & Papaleo, F. (2018). Understanding others: emotion recognition abilities in humans and other animals. Genes, Brain and Behavior, e12544. doi:10.1111/gbb.12544Sergerie, K., Chochol, C., & Armony, J. L. (2008). The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 32(4), 811-830. doi:10.1016/j.neubiorev.2007.12.002Rapcsak, S. Z., Galper, S. R., Comer, J. F., Reminger, S. L., Nielsen, L., Kaszniak, A. W., … Cohen, R. A. (2000). Fear recognition deficits after focal brain damage: A cautionary note. Neurology, 54(3), 575-575. doi:10.1212/wnl.54.3.575Radice-Neumann, D., Zupan, B., Tomita, M., & Willer, B. (2009). Training Emotional Processing in Persons With Brain Injury. Journal of Head Trauma Rehabilitation, 24(5), 313-323. doi:10.1097/htr.0b013e3181b09160Yuvaraj, R., Murugappan, M., Norlinah, M. I., Sundaraj, K., & Khairiyah, M. (2013). Review of Emotion Recognition in Stroke Patients. Dementia and Geriatric Cognitive Disorders, 36(3-4), 179-196. doi:10.1159/000353440Babbage, D. R., Yim, J., Zupan, B., Neumann, D., Tomita, M. R., & Willer, B. (2011). Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology, 25(3), 277-285. doi:10.1037/a0021908Milders, M., Fuchs, S., & Crawford, J. R. (2003). Neuropsychological Impairments and Changes in Emotional and Social Behaviour Following Severe Traumatic Brain Injury. Journal of Clinical and Experimental Neuropsychology, 25(2), 157-172. doi:10.1076/jcen.25.2.157.13642Genova, H. M., Genualdi, A., Goverover, Y., Chiaravalloti, N. D., Marino, C., & Lengenfelder, J. (2016). An investigation of the impact of facial affect recognition impairments in moderate to severe TBI on fatigue, depression, and quality of life. Social Neuroscience, 12(3), 303-307. doi:10.1080/17470919.2016.1173584Rigon, A., Voss, M. W., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2018). Different aspects of facial affect recognition impairment following traumatic brain injury: The role of perceptual and interpretative abilities. Journal of Clinical and Experimental Neuropsychology, 40(8), 805-819. doi:10.1080/13803395.2018.1437120Rosenberg, H., McDonald, S., Dethier, M., Kessels, R. P. C., & Westbrook, R. F. (2014). Facial Emotion Recognition Deficits following Moderate–Severe Traumatic Brain Injury (TBI): Re-examining the Valence Effect and the Role of Emotion Intensity. Journal of the International Neuropsychological Society, 20(10), 994-1003. doi:10.1017/s1355617714000940Lancelot, C., & Gilles, C. (2018). How does visual context influence recognition of facial emotion in people with traumatic brain injury? Brain Injury, 33(1), 4-11. doi:10.1080/02699052.2018.1531308McDonald, S. (2013). Impairments in Social Cognition Following Severe Traumatic Brain Injury. Journal of the International Neuropsychological Society, 19(3), 231-246. doi:10.1017/s1355617712001506Vallat-Azouvi, C., Azouvi, P., Le-Bornec, G., & Brunet-Gouet, E. (2018). Treatment of social cognition impairments in patients with traumatic brain injury: a critical review. Brain Injury, 33(1), 87-93. doi:10.1080/02699052.2018.1531309Godin, B., Oishi, K., Oishi, K., Davis, C., Gomez, Y., Trupe, L., … Tippett, D. (2018). Impaired Recognition of Emotional Faces after Stroke Involving Right Amygdala or Insula. Seminars in Speech and Language, 39(01), 087-100. doi:10.1055/s-0037-1608859Abbott, J. D., Cumming, G., Fidler, F., & Lindell, A. K. (2013). The perception of positive and negative facial expressions in unilateral brain-damaged patients: A meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 18(4), 437-459. doi:10.1080/1357650x.2012.703206Abbott, J. D., Wijeratne, T., Hughes, A., Perre, D., & Lindell, A. K. (2014). The perception of positive and negative facial expressions by unilateral stroke patients. Brain and Cognition, 86, 42-54. doi:10.1016/j.bandc.2014.01.017Delazer, M., Sojer, M., Ellmerer, P., Boehme, C., & Benke, T. (2018). Eye-Tracking Provides a Sensitive Measure of Exploration Deficits After Acute Right MCA Stroke. Frontiers in Neurology, 9. doi:10.3389/fneur.2018.00359Lech, M., Kucewicz, M. T., & Czyżewski, A. (2019). Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries. Frontiers in Neurology, 10. doi:10.3389/fneur.2019.00006Spikman, J. M., Milders, M. V., Visser-Keizer, A. C., Westerhof-Evers, H. J., Herben-Dekker, M., & van der Naalt, J. (2013). Deficits in Facial Emotion Recognition Indicate Behavioral Changes and Impaired Self-Awareness after Moderate to Severe Traumatic Brain Injury. PLoS ONE, 8(6), e65581. doi:10.1371/journal.pone.0065581Knox, L., & Douglas, J. (2009). Long-term ability to interpret facial expression after traumatic brain injury and its relation to social integration. Brain and Cognition, 69(2), 442-449. doi:10.1016/j.bandc.2008.09.009Struchen, M. A., Clark, A. N., Sander, A. M., Mills, M. R., Evans, G., & Kurtz, D. (2008). Relation of executive functioning and social communication measures to functional outcomes following traumatic brain injury. NeuroRehabilitation, 23(2), 185-198. doi:10.3233/nre-2008-23208Ferro, J. M., Caeiro, L., & Santos, C. (2009). Poststroke Emotional and Behavior Impairment: A Narrative Review. Cerebrovascular Diseases, 27(1), 197-203. doi:10.1159/000200460Bortolon, C., Capdevielle, D., & Raffard, S. (2015). Face recognition in schizophrenia disorder: A comprehensive review of behavioral, neuroimaging and neurophysiological studies. Neuroscience & Biobehavioral Reviews, 53, 79-107. doi:10.1016/j.neubiorev.2015.03.006Harms, M. B., Martin, A., & Wallace, G. L. (2010). Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies. Neuropsychology Review, 20(3), 290-322. doi:10.1007/s11065-010-9138-6Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). «Mini-mental state». Journal of Psychiatric Research, 12(3), 189-198. doi:10.1016/0022-3956(75)90026-6Romero, M., Sánchez, A., Marín, C., Navarro, M. D., Ferri, J., & Noé, E. (2012). Clinical usefulness of the Spanish version of the Mississippi Aphasia Screening Test (MASTsp): validation in stroke patients. Neurología (English Edition), 27(4), 216-224. doi:10.1016/j.nrleng.2011.06.001Aguillon-Hernandez, N., Roché, L., Bonnet-Brilhault, F., Roux, S., Barthelemy, C., & Martineau, J. (2016). Eye Movement Monitoring and Maturation of Human Face Exploration. Medical Principles and Practice, 25(6), 548-554. doi:10.1159/000447971Mathôt, S., Fabius, J., Van Heusden, E., & Van der Stigchel, S. (2018). Safe and sensible preprocessing and baseline correction of pupil-size data. Behavior Research Methods, 50(1), 94-106. doi:10.3758/s13428-017-1007-2Green, C., & Guo, K. (2016). Factors contributing to individual differences in facial expression categorisation. Cognition and Emotion, 32(1), 37-48. doi:10.1080/02699931.2016.1273200Burley, D. T., Gray, N. S., & Snowden, R. J. (2017). As Far as the Eye Can See: Relationship between Psychopathic Traits and Pupil Response to Affective Stimuli. PLOS ONE, 12(1), e0167436. doi:10.1371/journal.pone.0167436Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1-2), 185-198. doi:10.1016/s1071-5819(03)00017-xGotham, K. O., Siegle, G. J., Han, G. T., Tomarken, A. J., Crist, R. N., Simon, D. M., & Bodfish, J. W. (2018). Pupil response to social-emotional material is associated with rumination and depressive symptoms in adults with autism spectrum disorder. PLOS ONE, 13(8), e0200340. doi:10.1371/journal.pone.0200340Demenescu, L. R., Mathiak, K. A., & Mathiak, K. (2014). Age- and Gender-Related Variations of Emotion Recognition in Pseudowords and Faces. Experimental Aging Research, 40(2), 187-207. doi:10.1080/0361073x.2014.882210Grainger, S. A., Henry, J. D., Phillips, L. H., Vanman, E. J., & Allen, R. (2015). Age Deficits in Facial Affect Recognition: The Influence of Dynamic Cues. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, gbv100. doi:10.1093/geronb/gbv100Wagner, H. L. (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1), 3-28. doi:10.1007/bf00987006Bradley, M. M., & Lang, P. J. (2015). Memory, emotion, and pupil diameter: Repetition of natural scenes. Psychophysiology, 52(9), 1186-1193. doi:10.1111/psyp.12442Larsen, R. S., & Waters, J. (2018). Neuromodulatory Correlates of Pupil Dilation. Frontiers in Neural Circuits, 12. doi:10.3389/fncir.2018.0002