PDF hosted at the Radboud Repository of the Radboud UniversityNijmegen The following full text is a publisher's version. For additional information about this publication click this link.http://hdl.handle.net/2066/99151 Please be advised that this information was generated on 2016-05-02 and may be subject tochange. Gaze behavior in face-to-face interaction Copyright © 2012, Federico Rossano. All rights reserved. Cover Illustration: Tilman Harpe Printed and bound by Ipskamp Drukkers b.v., Nijmegen Gaze behavior in face-to-face interaction Proefschrift ter verkrijging van de graad van doctor aan de Radboud Universiteit Nijmegen op gezag van de rector magnificus prof. mr. S.C.J.J. Kortmann, volgens besluit van het college van decanen in het openbaar te verdedigen op vrijdag 5 oktober 2012 om 10.30 uur precies door Federico Rossano geboren op 27 mei 1979 te Bologna (Italië) Copyright © 2012, Federico Rossano. All rights reserved. Cover Illustration: Tilman Harpe Printed and bound by Ipskamp Drukkers b.v., Nijmegen Gaze behavior in face-to-face interaction Proefschrift ter verkrijging van de graad van doctor aan de Radboud Universiteit Nijmegen op gezag van de rector magnificus prof. mr. S.C.J.J. Kortmann, volgens besluit van het college van decanen in het openbaar te verdedigen op vrijdag 5 oktober 2012 om 10.30 uur precies door Federico Rossano geboren op 27 mei 1979 te Bologna (Italië) Promotoren: Prof. dr. S. C. Levinson Prof. dr. E. A. Schegloff (University of California, Los Angeles, USA) Copromotor: Dr. T. Stivers (University of California, Los Angeles, USA) Manuscriptcommissie: Prof. dr. N. J. Enfield Prof. dr. L. Mondada (Universität Basel, Switzerland) Prof. dr. P. Drew (University of York, UK) The research reported in this thesis was supported by a grant from the Max-Planck-Gesellschaft zur Förderung der Wissenschaften, München, Germany. Promotoren: Prof. dr. S. C. Levinson Prof. dr. E. A. Schegloff (University of California, Los Angeles, USA) Copromotor: Dr. T. Stivers (University of California, Los Angeles, USA) Manuscriptcommissie: Prof. dr. N. J. Enfield Prof. dr. L. Mondada (Universität Basel, Switzerland) Prof. dr. P. Drew (University of York, UK) The research reported in this thesis was supported by a grant from the Max-Planck-Gesellschaft zur Förderung der Wissenschaften, München, Germany. i Acknowledgments “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness.” It was my life as a Ph.D. student. Exciting, challenging, unpredictable. Some people say that, when they were young, they already knew what they wanted to be as grown ups. As children, they experienced a moment that made clear to them what they wanted to pursue in life and everything would naturally follow from that crucial moment in their childhood. I knew it too. As a child, I wanted to compete in the Olympic games as a swimmer. I didn’t want to win the Olympic games, I just wanted to participate. In my eyes as a young child, being amongst the top swimmers in the world was like the maximum realization for a human being. What else could one want in life? It took me many years of hard training and poor performances in the pool to realize that I was not talented enough to make it to the Olympics. But the desire remained. Throughout my (still young) academic life, I have been fortunate to meet and learn from many incredible scholars, but only when Steve Levinson offered me a Ph.D. fellowship to work in his group, I thought that I had almost realized my childhood dream. Becoming a researcher at a Max Planck Institute is, to my eyes, the closest thing to participating in the Olympics that I will ever experience, for it is hard to imagine a research center with as high a concentration of young, bright international scholars with so much talent. Thank you, Steve, for thinking that I could ever deserve to be part of such an impressive research family. I have made Steve’s life as a supervisor incredibly difficult and I have resisted his advice in many ways (the “age of foolishness” of the incipit). Despite these difficulties, each time I went to his office to discuss my work-in-progress, I always left with the feeling that his comments had suddenly unlocked some secret about my data that I had been incapable of seeing until that very moment. When I am trying to see the big picture in anything I study, I ask myself, “What would Steve say about this?” If I have been difficult with Steve, words cannot begin to describe my behavior toward my supervisor Tanya Stivers. No doubt she never expected so much stubbornness from a student. She taught me how to make sense of my data, how to give a good presentation and how to write a good paper, and she helped improve my English the hard way: via countless corrections in i Acknowledgments “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness.” It was my life as a Ph.D. student. Exciting, challenging, unpredictable. Some people say that, when they were young, they already knew what they wanted to be as grown ups. As children, they experienced a moment that made clear to them what they wanted to pursue in life and everything would naturally follow from that crucial moment in their childhood. I knew it too. As a child, I wanted to compete in the Olympic games as a swimmer. I didn’t want to win the Olympic games, I just wanted to participate. In my eyes as a young child, being amongst the top swimmers in the world was like the maximum realization for a human being. What else could one want in life? It took me many years of hard training and poor performances in the pool to realize that I was not talented enough to make it to the Olympics. But the desire remained. Throughout my (still young) academic life, I have been fortunate to meet and learn from many incredible scholars, but only when Steve Levinson offered me a Ph.D. fellowship to work in his group, I thought that I had almost realized my childhood dream. Becoming a researcher at a Max Planck Institute is, to my eyes, the closest thing to participating in the Olympics that I will ever experience, for it is hard to imagine a research center with as high a concentration of young, bright international scholars with so much talent. Thank you, Steve, for thinking that I could ever deserve to be part of such an impressive research family. I have made Steve’s life as a supervisor incredibly difficult and I have resisted his advice in many ways (the “age of foolishness” of the incipit). Despite these difficulties, each time I went to his office to discuss my work-in-progress, I always left with the feeling that his comments had suddenly unlocked some secret about my data that I had been incapable of seeing until that very moment. When I am trying to see the big picture in anything I study, I ask myself, “What would Steve say about this?” If I have been difficult with Steve, words cannot begin to describe my behavior toward my supervisor Tanya Stivers. No doubt she never expected so much stubbornness from a student. She taught me how to make sense of my data, how to give a good presentation and how to write a good paper, and she helped improve my English the hard way: via countless corrections in iiiearly drafts of this manuscript. She has been the toughest supervisor I could have ever imagined, but if I am now capable of effectively participating in academic discourse, I owe it mostly to her. Speaking of tough supervisors, it is difficult to put to words just what having Manny Schegloff as a supervisor has meant to me. He embodies Conversation Analysis and it has been the biggest honor and privilege to be able to discuss ideas with him and to have him commenting on my work with his famously merciless red pencil. He taught me more than a few sentences could ever capture. Most importantly, he has been supportive of me every step of the way, even when we strongly disagreed. I have always been so impressed by his charisma, his intellectual integrity and his analytical mind that when in 2002 he asked me about my plans for the future, I confess having answered: “in the future, I want to be you”. Although I will never succeed, just like my desire to be in the Olympic games, I still hope to become Manny Schegloff one day. There are four other people who have not played a supervisory role in my studies, but who have nonetheless inspired me in unimaginable ways: Adam Kendon, Charles Goodwin, Jack Sidnell and Gail Jefferson. I have had the privilege of interacting with and learning from Adam and Chuck for many years and it is no surprise that my dissertation is on gaze in interaction. Whenever I disagreed with these Masters about the organization or function of gaze in interaction, that disagreement stemmed from a desire to take their lessons seriously: be honest with your data and never take previous claims for granted. Jack has been a friend and a model for how a young academic should be. He had the patience to engage in endless discussions with me on any topic and to bear, as book editor, with my tendency to see deadlines as suggestions, rather than obligations. I met Gail in the Netherlands in 2004 and I had the privilege of participating in monthly data sessions with her until her untimely departure. I will never forget her energy, her sharp comments and her directness. I wish I could look at data with her one more time. I miss you, Gail! All of them remain, together with my supervisors, the giants and I am simply an apprentice. The members of the Language and Cognition Group at MPI have been teachers, colleagues and friends to me, and their talent continues to amaze me. I have been fortunate in finding friends in all of the groups at the MPI and I felt supported by all of them, not just academically. Thank you Mandana, Marianne, Elizabeth, Malte, Lara, Paul, Lilla, Tilman, Kaoru, Clair, Gertie, Mark, Giovanni, Keren, Sarah, Jan Peter, Laura, Annelie. Two of them hold a very special place in my heart and I cannot thank them enough for agreeing to be my iiiparanimfen: Pamela Perniss and Sylvia Tufvesson. They are the best friends that anyone could ask for: they are smart, funny, charming, and have the most remarkable taste in literature, movies and music. Their patience in handling my disorganization and last-minute crises is honestly unbelievable. Having these two as friends are gifts to be treasured. I would also like to thank Michael Tomasello for offering me yet another ideal intellectual environment where I could complete my dissertation while at the same time engage in new and exciting intellectual pursuits. Mike and my colleagues and friends at the MPI in Leipzig, Amrisha and Tobias, Robert, Bahar, Katharina, Jonathan, Katja, Daniel, Elena and Claudia, have taught me another precious craft and they continue to show me the beauty of science and the pure joy of scientific investigation at a level that is truly exceptional. These are my second Olympic games. A few other friends deserve special mention because they have blessed me with their generosity, cheered me for the successes and comforted me in times of need. Thank you Beggia, Ubba, Michele, Federico, Emiel, Chris, Schnine, Jana, Jasmine, Christina! Thank you also to my Italian professors Fabrizio Bercelli, Paolo Leonardi and Patrizia Violi for being great teachers and mentors. Your advice has shaped many of my decisions, not just in terms of my career, but also in life. Without another good friend, Tanya Romaniuk, I never would have finished this dissertation. She is not just a wonderful person and a brilliant scholar, she is also the most patient editor and simultaneously the most caring and relentless friend. She decided one day that I was going to finish my dissertation by the end of last year and through countless skype interactions across the ocean, she succeeded. Her indefatigable editing work made this manuscript readable. Her stubbornness in believing that I could do it, made me complete it. Infine, un ringraziamento speciale va ai miei nonni, ai miei genitori e a mio fratello per la loro pazienza e la loro fiducia. Il loro supporto e’ stato indispensabile per la riuscita di questo progetto. Li ringrazio per avermi aiutato a rialzarmi ogni volta che sono caduto (non solo metaforicamente). Ora ne sono sicuro: Con loro al mio fianco niente e’ impossibile. iiiiearly drafts of this manuscript. She has been the toughest supervisor I could have ever imagined, but if I am now capable of effectively participating in academic discourse, I owe it mostly to her. Speaking of tough supervisors, it is difficult to put to words just what having Manny Schegloff as a supervisor has meant to me. He embodies Conversation Analysis and it has been the biggest honor and privilege to be able to discuss ideas with him and to have him commenting on my work with his famously merciless red pencil. He taught me more than a few sentences could ever capture. Most importantly, he has been supportive of me every step of the way, even when we strongly disagreed. I have always been so impressed by his charisma, his intellectual integrity and his analytical mind that when in 2002 he asked me about my plans for the future, I confess having answered: “in the future, I want to be you”. Although I will never succeed, just like my desire to be in the Olympic games, I still hope to become Manny Schegloff one day. There are four other people who have not played a supervisory role in my studies, but who have nonetheless inspired me in unimaginable ways: Adam Kendon, Charles Goodwin, Jack Sidnell and Gail Jefferson. I have had the privilege of interacting with and learning from Adam and Chuck for many years and it is no surprise that my dissertation is on gaze in interaction. Whenever I disagreed with these Masters about the organization or function of gaze in interaction, that disagreement stemmed from a desire to take their lessons seriously: be honest with your data and never take previous claims for granted. Jack has been a friend and a model for how a young academic should be. He had the patience to engage in endless discussions with me on any topic and to bear, as book editor, with my tendency to see deadlines as suggestions, rather than obligations. I met Gail in the Netherlands in 2004 and I had the privilege of participating in monthly data sessions with her until her untimely departure. I will never forget her energy, her sharp comments and her directness. I wish I could look at data with her one more time. I miss you, Gail! All of them remain, together with my supervisors, the giants and I am simply an apprentice. The members of the Language and Cognition Group at MPI have been teachers, colleagues and friends to me, and their talent continues to amaze me. I have been fortunate in finding friends in all of the groups at the MPI and I felt supported by all of them, not just academically. Thank you Mandana, Marianne, Elizabeth, Malte, Lara, Paul, Lilla, Tilman, Kaoru, Clair, Gertie, Mark, Giovanni, Keren, Sarah, Jan Peter, Laura, Annelie. Two of them hold a very special place in my heart and I cannot thank them enough for agreeing to be my iiiparanimfen: Pamela Perniss and Sylvia Tufvesson. They are the best friends that anyone could ask for: they are smart, funny, charming, and have the most remarkable taste in literature, movies and music. Their patience in handling my disorganization and last-minute crises is honestly unbelievable. Having these two as friends are gifts to be treasured. I would also like to thank Michael Tomasello for offering me yet another ideal intellectual environment where I could complete my dissertation while at the same time engage in new and exciting intellectual pursuits. Mike and my colleagues and friends at the MPI in Leipzig, Amrisha and Tobias, Robert, Bahar, Katharina, Jonathan, Katja, Daniel, Elena and Claudia, have taught me another precious craft and they continue to show me the beauty of science and the pure joy of scientific investigation at a level that is truly exceptional. These are my second Olympic games. A few other friends deserve special mention because they have blessed me with their generosity, cheered me for the successes and comforted me in times of need. Thank you Beggia, Ubba, Michele, Federico, Emiel, Chris, Schnine, Jana, Jasmine, Christina! Thank you also to my Italian professors Fabrizio Bercelli, Paolo Leonardi and Patrizia Violi for being great teachers and mentors. Your advice has shaped many of my decisions, not just in terms of my career, but also in life. Without another good friend, Tanya Romaniuk, I never would have finished this dissertation. She is not just a wonderful person and a brilliant scholar, she is also the most patient editor and simultaneously the most caring and relentless friend. She decided one day that I was going to finish my dissertation by the end of last year and through countless skype interactions across the ocean, she succeeded. Her indefatigable editing work made this manuscript readable. Her stubbornness in believing that I could do it, made me complete it. Infine, un ringraziamento speciale va ai miei nonni, ai miei genitori e a mio fratello per la loro pazienza e la loro fiducia. Il loro supporto e’ stato indispensabile per la riuscita di questo progetto. Li ringrazio per avermi aiutato a rialzarmi ogni volta che sono caduto (non solo metaforicamente). Ora ne sono sicuro: Con loro al mio fianco niente e’ impossibile. iiiivContents Acknowledgments……………………………………………………………………………….i List of abbreviations.…………………………………………………………………………viii 1 Introduction 1.1 The Problem……………………………………………………………………………….1 1.2 To See: An Evolutionary Perspective on Sight…………………………………………..15 1.2.1 The Evolution of the Eye…………………………………………………………...18 1.2.2 Eye Morphology in Primates and Eye Movements………………………………...21 1.2.3 Eyes and Vision…………………………………………………………………….24 1.3 To Be Seen……………………………………………………………………………….27 1.3.1 Eyes, Faces and Brain……………………………………………………………..31 1.4 Gaze and Social Interaction……………………………………………………………...33 1.4.1 Participation Roles and Participation Frameworks, Engagement and Disengagement…………………………………………………..35 1.4.2 Regulatory Functions of Gaze……………………………………………………..38 1.4.3 Gaze in Action Formation and as a Social Act…………………………………….42 1.5 Data and Method…………………………………………………………………………45 1.5.1 Transcription System………………………………………………………………49 1.6 Structure of the Dissertation……………………………………………………………..51 2 Gaze Behavior in Two Sequential Environments 2.1 Introduction………………………………………………………………………………53 2.2 Speaker and Recipient Gaze…………………………………………………………......54 2.2.1 Background………………………………………………………………………...54 2.2.2 Contrastive Evidence for Kendon’s Pattern……………………………………….57 2.2.2.1 Supportive Evidence………………………………………………………57 2.2.2.2 Counter Evidence………………………………………………………….62 v 2.2.3 Two Sequential Environments……………………………………………………...66 2.2.4 Extended Tellings and Their Beginnings…………………………………………..67 2.2.5 Further Qualitative Evidence……………………………………………………...71 2.2.6 Competing Activities and Their Relation to the Distribution of Gaze……………..78 2.3 Generalizing The Claim………………………………………………………………….85 2.3.1 How These Observations Fit (or not) with Prior Literature……………………….85 2.3.2 Conversation Analytic Literature on Extended Tellings…………………………...86 2.3.3 Looking Up during First TCU……………………………………………………..88 2.3.3.1 Beginning of Tellings vs. Beginning of Adjacency-Pair-Based Sequences…………………………...88 2.3.4 A Deviant Case…………………………………………………………………….91 2.4 On Projectability…………………………………………………………………………95 2.4.1 Projecting an ETS………………………………………………………………….95 2.5 Discussion………………………………………………………………………………121 3 Gaze as a Method of Pursuing Responses 3.1 Introduction……………………………………………………………………………..127 3.2 Speaker Looking Toward Addressee…………………………………………………...129 3.3 Gaze and Sequence Organization………………………………………………………134 3.3.1 Sequential Patterns in Conversation……………………………………………..136 3.3.2 Gaze During FPP, Response and Its Timing……………………………………..147 3.3.3 Discussion………………………………………………………………………...153 3.4 Pursuing………………………………………………………………………………...157 3.4.1 Gaze in the Transition Relevance Place………………………………………….158 3.4.2 Post-Positioned Gaze……………………………………………………………..159 3.4.3 A Continuum of Pursuits?………………………………………………………...165 3.4.4 Pursuing Responses With Gaze…………………………………………………..167 3.4.5 Gaze Pursuits That Fail or Get Very Delayed Responses………………………..171 3.4.6 Types of Pursuits………………………………………………………………….180 3.4.7 Timing of Responses and Pursuits………………………………………………..184 3.4.8 Examples of Gaze + Verbal Pursuits……………………………………………..185 ivivContents Acknowledgments……………………………………………………………………………….i List of abbreviations.…………………………………………………………………………viii 1 Introduction 1.1 The Problem……………………………………………………………………………….1 1.2 To See: An Evolutionary Perspective on Sight…………………………………………..15 1.2.1 The Evolution of the Eye…………………………………………………………...18 1.2.2 Eye Morphology in Primates and Eye Movements………………………………...21 1.2.3 Eyes and Vision…………………………………………………………………….24 1.3 To Be Seen……………………………………………………………………………….27 1.3.1 Eyes, Faces and Brain……………………………………………………………..31 1.4 Gaze and Social Interaction……………………………………………………………...33 1.4.1 Participation Roles and Participation Frameworks, Engagement and Disengagement…………………………………………………..35 1.4.2 Regulatory Functions of Gaze……………………………………………………..38 1.4.3 Gaze in Action Formation and as a Social Act…………………………………….42 1.5 Data and Method…………………………………………………………………………45 1.5.1 Transcription System………………………………………………………………49 1.6 Structure of the Dissertation……………………………………………………………..51 2 Gaze Behavior in Two Sequential Environments 2.1 Introduction………………………………………………………………………………53 2.2 Speaker and Recipient Gaze…………………………………………………………......54 2.2.1 Background………………………………………………………………………...54 2.2.2 Contrastive Evidence for Kendon’s Pattern……………………………………….57 2.2.2.1 Supportive Evidence………………………………………………………57 2.2.2.2 Counter Evidence………………………………………………………….62 v 2.2.3 Two Sequential Environments……………………………………………………...66 2.2.4 Extended Tellings and Their Beginnings…………………………………………..67 2.2.5 Further Qualitative Evidence……………………………………………………...71 2.2.6 Competing Activities and Their Relation to the Distribution of Gaze……………..78 2.3 Generalizing The Claim………………………………………………………………….85 2.3.1 How These Observations Fit (or not) with Prior Literature……………………….85 2.3.2 Conversation Analytic Literature on Extended Tellings…………………………...86 2.3.3 Looking Up during First TCU……………………………………………………..88 2.3.3.1 Beginning of Tellings vs. Beginning of Adjacency-Pair-Based Sequences…………………………...88 2.3.4 A Deviant Case…………………………………………………………………….91 2.4 On Projectability…………………………………………………………………………95 2.4.1 Projecting an ETS………………………………………………………………….95 2.5 Discussion………………………………………………………………………………121 3 Gaze as a Method of Pursuing Responses 3.1 Introduction……………………………………………………………………………..127 3.2 Speaker Looking Toward Addressee…………………………………………………...129 3.3 Gaze and Sequence Organization………………………………………………………134 3.3.1 Sequential Patterns in Conversation……………………………………………..136 3.3.2 Gaze During FPP, Response and Its Timing……………………………………..147 3.3.3 Discussion………………………………………………………………………...153 3.4 Pursuing………………………………………………………………………………...157 3.4.1 Gaze in the Transition Relevance Place………………………………………….158 3.4.2 Post-Positioned Gaze……………………………………………………………..159 3.4.3 A Continuum of Pursuits?………………………………………………………...165 3.4.4 Pursuing Responses With Gaze…………………………………………………..167 3.4.5 Gaze Pursuits That Fail or Get Very Delayed Responses………………………..171 3.4.6 Types of Pursuits………………………………………………………………….180 3.4.7 Timing of Responses and Pursuits………………………………………………..184 3.4.8 Examples of Gaze + Verbal Pursuits……………………………………………..185 vvi 3.4.9 Multiple Pursuits………………………………………………………………….191 3.4.9.1 Two Pursuits: 1st Pursuit Verbal Only 2nd Pursuit Gaze Only…………191 3.4.9.2 Two Pursuits: 1st Pursuit Only Gaze 2nd Pursuit Both Modalities……..193 3.4.9.3 Two Pursuits: 1st Pursuit Both Modalities 2nd Pursuit Both Modalities.195 3.4.9.4 Double Pursuit 1st Pursuit Only Gaze 2nd Pursuit Only Verbal………..197 3.4.10 Patterns of Pursuits……………………………………………………………..199 3.5 Schemes of Pursuits…………………………………………………………………….202 3.6 Gaze Pursuits in Different Sequential Environments…………………………………..206 3.6.1 During a Telling………………………………………………………………….207 3.6.2 After a Laughable Item…………………………………………………………...209 3.6.3 Pursuing Uptake Mid-TCU………………………………………………………212 3.6.3.1 After a Preliminary Component………………………………………….212 3.6.3.2 After a Recognizable Reference………………………………………….215 3.7 Why Gaze Now?………………………………………………………………………..220 3.8 Summary………………………………………………………………………………..223 4 Gaze Withdrawal in Sequence Closing 4.1 Introduction……………………………………………………………………………..227 4.2 Background……………………………………………………………………………..230 4.3 Transition Relevance Places……………………………………………………………237 4.4 Possible Sequence Completion…………………………………………………………237 4.4.1 Configuration 1: Gaze Down at the Possible Completion of a Sequence………..241 4.4.2 Configuration 2: Gaze Up By Both Participants at the Possible Completion of a Sequence………………………………………………252 4.4.3 Configurations 3 & 4: Gaze Up By One Participant at the Possible Completion of a Sequence………………………………………………258 4.5 Gaze Behavior’s Relationship To Courses of Action…………………………………..267 4.5.1 Completion of Sequence vs. Completion of Course of Action……………………268 4.5.2 Is the Course of Action Still Accomplishable?……………………………………280 4.5.3 Looking Away Before Starting a New Course of Action………………………….285 4.6 A Systematic Practice…………………………………………………………………..288 vii4.7 “Deviant Cases”………………………………………………………………………...302 4.7.1 Gaze Up No Sequence Expansion……………………………………………..302 4.8 Discussion………………………………………………………………………………308 5 Conclusions 5.1 The Findings……………………………………………………………………………313 5.2 Methodological Contribution…………………………………………………………...316 5.3 Implications for a Model of Gaze Behavior in Social Interaction……………………...325 Appendix A: Information about the Data…………………………………………………...333 Appendix B: Transcription Conventions…………………………………………………….337 References……………………………………………………………………………………...343 Samenvatting…………………………………………………………………………………..369 Curriculum Vitae……………………………………………………………………………...375 MPI Series in Psycholinguistics……………………………………………………………....377 vivi 3.4.9 Multiple Pursuits………………………………………………………………….191 3.4.9.1 Two Pursuits: 1st Pursuit Verbal Only 2nd Pursuit Gaze Only…………191 3.4.9.2 Two Pursuits: 1st Pursuit Only Gaze 2nd Pursuit Both Modalities……..193 3.4.9.3 Two Pursuits: 1st Pursuit Both Modalities 2nd Pursuit Both Modalities.195 3.4.9.4 Double Pursuit 1st Pursuit Only Gaze 2nd Pursuit Only Verbal………..197 3.4.10 Patterns of Pursuits……………………………………………………………..199 3.5 Schemes of Pursuits…………………………………………………………………….202 3.6 Gaze Pursuits in Different Sequential Environments…………………………………..206 3.6.1 During a Telling………………………………………………………………….207 3.6.2 After a Laughable Item…………………………………………………………...209 3.6.3 Pursuing Uptake Mid-TCU………………………………………………………212 3.6.3.1 After a Preliminary Component………………………………………….212 3.6.3.2 After a Recognizable Reference………………………………………….215 3.7 Why Gaze Now?………………………………………………………………………..220 3.8 Summary………………………………………………………………………………..223 4 Gaze Withdrawal in Sequence Closing 4.1 Introduction……………………………………………………………………………..227 4.2 Background……………………………………………………………………………..230 4.3 Transition Relevance Places……………………………………………………………237 4.4 Possible Sequence Completion…………………………………………………………237 4.4.1 Configuration 1: Gaze Down at the Possible Completion of a Sequence………..241 4.4.2 Configuration 2: Gaze Up By Both Participants at the Possible Completion of a Sequence………………………………………………252 4.4.3 Configurations 3 & 4: Gaze Up By One Participant at the Possible Completion of a Sequence………………………………………………258 4.5 Gaze Behavior’s Relationship To Courses of Action…………………………………..267 4.5.1 Completion of Sequence vs. Completion of Course of Action……………………268 4.5.2 Is the Course of Action Still Accomplishable?……………………………………280 4.5.3 Looking Away Before Starting a New Course of Action………………………….285 4.6 A Systematic Practice…………………………………………………………………..288 vii4.7 “Deviant Cases”………………………………………………………………………...302 4.7.1 Gaze Up No Sequence Expansion……………………………………………..302 4.8 Discussion………………………………………………………………………………308 5 Conclusions 5.1 The Findings……………………………………………………………………………313 5.2 Methodological Contribution…………………………………………………………...316 5.3 Implications for a Model of Gaze Behavior in Social Interaction……………………...325 Appendix A: Information about the Data…………………………………………………...333 Appendix B: Transcription Conventions…………………………………………………….337 References……………………………………………………………………………………...343 Samenvatting…………………………………………………………………………………..369 Curriculum Vitae……………………………………………………………………………...375 MPI Series in Psycholinguistics……………………………………………………………....377 viiviiiList of Abbreviations APBS: Adjacency-Pair-Based Sequence BS: Base Sequence CA: Conversation Analysis ETS: Extended-Telling Sequence FPP: First Pair Part Post Ex. BS: Post-Expansion of the Base Sequence SCT: Sequence Closing Third SPP: Second Pair Part STS: Superior Temporal Sulcus TCU: Turn Constructional Unit TRP: Transition Relevance Place In transcripts: Cl.: Clitic Throw.2s: Verb Throw 2nd person singular viii 11 Introduction One might have thought of sight, but who could think of what it sees? Wallace Stevens Esthetique du Mal (1944) 1.1 The Problem What does it mean to be able to see and to be seen by others? What do we do with our eyes when we talk to one other? This dissertation investigates how people use their eyes and their body during face-to-face interactions and outlines the orderliness of their behavior. In particular, it shows how individuals from a specific culture (Italians from Emilia-Romagna, a region in Northern Italy) deploy their gaze during dyadic naturally occurring interactions in their homes. The aim is to outline some of the practices and norms that those individuals are sensitive to, with the goal of providing a detailed description of the orderliness of gaze behavior in human interaction and refining our understanding of participation in social situations. Investigating gaze in face-to-face interaction helps us better understand how participants manage rights and obligations in interaction, specifically with respect to cooperation, agency and the accountability of social behavior. It also highlights the fine-grained calibration and organization of turns-at-talk and sequences of social actions. A more specific goal of the analysis is to demonstrate that gaze behavior may differ when all other aspects of the interactional scene but the sequential context remain the same. The analysis focuses on how participants use their gaze at the beginning, middle and end of sequences of talk. The findings are situated in a comparative and evolutionary context, contributing to a more general framework for the understanding of how human beings use their eyes in interaction. Additionally, they provide important information for computer scientists and experts in artificial intelligence, in terms of the predictability of gaze behavior in humans and its potential reproducibility in robots or avatars. First, it is useful to take a closer look at the phenomena that will be discussed here. Example 1.1 is an extract from a conversation between two young women (a university student, A, and a recent graduate, B, both in their early twenties). This conversation occurs in the kitchen of the student’s house. The two participants are good friends, have known each 12other for several years and are sitting at a table, at a 90-degree angle from each other. When the conversation begins they both have an empty glass and a can of soft drink in front of them. For the moment, the focus should be on how they use their eyes while they are talking to each other, considering what kinds of social action they are performing during their conversation. The conventions used in the transcript and in particular the meaning of the oval symbols are explained in Appendix B and in § 1.5.1, but basically, the arrows show whether the interlocutors are looking at each other, up, down, or at specific objects in the surrounding environment.(1.1) 2GGOSS-stasera 00:151 01 B: .hh A(h)l l o r a stas(h)era cos’ e’ che fate So tonight what is that do.2s .hh S(h)o t(h)onight what are you doing 02 A: Eh andiamo a Villa Chiara= Eh go.1p to Villa Chiara Eh we go to Villa Chiara= 03 B: =Ma a che ora vi incontrate But at which hour you meet.2p =But at what time do you meet 04 A: Vado alle nove e mezza dalla Gloria. Go.1s at nine and half to Gloria I go to Gloria’s (house) at nine thirty. 05 ((B annuisce)) ((B nods)) 1 See Appendix A for more information concerning this and all the other interactions analyzed in this dissertation. 306 ((some turns with side sequence omitted)) 07 B: [Nove e mezza ma] andate subito a Villa Nine and half but go.2p immediately to Villa [Nine thirty but] do you go immediately to Villa 08 Chiara alle nove e mezza?= Chiara at nine and half Chiara at nine thirty?= 09 A: =Con- le dieci con gli altri. With the ten with the others =With- at ten with the others. 10 (0.4) 11 A: Ci incontriamo. Cl. meet.1p We meet. 12 (0.6) 13 A: Ci [v i e n i? Cl. come.2s Will [you come? ((to Villa Chiara)) 14 B: [Io esco alle nove_ (0.5) ah io pero’ se arrivo I get.1s out at nine oh I but if arrive.1s [I get out (of work) at nine_ (0.5) oh but if I come 22other for several years and are sitting at a table, at a 90-degree angle from each other. When the conversation begins they both have an empty glass and a can of soft drink in front of them. For the moment, the focus should be on how they use their eyes while they are talking to each other, considering what kinds of social action they are performing during their conversation. The conventions used in the transcript and in particular the meaning of the oval symbols are explained in Appendix B and in § 1.5.1, but basically, the arrows show whether the interlocutors are looking at each other, up, down, or at specific objects in the surrounding environment.(1.1) 2GGOSS-stasera 00:151 01 B: .hh A(h)l l o r a stas(h)era cos’ e’ che fate So tonight what is that do.2s .hh S(h)o t(h)onight what are you doing 02 A: Eh andiamo a Villa Chiara= Eh go.1p to Villa Chiara Eh we go to Villa Chiara= 03 B: =Ma a che ora vi incontrate But at which hour you meet.2p =But at what time do you meet 04 A: Vado alle nove e mezza dalla Gloria. Go.1s at nine and half to Gloria I go to Gloria’s (house) at nine thirty. 05 ((B annuisce)) ((B nods)) 1 See Appendix A for more information concerning this and all the other interactions analyzed in this dissertation. 306 ((some turns with side sequence omitted)) 07 B: [Nove e mezza ma] andate subito a Villa Nine and half but go.2p immediately to Villa [Nine thirty but] do you go immediately to Villa 08 Chiara alle nove e mezza?= Chiara at nine and half Chiara at nine thirty?= 09 A: =Con- le dieci con gli altri. With the ten with the others =With- at ten with the others. 10 (0.4) 11 A: Ci incontriamo. Cl. meet.1p We meet. 12 (0.6) 13 A: Ci [v i e n i? Cl. come.2s Will [you come? ((to Villa Chiara)) 14 B: [Io esco alle nove_ (0.5) ah io pero’ se arrivo I get.1s out at nine oh I but if arrive.1s [I get out (of work) at nine_ (0.5) oh but if I come 34 15 arrivo a mezzanotte eh arrive.1s at midnight eh I arrive at midnight eh 16 (0.6) ((A displays shocked facial expression see Fig. 1.1)) Figure 1.1. A’s facial expression during line 16. 17 A: Alle nove esci? At nine get.2s out At nine you get out (of work)? 18 B: E[h Eh Y[es ((confirming something already said)) 19 A: [Cazzo ((Poi A annuisce, guardando in basso)) Dick [Shit ((and then A nods, looking down)) Figure 1.2 Figure 1.3 5 20 (0.6) (Fig. 1.2) (1.2) (Fig. 1.3) (1.8) 21 A: Domani sera che fai? Tomorrow evening what do.2s Tomorrow evening what do you do? 22 (0.5) The first things to note concerning this interaction are the setting and the spatial configurations that make this social situation possible. This interaction occurs in the kitchen room of a house, with the two participants sitting at a table with soft drinks available (see Figures 1.2 and 1.3 in the transcript). It is known that the place where a conversation takes place, the seating arrangements, the possible occurrence of drinking and eating, the socio-economic status of the participants and the kind of relationship that exists between them are all factors that might affect their conversation. Many studies have shown, for example, that participants tend to arrange themselves according to the interactions they plan to have (Ekman & Friesen, 1974; Kendon, 1977; Sommer, 1959, 1962). While it is true that the physical structure of the environment constrains interactional positioning (Goffman, 1971; Goodwin, 1981), these physical arrangements tend to directly reflect preferred cultural patterns. It is clear, indeed, that a number of these aspects might be typical of specific cultures such as Western ones (e.g., the fact that when two friends want to have an extended conversation they tend to sit at a table and have food or drinks available). Yet, what might be culturally specific about the environment and the modalities through which this meeting is arranged should not hide a more generic fact, which is that when participants want to have a conversation, they arrange their bodies close to each other with a specific function, beyond hearing better what the other might be saying. Usually, they position themselves in “an eye-to-eye ecological huddle, which tends to be carefully maintained, maximizing the opportunity to monitor one another’s mutual perceivings” (Goffman, 1963: 95), so that each participant can see the other’s face, but also what the other might be looking at and doing. All these aspects will be further discussed later. 44 15 arrivo a mezzanotte eh arrive.1s at midnight eh I arrive at midnight eh 16 (0.6) ((A displays shocked facial expression see Fig. 1.1)) Figure 1.1. A’s facial expression during line 16. 17 A: Alle nove esci? At nine get.2s out At nine you get out (of work)? 18 B: E[h Eh Y[es ((confirming something already said)) 19 A: [Cazzo ((Poi A annuisce, guardando in basso)) Dick [Shit ((and then A nods, looking down)) Figure 1.2 Figure 1.3 5 20 (0.6) (Fig. 1.2) (1.2) (Fig. 1.3) (1.8) 21 A: Domani sera che fai? Tomorrow evening what do.2s Tomorrow evening what do you do? 22 (0.5) The first things to note concerning this interaction are the setting and the spatial configurations that make this social situation possible. This interaction occurs in the kitchen room of a house, with the two participants sitting at a table with soft drinks available (see Figures 1.2 and 1.3 in the transcript). It is known that the place where a conversation takes place, the seating arrangements, the possible occurrence of drinking and eating, the socio-economic status of the participants and the kind of relationship that exists between them are all factors that might affect their conversation. Many studies have shown, for example, that participants tend to arrange themselves according to the interactions they plan to have (Ekman & Friesen, 1974; Kendon, 1977; Sommer, 1959, 1962). While it is true that the physical structure of the environment constrains interactional positioning (Goffman, 1971; Goodwin, 1981), these physical arrangements tend to directly reflect preferred cultural patterns. It is clear, indeed, that a number of these aspects might be typical of specific cultures such as Western ones (e.g., the fact that when two friends want to have an extended conversation they tend to sit at a table and have food or drinks available). Yet, what might be culturally specific about the environment and the modalities through which this meeting is arranged should not hide a more generic fact, which is that when participants want to have a conversation, they arrange their bodies close to each other with a specific function, beyond hearing better what the other might be saying. Usually, they position themselves in “an eye-to-eye ecological huddle, which tends to be carefully maintained, maximizing the opportunity to monitor one another’s mutual perceivings” (Goffman, 1963: 95), so that each participant can see the other’s face, but also what the other might be looking at and doing. All these aspects will be further discussed later. 5 6 A second thing to note in Example 1.1 is that A and B do not follow simple gaze patterns. It is not the case that every time A or B starts speaking they look down or away nor that they tend to look toward the addressee only when they approach the end of their turns of talk (see Kendon, 1967 for a claim that might suggest the opposite). Instead, they direct their eyes in many directions. Mostly they look at each other, sometimes they look away, but when they look away, it is often to look at objects that will be picked up later in the interaction. Sometimes they look up toward one side, displaying a thinking face (M. H. Goodwin & Goodwin, 1986). It also often happens that one looks at the other while the other is not looking back, yet this seems to be often a transitory phase, frequently followed by engagement of mutual gaze. In other words, gaze directions can be tracked and possibly explained, but the patterns they follow are not obvious. On the other hand, one thing that remains remarkable is how “ordinary” (Sacks, 1984b) their gaze behavior appears to be for the other participant and for anybody observing them. The two participants here are focused on the content of the conversation and the social actions performed in it (in this specific case, the main project is figuring out whether the two participants can go out together that evening or not), while the way each one uses her eyes is probably less central to their concerns. They certainly do not mention it at any point during the 45 minutes of conversation that follows. And this is what we would expect, because presenting our gaze behavior as ordinary is indeed what we wish to achieve during our daily interactions: we do not want participants to find our gaze behavior marked or deviant. There might be occasions in which this is not the case (e.g., when we are trying to flirt with someone or threaten someone), however, most of the time we attempt to avoid a “non-ordinary” use of our eyes. Like gestures, gaze is difficult to describe and yet appears to have a prominent role in our daily interactions. As Sapir (1963 [1927]: 556) wrote with respect to gestures, which we can easily extend to gaze: Gestures are hard to classify and it is difficult to make a conscious separation between that in gesture which is of merely individual origin and that which is referable to the habits of the group as a whole. In spite of these difficulties of conscious analysis, we respond to gestures with an extreme alertness and, one might almost say, in accordance with an elaborate and secret code that is written nowhere, known by none, and understood by all. 7Cracking the gaze code, however, is not an easy task. The task of this dissertation is to make explicit how we come to produce “ordinary”, not marked gaze behavior in conversation. This means that by reaching the end of this work, it should be easier to detect and explain the regularities present in example 1.1. The findings of the next 3 chapters will provide a detailed account of what is going on in terms of gaze behavior and courses of action in example 1.1 (this example will be presented again as 3.3 in chapter 3 and 4.13 in chapter 4, to show how the different gaze practices under examination get implemented in the production of this extract). In perfect agreement with what Birdwhistell (1970: xi) claimed 40 years ago, this dissertation posits that “body motion is a learned form of communication, which is patterned within a culture and which can be broken down into an ordered system of isolable elements.” Yet, as it will be explained later, any body motion has to deal with the physiological limits of our bodies, which have evolved in a specific way for clear adaptive purposes and are common to all human beings (e.g., we have approximately 200° and not 360° visual range). Keeping in mind that our gaze behavior is partly affected by our biology and partly “a learned form of communication patterned within a culture”, it is important to specify some of the specific assumptions—grounded in previous empirical observations—that this dissertation is based on. The first assumption is that human beings did not evolve eyes to be able to communicate better. We have eyes to perceive the world around us, facilitate our movements and the detection of prey and more generally to coordinate body movements within the environment. As Gould and Lewontin (1979) explained with the metaphor of “the spandrels of San Marco”, one could think of the decorations in the spandrels (see Figure 1.4)—the approximately triangular spaces between two arches and a horizontal molding or cornice above them—as the main goal of the architect and the arches that create the spandrels as a consequence of that initial artistic goal. Alternatively, and more reasonably, one could assume that the architect wanted to create some arches, particularly functional and aesthetically pleasing in the architecture of the building. But once the arches had been built, the spandrels were there and allowed for further decorative opportunities. Similarly, humans, as primates, evolved eyes mainly for perceptual purposes. But once we had eyes, evolution facilitated the possibility of also deploying them for other purposes, such as communicative ones. The fact, for example, that our sclera (the white of our eyes) is not pigmented—being 6 6 A second thing to note in Example 1.1 is that A and B do not follow simple gaze patterns. It is not the case that every time A or B starts speaking they look down or away nor that they tend to look toward the addressee only when they approach the end of their turns of talk (see Kendon, 1967 for a claim that might suggest the opposite). Instead, they direct their eyes in many directions. Mostly they look at each other, sometimes they look away, but when they look away, it is often to look at objects that will be picked up later in the interaction. Sometimes they look up toward one side, displaying a thinking face (M. H. Goodwin & Goodwin, 1986). It also often happens that one looks at the other while the other is not looking back, yet this seems to be often a transitory phase, frequently followed by engagement of mutual gaze. In other words, gaze directions can be tracked and possibly explained, but the patterns they follow are not obvious. On the other hand, one thing that remains remarkable is how “ordinary” (Sacks, 1984b) their gaze behavior appears to be for the other participant and for anybody observing them. The two participants here are focused on the content of the conversation and the social actions performed in it (in this specific case, the main project is figuring out whether the two participants can go out together that evening or not), while the way each one uses her eyes is probably less central to their concerns. They certainly do not mention it at any point during the 45 minutes of conversation that follows. And this is what we would expect, because presenting our gaze behavior as ordinary is indeed what we wish to achieve during our daily interactions: we do not want participants to find our gaze behavior marked or deviant. There might be occasions in which this is not the case (e.g., when we are trying to flirt with someone or threaten someone), however, most of the time we attempt to avoid a “non-ordinary” use of our eyes. Like gestures, gaze is difficult to describe and yet appears to have a prominent role in our daily interactions. As Sapir (1963 [1927]: 556) wrote with respect to gestures, which we can easily extend to gaze: Gestures are hard to classify and it is difficult to make a conscious separation between that in gesture which is of merely individual origin and that which is referable to the habits of the group as a whole. In spite of these difficulties of conscious analysis, we respond to gestures with an extreme alertness and, one might almost say, in accordance with an elaborate and secret code that is written nowhere, known by none, and understood by all. 7Cracking the gaze code, however, is not an easy task. The task of this dissertation is to make explicit how we come to produce “ordinary”, not marked gaze behavior in conversation. This means that by reaching the end of this work, it should be easier to detect and explain the regularities present in example 1.1. The findings of the next 3 chapters will provide a detailed account of what is going on in terms of gaze behavior and courses of action in example 1.1 (this example will be presented again as 3.3 in chapter 3 and 4.13 in chapter 4, to show how the different gaze practices under examination get implemented in the production of this extract). In perfect agreement with what Birdwhistell (1970: xi) claimed 40 years ago, this dissertation posits that “body motion is a learned form of communication, which is patterned within a culture and which can be broken down into an ordered system of isolable elements.” Yet, as it will be explained later, any body motion has to deal with the physiological limits of our bodies, which have evolved in a specific way for clear adaptive purposes and are common to all human beings (e.g., we have approximately 200° and not 360° visual range). Keeping in mind that our gaze behavior is partly affected by our biology and partly “a learned form of communication patterned within a culture”, it is important to specify some of the specific assumptions—grounded in previous empirical observations—that this dissertation is based on. The first assumption is that human beings did not evolve eyes to be able to communicate better. We have eyes to perceive the world around us, facilitate our movements and the detection of prey and more generally to coordinate body movements within the environment. As Gould and Lewontin (1979) explained with the metaphor of “the spandrels of San Marco”, one could think of the decorations in the spandrels (see Figure 1.4)—the approximately triangular spaces between two arches and a horizontal molding or cornice above them—as the main goal of the architect and the arches that create the spandrels as a consequence of that initial artistic goal. Alternatively, and more reasonably, one could assume that the architect wanted to create some arches, particularly functional and aesthetically pleasing in the architecture of the building. But once the arches had been built, the spandrels were there and allowed for further decorative opportunities. Similarly, humans, as primates, evolved eyes mainly for perceptual purposes. But once we had eyes, evolution facilitated the possibility of also deploying them for other purposes, such as communicative ones. The fact, for example, that our sclera (the white of our eyes) is not pigmented—being 7 8 white rather than dark, as in every other primate species—appears to be an adaptation to facilitate the detection of our eye movements by conspecifics (see § 1.2.2 for more details). Figure 1.4. Schematic representation of a three-dimensional and a two-dimensional spandrel.2 The second assumption is that looking at somebody’s face is usually more interactionally relevant than looking anywhere else during a conversation (at least in Western cultures). Studies based mainly on American English and European languages (e.g. Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Exline, 1963; Goodwin, 1981; Gullberg & Holmqvist, 2006; Kendon, 1967; Nielsen, 1962) show that participants spend a significant amount of time looking at each other’s faces when they interact. Argyle and Graham (1976: 6) emphasized that “in our thinking about the processes underlying social interaction we have come to believe that gaze at other persons is of central importance”. However, in dyadic interactions in laboratory settings they also showed that while background stimuli had an unreliable effect on gaze, objects relevant for the subject’s task attracted gaze for large amounts of time and therefore reduced gaze at the other person. Moreover, Goodwin (e.g., 1981, 1984) has claimed that looking at objects in the nearby environment used for accomplishing activities competing with the talk (e.g., drinking, smoking, eating) is less problematic than looking away in general, as long as it remains a brief disengagement from looking at the other participant. Nonetheless, the claim that, in general, looking at each other’s face during a conversation should be more relevant than looking at any other element in the environment not directly relevant to the task-at-hand has been challenged cross-culturally. For example, the work of Rossano et al. (2009) shows how members of a Mayan community living in Mexico and speaking Tzeltal tend not to look at 2 Image from Gould (1997: 10751). 9speakers while listening to them, so that not looking at anything in particular while listening to somebody speaking is the default home position for the eyes, and looking at some object in particular might be considered a sign of disattending the conversation. The third assumption is that participants’ gaze behavior in interaction is related to the fact that the participants are talking to each other. In particular, the claim is that looking at each other is part of and contributes to the development of interactants’ conversation. There is no doubt that individuals monitor the environment around them regularly; however, when two people are co-present mutual gaze creates an opportunity for them to initiate conversation. In general, individuals tend to avoid looking at each other during silences unless the silence constitutes just a pause or a gap (Sacks, Schegloff, & Jefferson, 1974: 715) in the conversation, rather than a full disengagement. This assumption of a correlation between talk and gaze behavior is the basis of Kendon (1967) and Goodwin’s (e.g., 1979, 1980, 1981) work on the organization of gaze in interaction. Finally, the fourth underlying assumption of this work, again shared by Kendon and Goodwin and previously emphasized at the beginning of this introduction, is that gazing at someone is also a social act and not merely a perceptual tool to better understand speech. Nonetheless, if we do not understand how our eyes can also be used as a perceptual organ, we are bound to lose a powerful tool to understand what might be going on during a social interaction. The general claim of this dissertation, then, is that gaze (in Italian conversations, at least) can be understood in terms of action formation and a sequential analysis of actions in interaction (see, e.g., Schegloff, 2007b), and not simply in terms of turn-taking or displays of engagement and attentiveness as has been proposed (e.g., Goodwin, 1981). This work will show that gaze is used, for example, to coordinate the development and closure of sequences and courses of action, to pressure for responses and pursue them and to indicate special states of recipiency. Moreover, it will show that each individual deploys specific gaze behaviors according to her/his role as speaker or recipient but also in relation to what s/he is trying to achieve during a conversation. Mutual gaze, in this sense, will be interpreted as an interactional achievement that results from the behavior of each participant performing specific gaze patterns, rather than as a goal toward which participants are normatively driven. So speaker gaze toward the recipient might have very different motivations than recipient gaze toward speaker. And yet if both perform such behaviors, they will end up looking at each other’s eyes and thus engaging in mutual gaze. 8 8 white rather than dark, as in every other primate species—appears to be an adaptation to facilitate the detection of our eye movements by conspecifics (see § 1.2.2 for more details). Figure 1.4. Schematic representation of a three-dimensional and a two-dimensional spandrel.2 The second assumption is that looking at somebody’s face is usually more interactionally relevant than looking anywhere else during a conversation (at least in Western cultures). Studies based mainly on American English and European languages (e.g. Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Exline, 1963; Goodwin, 1981; Gullberg & Holmqvist, 2006; Kendon, 1967; Nielsen, 1962) show that participants spend a significant amount of time looking at each other’s faces when they interact. Argyle and Graham (1976: 6) emphasized that “in our thinking about the processes underlying social interaction we have come to believe that gaze at other persons is of central importance”. However, in dyadic interactions in laboratory settings they also showed that while background stimuli had an unreliable effect on gaze, objects relevant for the subject’s task attracted gaze for large amounts of time and therefore reduced gaze at the other person. Moreover, Goodwin (e.g., 1981, 1984) has claimed that looking at objects in the nearby environment used for accomplishing activities competing with the talk (e.g., drinking, smoking, eating) is less problematic than looking away in general, as long as it remains a brief disengagement from looking at the other participant. Nonetheless, the claim that, in general, looking at each other’s face during a conversation should be more relevant than looking at any other element in the environment not directly relevant to the task-at-hand has been challenged cross-culturally. For example, the work of Rossano et al. (2009) shows how members of a Mayan community living in Mexico and speaking Tzeltal tend not to look at 2 Image from Gould (1997: 10751). 9speakers while listening to them, so that not looking at anything in particular while listening to somebody speaking is the default home position for the eyes, and looking at some object in particular might be considered a sign of disattending the conversation. The third assumption is that participants’ gaze behavior in interaction is related to the fact that the participants are talking to each other. In particular, the claim is that looking at each other is part of and contributes to the development of interactants’ conversation. There is no doubt that individuals monitor the environment around them regularly; however, when two people are co-present mutual gaze creates an opportunity for them to initiate conversation. In general, individuals tend to avoid looking at each other during silences unless the silence constitutes just a pause or a gap (Sacks, Schegloff, & Jefferson, 1974: 715) in the conversation, rather than a full disengagement. This assumption of a correlation between talk and gaze behavior is the basis of Kendon (1967) and Goodwin’s (e.g., 1979, 1980, 1981) work on the organization of gaze in interaction. Finally, the fourth underlying assumption of this work, again shared by Kendon and Goodwin and previously emphasized at the beginning of this introduction, is that gazing at someone is also a social act and not merely a perceptual tool to better understand speech. Nonetheless, if we do not understand how our eyes can also be used as a perceptual organ, we are bound to lose a powerful tool to understand what might be going on during a social interaction. The general claim of this dissertation, then, is that gaze (in Italian conversations, at least) can be understood in terms of action formation and a sequential analysis of actions in interaction (see, e.g., Schegloff, 2007b), and not simply in terms of turn-taking or displays of engagement and attentiveness as has been proposed (e.g., Goodwin, 1981). This work will show that gaze is used, for example, to coordinate the development and closure of sequences and courses of action, to pressure for responses and pursue them and to indicate special states of recipiency. Moreover, it will show that each individual deploys specific gaze behaviors according to her/his role as speaker or recipient but also in relation to what s/he is trying to achieve during a conversation. Mutual gaze, in this sense, will be interpreted as an interactional achievement that results from the behavior of each participant performing specific gaze patterns, rather than as a goal toward which participants are normatively driven. So speaker gaze toward the recipient might have very different motivations than recipient gaze toward speaker. And yet if both perform such behaviors, they will end up looking at each other’s eyes and thus engaging in mutual gaze. 9 10 The news here is not simply that to understand gaze in interaction one needs to take into account what the social situation is and what people are trying to do, in terms of physical and social actions, at a micro level. The news consists in suggesting that there are multiple levels of order that play out at the same time and that can be comprehended through a careful and detailed analysis of people’s use of their eyes during conversations. Gaze, indeed, is used to perceive the world, to control the accuracy of our body movements and other people’s movements in the environment, to display attentiveness and engagement, but also for structural communicative purposes as well. Too often two different functions of gaze behavior are emphasized in scientific investigations: gaze behavior as responsive to environmental demands (e.g., attention) vs. gaze behavior as shaping the interactional environment and the actions accomplishable (e.g., projecting next action). To simplify the dichotomy, one could say that gaze is usually either studied as “a measure of” something or “as a means for” something. Yet it is clear that in a single stretch of interaction, the same individual will be confronted with the need to pay attention to the environment and still use her/his eyes for her/his communicative needs. A more fruitful analysis of gaze behavior in social interaction, then, should be aware of the importance of each component, informed of their limits and biases and somehow make sense of both functions by extracting some basic rules concerning gaze deployment in social interaction. Therefore, the logic followed in this work requires trying to unravel the gaze machinery, the system of norms, biases and habits that guide people’s eyes and actions during a conversation. Once the system is known, we may be able to focus on what the specific participants in a specific interaction are trying to do. As Sacks (1987: 67) said: You cannot find what [people] are trying to do until you find the kind of things they work with. If the system had a built-in bias for agreement, and you did not know about it, then you’d be counting a whole bunch of things as agreements that might well be accounted for in other ways. […] Now, what they wanted to do is another question, and it is only, I think when we begin to have some considerable idea about the sort of things they are operating with (like a preference for agreement), and the sorts of ways they operate with those things, that we have much of an idea about such matters as ‘what they’re trying to do’. In other words, an investigation cannot start from trying to interpret what people are trying to do with their eyes, but rather in documenting the patterns and regularities of their behavior, 11including whether and to what extent their behavior is associated with what we know participants are doing through their talk and bodies. As Lorenz (1966: 273) suggests, while proposing the use of a biological approach to the study of behavior: Biological approach and method can successfully be applied in the study of behaviour, exactly as Darwin had done in his book, The expression of the emotions in man and animals. What, then, are these good old Darwinian procedures? I would name the most obvious one first, as being most important. It is the unbiased observation of the organic system and the inventorizing of its component parts. In all natural sciences description has to precede systematization, and both together are the prerequisite for abstracting the natural laws prevailing in the operations of the whole. Observation and description always have to precede systematization. As will be shown in every chapter, any initial description of a possible practice will then be followed by a systematic search for more instances of the practice and an attempt to provide falsifiable hypotheses about its functions, to be tested via quantification. Finally, and not in every instance, a question can be raised concerning how participants can detect the current unfolding of a specific practice or how that practice can come to be; that is, how that practice might evolve. An example of how the latter point could be approached, and of the importance of taking into account more than one level of organization, comes, for example, from ethology, in the words of Lorenz (1966: 276): A phylogenetically adapted motor pattern which originally served the species in dealing with some environmental necessities, acquires a new function, that of communication. The primary function may persist […]. In many cases, however, the primary function recedes into the background or disappears altogether so that a complete change of function is achieved. Out of communication, two new, equally important functions may arise […] the first of these functions is the canalization of aggression in a manner permitting its discharge without damaging fellow members of the species […] the second is the formation of a bond which keeps together two or more individuals. This is achieved by most so-called greeting ceremonies which an animal can perform only with a certain, individually known partner, whose presence, for this reason, becomes an indispensable need in the animal’s life. It is quite erroneous to say that such ceremonies are “the expression of” a bond; indeed they themselves constitute it. 10 10 The news here is not simply that to understand gaze in interaction one needs to take into account what the social situation is and what people are trying to do, in terms of physical and social actions, at a micro level. The news consists in suggesting that there are multiple levels of order that play out at the same time and that can be comprehended through a careful and detailed analysis of people’s use of their eyes during conversations. Gaze, indeed, is used to perceive the world, to control the accuracy of our body movements and other people’s movements in the environment, to display attentiveness and engagement, but also for structural communicative purposes as well. Too often two different functions of gaze behavior are emphasized in scientific investigations: gaze behavior as responsive to environmental demands (e.g., attention) vs. gaze behavior as shaping the interactional environment and the actions accomplishable (e.g., projecting next action). To simplify the dichotomy, one could say that gaze is usually either studied as “a measure of” something or “as a means for” something. Yet it is clear that in a single stretch of interaction, the same individual will be confronted with the need to pay attention to the environment and still use her/his eyes for her/his communicative needs. A more fruitful analysis of gaze behavior in social interaction, then, should be aware of the importance of each component, informed of their limits and biases and somehow make sense of both functions by extracting some basic rules concerning gaze deployment in social interaction. Therefore, the logic followed in this work requires trying to unravel the gaze machinery, the system of norms, biases and habits that guide people’s eyes and actions during a conversation. Once the system is known, we may be able to focus on what the specific participants in a specific interaction are trying to do. As Sacks (1987: 67) said: You cannot find what [people] are trying to do until you find the kind of things they work with. If the system had a built-in bias for agreement, and you did not know about it, then you’d be counting a whole bunch of things as agreements that might well be accounted for in other ways. […] Now, what they wanted to do is another question, and it is only, I think when we begin to have some considerable idea about the sort of things they are operating with (like a preference for agreement), and the sorts of ways they operate with those things, that we have much of an idea about such matters as ‘what they’re trying to do’. In other words, an investigation cannot start from trying to interpret what people are trying to do with their eyes, but rather in documenting the patterns and regularities of their behavior, 11including whether and to what extent their behavior is associated with what we know participants are doing through their talk and bodies. As Lorenz (1966: 273) suggests, while proposing the use of a biological approach to the study of behavior: Biological approach and method can successfully be applied in the study of behaviour, exactly as Darwin had done in his book, The expression of the emotions in man and animals. What, then, are these good old Darwinian procedures? I would name the most obvious one first, as being most important. It is the unbiased observation of the organic system and the inventorizing of its component parts. In all natural sciences description has to precede systematization, and both together are the prerequisite for abstracting the natural laws prevailing in the operations of the whole. Observation and description always have to precede systematization. As will be shown in every chapter, any initial description of a possible practice will then be followed by a systematic search for more instances of the practice and an attempt to provide falsifiable hypotheses about its functions, to be tested via quantification. Finally, and not in every instance, a question can be raised concerning how participants can detect the current unfolding of a specific practice or how that practice can come to be; that is, how that practice might evolve. An example of how the latter point could be approached, and of the importance of taking into account more than one level of organization, comes, for example, from ethology, in the words of Lorenz (1966: 276): A phylogenetically adapted motor pattern which originally served the species in dealing with some environmental necessities, acquires a new function, that of communication. The primary function may persist […]. In many cases, however, the primary function recedes into the background or disappears altogether so that a complete change of function is achieved. Out of communication, two new, equally important functions may arise […] the first of these functions is the canalization of aggression in a manner permitting its discharge without damaging fellow members of the species […] the second is the formation of a bond which keeps together two or more individuals. This is achieved by most so-called greeting ceremonies which an animal can perform only with a certain, individually known partner, whose presence, for this reason, becomes an indispensable need in the animal’s life. It is quite erroneous to say that such ceremonies are “the expression of” a bond; indeed they themselves constitute it. 11 12 The second characteristic of ritualized motor patterns is a change of form which the unritualized prototype underwent in the service of its new communicative function and which quite obviously was brought about by the selection pressure exerted by the survival value of communication. This quote suggests that there are multiple elements to take into account to understand the communicative system we use, how it came about and how it works. From an evolutionary perspective, the emergence of a communicative pattern presumably requires some level of ritualization of a behavior that originally had some specific function other than communication becomes a communicative signal by getting reinforced and modified in its meaning via repeated interactions with conspecifics. Lorenz suggests that among the new functions of a communicative act, the canalization of aggression and the development of a bond may occur through the occurrence of this ritualized act. Additionally, the act is not (just) “an expression of” the relationship between individuals, but rather it “constitutes it”. We do not act in a certain way simply because we are friends, for example, but rather, by acting in a certain way we reinforce our friendship and display to one another the kind of relationship we think we have or want to have. That is, we are “expressing” and “constituting” friendship in every act we perform toward a potential or current friend. The two additional functions mentioned above are particularly relevant for a work on gaze in interaction because looking at someone in the eyes is a highly arousing act, e.g., skin conductance increases when gaze is directed toward the observer (Hietanen, Leppanen, Peltola, Linna-Aho, & Ruuhiala, 2008; Nichols & Champness, 1971) and people often look at each other’s eyes to threaten each other (aggression) and to flirt (create intimacy) (Eibl-Eibesfeldt, 1989). Many authors have claimed a relationship between gaze and emotional arousal in humans, in particular claiming that gaze aversion is a way to reduce emotional arousal, both in adults (e.g., Ellsworth & Ludwig, 1972; Kendon, 1967) and in young infants (e.g., Brazelton, Koslowski, & Main, 1974; Field, 1979; Stern, 1974), and showing, for example, that in 4 month olds, gaze aversion is usually preceded by a heart rate increase and their heart rate decreases while the gaze is averted from the adult’s face (Field, 1981). Yet, aside from using our eyes for threatening or creating intimacy, we also tend to look at each other when we talk, and this is not just to better parse the sounds we produce, but also to display attentiveness or commitment, to pressure for response, to select next speaker, to monitor each other and so on and so forth. Mutual gaze, then, may indeed be one of the motor patterns originally deployed with a specific function that then obtained new functions, by 13canalizing aggression, helping in establishing a bond and more generally allowing human beings to do other things with it, making their interactions more effective. It was suggested earlier that the study of gaze in interaction needs to take into account multiple levels of organization, multiple systems that affect the final deployment of people’s eyes, and ought to develop a more fine grained model of when and how people move their eyes, with the hope of understanding why as well. Considering some of the current theories, as far as levels of organization needed to understand human behavior in social interaction is concerned, three different positions stand out. One insists that there is mainly one core level of organization, the interactional level that shapes every other level (see, e.g., Schegloff, 2006). A second one suggests the existence of three different systems that interact at the same time, the socio-cultural system, the interactional system and the language system (see, e.g., Levinson, 2005). Finally, a third one also focuses on three levels of organization, cognition, interaction and culture, which are usually investigated by three different disciplines, but are necessarily intertwined, and, as such, require a more complex kind of investigation (see, e.g., Enfield & Levinson, 2006b; Sperber, 2006). In line with the above ethological concerns, I propose an approach that requires a basic understanding of four different levels of organization: 1. Individual 2. Micro-social (dialogical/conversational) 3. Macro-social/Cultural 4. Species To understand what happens in any single interaction, one should take into account the culture a participants belongs to, his/her socio-economic status and relationships, the rules of conversation or of politeness, idiosyncratic behaviors, cognitive abilities and competences of the single individual, as well as her or his own desires and beliefs. That said, one should also realize that we are biological creatures, that we have bodies with limits and specific capacities that necessarily affect what we can and cannot do when faced with a problem or a specific situation. For example, we are endowed with a specific visual system that allows us to perceive the world, act in it and move through it. We are endowed with two eyes positioned frontally, rather than laterally. The limits of our bodies are the limits of our natural repertoire of possible movements. All of these biological developments came about through evolution, in adaptation to the contingencies of a constantly changing environment. If we 12 12 The second characteristic of ritualized motor patterns is a change of form which the unritualized prototype underwent in the service of its new communicative function and which quite obviously was brought about by the selection pressure exerted by the survival value of communication. This quote suggests that there are multiple elements to take into account to understand the communicative system we use, how it came about and how it works. From an evolutionary perspective, the emergence of a communicative pattern presumably requires some level of ritualization of a behavior that originally had some specific function other than communication becomes a communicative signal by getting reinforced and modified in its meaning via repeated interactions with conspecifics. Lorenz suggests that among the new functions of a communicative act, the canalization of aggression and the development of a bond may occur through the occurrence of this ritualized act. Additionally, the act is not (just) “an expression of” the relationship between individuals, but rather it “constitutes it”. We do not act in a certain way simply because we are friends, for example, but rather, by acting in a certain way we reinforce our friendship and display to one another the kind of relationship we think we have or want to have. That is, we are “expressing” and “constituting” friendship in every act we perform toward a potential or current friend. The two additional functions mentioned above are particularly relevant for a work on gaze in interaction because looking at someone in the eyes is a highly arousing act, e.g., skin conductance increases when gaze is directed toward the observer (Hietanen, Leppanen, Peltola, Linna-Aho, & Ruuhiala, 2008; Nichols & Champness, 1971) and people often look at each other’s eyes to threaten each other (aggression) and to flirt (create intimacy) (Eibl-Eibesfeldt, 1989). Many authors have claimed a relationship between gaze and emotional arousal in humans, in particular claiming that gaze aversion is a way to reduce emotional arousal, both in adults (e.g., Ellsworth & Ludwig, 1972; Kendon, 1967) and in young infants (e.g., Brazelton, Koslowski, & Main, 1974; Field, 1979; Stern, 1974), and showing, for example, that in 4 month olds, gaze aversion is usually preceded by a heart rate increase and their heart rate decreases while the gaze is averted from the adult’s face (Field, 1981). Yet, aside from using our eyes for threatening or creating intimacy, we also tend to look at each other when we talk, and this is not just to better parse the sounds we produce, but also to display attentiveness or commitment, to pressure for response, to select next speaker, to monitor each other and so on and so forth. Mutual gaze, then, may indeed be one of the motor patterns originally deployed with a specific function that then obtained new functions, by 13canalizing aggression, helping in establishing a bond and more generally allowing human beings to do other things with it, making their interactions more effective. It was suggested earlier that the study of gaze in interaction needs to take into account multiple levels of organization, multiple systems that affect the final deployment of people’s eyes, and ought to develop a more fine grained model of when and how people move their eyes, with the hope of understanding why as well. Considering some of the current theories, as far as levels of organization needed to understand human behavior in social interaction is concerned, three different positions stand out. One insists that there is mainly one core level of organization, the interactional level that shapes every other level (see, e.g., Schegloff, 2006). A second one suggests the existence of three different systems that interact at the same time, the socio-cultural system, the interactional system and the language system (see, e.g., Levinson, 2005). Finally, a third one also focuses on three levels of organization, cognition, interaction and culture, which are usually investigated by three different disciplines, but are necessarily intertwined, and, as such, require a more complex kind of investigation (see, e.g., Enfield & Levinson, 2006b; Sperber, 2006). In line with the above ethological concerns, I propose an approach that requires a basic understanding of four different levels of organization: 1. Individual 2. Micro-social (dialogical/conversational) 3. Macro-social/Cultural 4. Species To understand what happens in any single interaction, one should take into account the culture a participants belongs to, his/her socio-economic status and relationships, the rules of conversation or of politeness, idiosyncratic behaviors, cognitive abilities and competences of the single individual, as well as her or his own desires and beliefs. That said, one should also realize that we are biological creatures, that we have bodies with limits and specific capacities that necessarily affect what we can and cannot do when faced with a problem or a specific situation. For example, we are endowed with a specific visual system that allows us to perceive the world, act in it and move through it. We are endowed with two eyes positioned frontally, rather than laterally. The limits of our bodies are the limits of our natural repertoire of possible movements. All of these biological developments came about through evolution, in adaptation to the contingencies of a constantly changing environment. If we 13 14 neglect these aspects of human biology and physiology, we may provide accurate descriptions of what we can see people doing, without necessarily knowing why it is so, how it came about and most importantly, what the other possibilities may be, whether other movements or other actions could easily emerge. In addition, we might interpret acts that occur only because of the limits of our bodies as intentionally communicative. For example, having the capacity to only see 15-30° of the visual range with proper visual acuity, out of a 200° range (see, e.g., Bruce, Green, & Georgeson, 2003), means that we need to move our eyes often if we want to ensure we accurately perceive what is happening in front of us, or even just for reading this written page. But these eye movements obviously do not occur in order to intentionally communicate to other participants which part of this page we are reading. Rather, it is a necessary requirement for our eyes to be able to perform this task. It can be informative for a recipient, but it is not designed for the recipient, as will be briefly outlined in what follows. What is proposed here, though, is not an embodied theory of social interaction. What I am suggesting is an approach that recognizes that the machinery, the repertoire that has emerged and has been reinforced through individual ritualization and social imitation —making a specific behavior a cultural praxis through historical sedimentation — has a biological foundation that continues to exert pressure on what the limits of those movements are and what the limits of our interpretation of those movements should be. Moreover, if we find that some practice is shared by all cultures and all human communities, it may be because this practice might have some strong adaptive function and be evolutionarily the best adaptation for our species, and not just for our specific community. To summarize, if we want to be able to better understand what humans do when they interact with each other, we need to keep in mind the following four points: 151. We have individual cognition and individual memories of our previous interactions. 2. We use the interactional machinery we believe we share with other interactants and we recipient-design our communications and actions, because of what we know about others. 3. Our actions are informed by what we deem to be socially and culturally appropriate. 4. We are biological organisms with specific limits and capacities, and our bodies have evolved for specific adaptive purposes. To achieve an analysis that keeps these four points in mind, a broad literature has to be introduced and discussed.3 This will constitute the background against which the empirical chapters will be built. It will also provide a glimpse of the theoretical underpinnings of this investigation. To understand how a plane can fly, it is not enough to look at it, and yet without looking at it, we would not be able to explain how that specific plane can fly while another might not. In the same way, in order to understand how we use our eyes when we interact with each other, aside from describing the orientation of the gaze of the participants, it is useful to know something about the hardware (i.e., our eyes), about vision and about human sociality to make sense of how and when we direct our eyes during a conversation. To begin, then, a first question is: what difference does it make that human beings have eyes? 1.2 To See: An Evolutionary Perspective on Sight In 1859 while outlining his evolutionary theory about the origin of species by means of natural selection, Darwin raised a distinct and serious issue for his own theory: the existence of the eye. In a section specifically dedicated to the eye called, “organs of extreme perfection and complication,” he wrote: To suppose that the eye, with all its inimitable contrivances for adjusting the focus to different distances, for admitting different amounts of light, and for the correction of spherical and chromatic aberration, could have been formed by natural selection, seems, I freely confess, absurd in the highest possible degree. […] If it could be 3 I follow here the model outlined in the volume Roots of Human Sociality (Enfield & Levinson, 2006a) where contributors and contributions from a range of disciplines were combined to address a single, broad question: what are the roots of human sociality? 14 14 neglect these aspects of human biology and physiology, we may provide accurate descriptions of what we can see people doing, without necessarily knowing why it is so, how it came about and most importantly, what the other possibilities may be, whether other movements or other actions could easily emerge. In addition, we might interpret acts that occur only because of the limits of our bodies as intentionally communicative. For example, having the capacity to only see 15-30° of the visual range with proper visual acuity, out of a 200° range (see, e.g., Bruce, Green, & Georgeson, 2003), means that we need to move our eyes often if we want to ensure we accurately perceive what is happening in front of us, or even just for reading this written page. But these eye movements obviously do not occur in order to intentionally communicate to other participants which part of this page we are reading. Rather, it is a necessary requirement for our eyes to be able to perform this task. It can be informative for a recipient, but it is not designed for the recipient, as will be briefly outlined in what follows. What is proposed here, though, is not an embodied theory of social interaction. What I am suggesting is an approach that recognizes that the machinery, the repertoire that has emerged and has been reinforced through individual ritualization and social imitation —making a specific behavior a cultural praxis through historical sedimentation — has a biological foundation that continues to exert pressure on what the limits of those movements are and what the limits of our interpretation of those movements should be. Moreover, if we find that some practice is shared by all cultures and all human communities, it may be because this practice might have some strong adaptive function and be evolutionarily the best adaptation for our species, and not just for our specific community. To summarize, if we want to be able to better understand what humans do when they interact with each other, we need to keep in mind the following four points: 151. We have individual cognition and individual memories of our previous interactions. 2. We use the interactional machinery we believe we share with other interactants and we recipient-design our communications and actions, because of what we know about others. 3. Our actions are informed by what we deem to be socially and culturally appropriate. 4. We are biological organisms with specific limits and capacities, and our bodies have evolved for specific adaptive purposes. To achieve an analysis that keeps these four points in mind, a broad literature has to be introduced and discussed.3 This will constitute the background against which the empirical chapters will be built. It will also provide a glimpse of the theoretical underpinnings of this investigation. To understand how a plane can fly, it is not enough to look at it, and yet without looking at it, we would not be able to explain how that specific plane can fly while another might not. In the same way, in order to understand how we use our eyes when we interact with each other, aside from describing the orientation of the gaze of the participants, it is useful to know something about the hardware (i.e., our eyes), about vision and about human sociality to make sense of how and when we direct our eyes during a conversation. To begin, then, a first question is: what difference does it make that human beings have eyes? 1.2 To See: An Evolutionary Perspective on Sight In 1859 while outlining his evolutionary theory about the origin of species by means of natural selection, Darwin raised a distinct and serious issue for his own theory: the existence of the eye. In a section specifically dedicated to the eye called, “organs of extreme perfection and complication,” he wrote: To suppose that the eye, with all its inimitable contrivances for adjusting the focus to different distances, for admitting different amounts of light, and for the correction of spherical and chromatic aberration, could have been formed by natural selection, seems, I freely confess, absurd in the highest possible degree. […] If it could be 3 I follow here the model outlined in the volume Roots of Human Sociality (Enfield & Levinson, 2006a) where contributors and contributions from a range of disciplines were combined to address a single, broad question: what are the roots of human sociality? 15 16 demonstrated that any complex organ existed, which could not possibly have been formed by numerous, successive, slight modifications, my theory would absolutely break down. (1859: 186-189) He also suggested, however, what kind of evidence would support his theory, even if it was difficult to envision rationally at the time: Yet reason tells me, that if numerous gradations from a perfect and complex eye to one very imperfect and simple, each grade being useful to its possessor, can be shown to exist; if further, the eye does vary ever so slightly, and the variations be inherited, which is certainly the case; and if any variation or modification in the organ be ever useful to an animal under changing conditions of life, then the difficulty of believing that a perfect and complex eye could be formed by natural selection, though insuperable by our imagination, can hardly be considered real. (1859: 186) What Darwin was looking for was a range of organs less sophisticated than the human eye that could still be used for vision and of any use for the animal possessing it, maintaining that the modifications could be inherited. Recent evidence seems to have solved this Darwinian puzzle. The first eyes that appeared on Earth belonged to the trilobite, a voracious predator already known at the time of Darwin’s writing. It can be dated to about 543 million years ago. This is not to say that this is the first mechanism ever developed to perceive light, as there were already single-celled organisms like amoebae that could detect sunlight and were using this information to distinguish up from down in the water. Moreover, there were also multi-celled animals that had light sensitive organs, called ocelli, which were similarly used to detect light direction. The difference, however, is that those organs were not capable of forming images; they could only detect light direction (like plants). Vision is certainly the most sophisticated mechanism to detect light direction, but in addition to it, it allows us to detect images through the retina. The retina is a multilayered tissue made of nerve cells that convert light signals into neural signals and through the optic nerve communicate directly with the brain. With the help of some apparatus such as a lens, it is possible to detect with some precision any image that is projected onto the retina. This allows the perception of shapes and therefore helps in the detection of other animals moving around the environment. 17It has been claimed (Parker, 2003) that the development of an organ such as the eye changed the face of the planet by allowing some animals to perceive others much better and allowing them to move better and faster through the environment. In this sense, it becomes easier for the predator to hunt and the prey must develop other means to defend itself. Noticeably, apart from the presence of outer defensive hard parts such as shells and skeletons, animals also developed another outstanding resource: color.4 Indeed, once light and images can be perceived with some acuity, color perception becomes possible for animals, and nature evolved in accordance with this new capacity. This is of great interest if we think about the amazing variety of colors present in nature and the fact that developing and changing colors is costly for any organism. What would be the point of, for example, the mimetic behavior of the chameleon or the colorful plumage of so many birds or even the amazing variety of colors of so many fruits and flowers, if these colors could not be perceived? The existence of all of the above shows how a great deal of the external appearance of animals and plants on our planet has developed as a result of the driving force of the sense of sight and the presence of sunlight. Sight might help, but how advantageous is it? Only 6 phyla of animals have eyes out of the extant 33 phyla, so it might look like there was not much need for their evolution. On the other hand, about one-third of those 33 phyla have specialized organs for detecting light (Fernald, 2006; Land & Nilsson, 2002) and the 6 phyla that have eyes constitute about 96% of the existing animal species on Earth (Land and Nilsson, 2002). In other words, they have been by far the most successful ones in terms of natural selection and differentiation. 4 Notice that color is a perceptual byproduct of light constructed by the brain and not a property of objects. It is the wavelength that the eye can perceive once light is reflected by a specific material surface (that partly absorbs and partly reflects the different wavelengths that compose light). So a blue object appears as such because it absorbs all of the wavelengths of light apart from the ones that our brain categorizes as blue. Different animals can perceive very different colors depending on the color receptors they have. So many mammals have monochromatic or dichromatic vision, whereas most primates, including humans, have trichromatic vision, and many other animals (e.g. many birds, see Bowmaker, 1991) can also perceive wavelengths in the ultraviolet spectrum, which human beings cannot perceive at all. 16 16 demonstrated that any complex organ existed, which could not possibly have been formed by numerous, successive, slight modifications, my theory would absolutely break down. (1859: 186-189) He also suggested, however, what kind of evidence would support his theory, even if it was difficult to envision rationally at the time: Yet reason tells me, that if numerous gradations from a perfect and complex eye to one very imperfect and simple, each grade being useful to its possessor, can be shown to exist; if further, the eye does vary ever so slightly, and the variations be inherited, which is certainly the case; and if any variation or modification in the organ be ever useful to an animal under changing conditions of life, then the difficulty of believing that a perfect and complex eye could be formed by natural selection, though insuperable by our imagination, can hardly be considered real. (1859: 186) What Darwin was looking for was a range of organs less sophisticated than the human eye that could still be used for vision and of any use for the animal possessing it, maintaining that the modifications could be inherited. Recent evidence seems to have solved this Darwinian puzzle. The first eyes that appeared on Earth belonged to the trilobite, a voracious predator already known at the time of Darwin’s writing. It can be dated to about 543 million years ago. This is not to say that this is the first mechanism ever developed to perceive light, as there were already single-celled organisms like amoebae that could detect sunlight and were using this information to distinguish up from down in the water. Moreover, there were also multi-celled animals that had light sensitive organs, called ocelli, which were similarly used to detect light direction. The difference, however, is that those organs were not capable of forming images; they could only detect light direction (like plants). Vision is certainly the most sophisticated mechanism to detect light direction, but in addition to it, it allows us to detect images through the retina. The retina is a multilayered tissue made of nerve cells that convert light signals into neural signals and through the optic nerve communicate directly with the brain. With the help of some apparatus such as a lens, it is possible to detect with some precision any image that is projected onto the retina. This allows the perception of shapes and therefore helps in the detection of other animals moving around the environment. 17It has been claimed (Parker, 2003) that the development of an organ such as the eye changed the face of the planet by allowing some animals to perceive others much better and allowing them to move better and faster through the environment. In this sense, it becomes easier for the predator to hunt and the prey must develop other means to defend itself. Noticeably, apart from the presence of outer defensive hard parts such as shells and skeletons, animals also developed another outstanding resource: color.4 Indeed, once light and images can be perceived with some acuity, color perception becomes possible for animals, and nature evolved in accordance with this new capacity. This is of great interest if we think about the amazing variety of colors present in nature and the fact that developing and changing colors is costly for any organism. What would be the point of, for example, the mimetic behavior of the chameleon or the colorful plumage of so many birds or even the amazing variety of colors of so many fruits and flowers, if these colors could not be perceived? The existence of all of the above shows how a great deal of the external appearance of animals and plants on our planet has developed as a result of the driving force of the sense of sight and the presence of sunlight. Sight might help, but how advantageous is it? Only 6 phyla of animals have eyes out of the extant 33 phyla, so it might look like there was not much need for their evolution. On the other hand, about one-third of those 33 phyla have specialized organs for detecting light (Fernald, 2006; Land & Nilsson, 2002) and the 6 phyla that have eyes constitute about 96% of the existing animal species on Earth (Land and Nilsson, 2002). In other words, they have been by far the most successful ones in terms of natural selection and differentiation. 4 Notice that color is a perceptual byproduct of light constructed by the brain and not a property of objects. It is the wavelength that the eye can perceive once light is reflected by a specific material surface (that partly absorbs and partly reflects the different wavelengths that compose light). So a blue object appears as such because it absorbs all of the wavelengths of light apart from the ones that our brain categorizes as blue. Different animals can perceive very different colors depending on the color receptors they have. So many mammals have monochromatic or dichromatic vision, whereas most primates, including humans, have trichromatic vision, and many other animals (e.g. many birds, see Bowmaker, 1991) can also perceive wavelengths in the ultraviolet spectrum, which human beings cannot perceive at all. 17 18 1.2.1 The Evolution of the Eye How did the eye evolve? As mentioned previously, the capacity of detecting light direction is not sufficient to consider an organ an eye. There must be some photoreceptors that transform light into some sort of neural signal that allows the brain to perceive the image projected on the retina usually through the help of a lens. There are, however, functional eyes that lack a lens (nautilus) but allow the animal to form an image on the retina (and therefore “see”), by using the principle deployed by the ancient Chinese and described by Leonardo da Vinci as the “camera obscura”.5 The nautilus is indeed provided with pinhole eyes. Moreover, there are eyes lacking a pupil or lacking the cornea, but every eye must have, by definition, “specialized phototransducing cells” (Fernald, 2000). All vertebrates (e.g. mammals) have a single type of eye called “camera eye”6 (because it works a bit like a camera), and human eyes consist of lenses, iris, cornea and pupils. Nilsson and Pelger (1994) made a “pessimistic estimate” of how long it would take for a light sensitive spot to evolve all the way to a fully developed lens eye by focusing on the development of spatial resolution (i.e. visual acuity). They modeled a sequence of evolutionary stages of the eye that can be found among animals existing today and they found that the whole evolution would require approximately half a million years. This is particularly important for Parker’s model, as outlined above, as it confirms that the development of the organ “eye” could indeed have happened rather quickly, at the beginning of the Cambrian and since then have tremendously affected animal evolution. Out of the Cambrian emerged eight different types of functional eyes (Fernald, 2000; Land & Fernald, 1992; Land & Nilsson, 2002); that is, eight optically distinct ways of forming an image on the retina which results in seeing something. This variety has been even larger in terms of different types of eye organizations, according to others (e.g., Salvini-Plawen & Mayr, 1977). 5 Light penetrates a dark environment through a tiny hole and forms an image on the dark surface opposite to the hole. 6 Interestingly also octopuses have camera eyes but they developed independently from the vertebrate ones and are actually functionally better because they do not have a blind spot which is present in the vertebrate eye. This is often used as an example of “convergent evolution” because the ancestors they had in common did not have a camera eye. 19The development of different mechanisms for processing light is particularly relevant for this investigation. Indeed, what vision does is extract information about the environment by using light and two of its properties, in particular: intensity and different wavelengths (frequency). Different animals have developed different solutions to the problem of processing light information and the solutions emerged through natural selection and therefore adaptation to environmental conditions. What matters here is the awareness that behaviors and practices are not the sole product of social negotiations or historical sedimentation, but rather the adaptive point of equilibrium for many needs and constraints in terms of coping with the world the individuals inhabit. This relationship between problems, evolution and development of specific behaviors can be easily understood by focusing for a moment on the fact that human beings have both eyes on the same (frontal) side of our head. What difference does it make that we have both eyes at the front of our head rather than on the sides, like rabbits? According to some scholars (e.g., Cartmill, 1974; Parker, 2003), the position of the eyes on the head tells us something about the position of the animal in the food chain. Specifically, Parker (2003) claims that animals with eyes on the sides of the head are mainly prey, while animals with frontal eyes are mainly predators. Clearly, the generality of this claim is problematic because not all predators have eyes that point in the same direction (see Cartmill, 1992 on this), and therefore other factors must come into play. However, some of the reasons given for this distinction warrant consideration. For example, having lateral eyes means having a wider angle of vision, which allows for improved detection of movements, although the perception of depth and distance becomes more difficult. In this sense, seeing more and being more sensitive to movement might allow prey to react faster, and the depth of vision becomes less crucial for survival. On the other hand, having eyes on the front of our head, in particular having the angles of vision crossing each other, enables stereopsis (i.e. the perception of binocular depth). This is particularly important in terms of determining distance, for example, in pinpointing where an object or prey might be. According to some (e.g., Le Gros Clark, 1959; Smith, 1924), primates, together with other arboreal animals, developed binocular stereoscopic vision because it helped them in determining distances between branches of the trees. However, other explanations for the frontal position of the eyes exist. Noticing that the closeness of the eyes in the middle of the face reduces parallax and therefore the distance at which stereoscopic vision can work (i.e., one can perceive depth and distances well only at close range), Cartmill (1974) suggested that the convergence of the eyes must have evolved to allow better vision at close range, just like cats and other predators. The development of good sight, the loss of 18 18 1.2.1 The Evolution of the Eye How did the eye evolve? As mentioned previously, the capacity of detecting light direction is not sufficient to consider an organ an eye. There must be some photoreceptors that transform light into some sort of neural signal that allows the brain to perceive the image projected on the retina usually through the help of a lens. There are, however, functional eyes that lack a lens (nautilus) but allow the animal to form an image on the retina (and therefore “see”), by using the principle deployed by the ancient Chinese and described by Leonardo da Vinci as the “camera obscura”.5 The nautilus is indeed provided with pinhole eyes. Moreover, there are eyes lacking a pupil or lacking the cornea, but every eye must have, by definition, “specialized phototransducing cells” (Fernald, 2000). All vertebrates (e.g. mammals) have a single type of eye called “camera eye”6 (because it works a bit like a camera), and human eyes consist of lenses, iris, cornea and pupils. Nilsson and Pelger (1994) made a “pessimistic estimate” of how long it would take for a light sensitive spot to evolve all the way to a fully developed lens eye by focusing on the development of spatial resolution (i.e. visual acuity). They modeled a sequence of evolutionary stages of the eye that can be found among animals existing today and they found that the whole evolution would require approximately half a million years. This is particularly important for Parker’s model, as outlined above, as it confirms that the development of the organ “eye” could indeed have happened rather quickly, at the beginning of the Cambrian and since then have tremendously affected animal evolution. Out of the Cambrian emerged eight different types of functional eyes (Fernald, 2000; Land & Fernald, 1992; Land & Nilsson, 2002); that is, eight optically distinct ways of forming an image on the retina which results in seeing something. This variety has been even larger in terms of different types of eye organizations, according to others (e.g., Salvini-Plawen & Mayr, 1977). 5 Light penetrates a dark environment through a tiny hole and forms an image on the dark surface opposite to the hole. 6 Interestingly also octopuses have camera eyes but they developed independently from the vertebrate ones and are actually functionally better because they do not have a blind spot which is present in the vertebrate eye. This is often used as an example of “convergent evolution” because the ancestors they had in common did not have a camera eye. 19The development of different mechanisms for processing light is particularly relevant for this investigation. Indeed, what vision does is extract information about the environment by using light and two of its properties, in particular: intensity and different wavelengths (frequency). Different animals have developed different solutions to the problem of processing light information and the solutions emerged through natural selection and therefore adaptation to environmental conditions. What matters here is the awareness that behaviors and practices are not the sole product of social negotiations or historical sedimentation, but rather the adaptive point of equilibrium for many needs and constraints in terms of coping with the world the individuals inhabit. This relationship between problems, evolution and development of specific behaviors can be easily understood by focusing for a moment on the fact that human beings have both eyes on the same (frontal) side of our head. What difference does it make that we have both eyes at the front of our head rather than on the sides, like rabbits? According to some scholars (e.g., Cartmill, 1974; Parker, 2003), the position of the eyes on the head tells us something about the position of the animal in the food chain. Specifically, Parker (2003) claims that animals with eyes on the sides of the head are mainly prey, while animals with frontal eyes are mainly predators. Clearly, the generality of this claim is problematic because not all predators have eyes that point in the same direction (see Cartmill, 1992 on this), and therefore other factors must come into play. However, some of the reasons given for this distinction warrant consideration. For example, having lateral eyes means having a wider angle of vision, which allows for improved detection of movements, although the perception of depth and distance becomes more difficult. In this sense, seeing more and being more sensitive to movement might allow prey to react faster, and the depth of vision becomes less crucial for survival. On the other hand, having eyes on the front of our head, in particular having the angles of vision crossing each other, enables stereopsis (i.e. the perception of binocular depth). This is particularly important in terms of determining distance, for example, in pinpointing where an object or prey might be. According to some (e.g., Le Gros Clark, 1959; Smith, 1924), primates, together with other arboreal animals, developed binocular stereoscopic vision because it helped them in determining distances between branches of the trees. However, other explanations for the frontal position of the eyes exist. Noticing that the closeness of the eyes in the middle of the face reduces parallax and therefore the distance at which stereoscopic vision can work (i.e., one can perceive depth and distances well only at close range), Cartmill (1974) suggested that the convergence of the eyes must have evolved to allow better vision at close range, just like cats and other predators. The development of good sight, the loss of 19 20 claws and the development of grasping extremities would point toward insect eating and predation with the important aid of vision. Allman (1977) confirmed that the frontal position of eyes with a spherical lens favors visual acuity even more than stereopsis (animals with eyes on the side have more blurred vision), and this is particularly important at night, in low light. In other words, the predators that favored frontal eyes are mainly nocturnal predators, because for diurnal ones the lateral position of the eyes is less of a problem. Alternatively, Sussman (1991) suggests that what would have favored certain evolutionary adaptation in primates is fruit foraging (better perception of edges, shapes and distance would help seeing them). Moreover, contrary to most other mammals, primates have trichromatic vision and this appears to have developed in relation to fruit foraging because it helped recognizing mature fruits through leaves and vegetation (see, e.g., Jacobs, 1995; Regan et al., 2001). If binocular vision and stereopsis contribute to 3-dimensional vision and allow a better perception of shapes and distances, then it is no surprise to find out that human beings look at objects anticipatorily, before reaching for them. They do so in order to calibrate their movements (see, e.g., Hayhoe & Ballard, 2005; Johansson, Westling, Backstrom, & Flanagan, 2001; Land, 2006), and also to guide their movements while approaching an object to grab it (see, e.g., Jeannerod, 1986 on how finger posturing anticipates the real grasp, occurs during the transportation of the hand and partly relies on visual feedback). Looking at objects helps establishing distances and therefore facilitates the motor control of our hands and arms when we reach for things. Whenever we are involved in a task such as preparing a cup of tea (Land, Mennie, & Rusted, 1999) or a sandwich (Hayhoe, 2000), our eyes mainly perform four monitoring functions: i) locating objects used later in the process; ii) directing the hand or object in the hand to a new location; iii) guiding the approach of one object to another; and, iv) checking the state of some variable relevant for the task. Usually the eyes reach the object that we reach for slightly less than a second before manipulating it, and usually our eyes only visit objects that are relevant for the task-at-hand, independently of how conspicuous other objects in the environment might be. The eyes, then, help us to plan our movements and provide feedback on our motor performance; feedback that becomes less salient the more routinized the action becomes. 21All of the above implies that when we observe individuals in interaction looking at their glass or a cookie before they reach for it, we are simply observing a behavior that has evolved because of the specificity of primate eyes and the advantages and limits of their frontal position. We will see how this phenomenon plays a role in everyday interactions in the empirical chapters that follow. 1.2.2 Eye Morphology in Primates and Eye Movements Human gaze behavior is a highly evolved system: unique amongst primate species, the human orbit has evolved to display the sclera, in humans also referred to as “the white of the eye”, and our eye outline is extraordinarily elongated in the horizontal direction. Human beings are the only primate species with a sclera without pigmentation (see Figure 1.5). The function of these physiological differences seems to be to make gaze direction discernable to others at a distance, as we have lost the “gaze-camouflaging” function common to other primate eyes and have enhanced the “gaze signaling” function (Kobayashi, 1997, 2001; Kobayashi & Hashiya, 2011; Morris, 1985). Some empirical evidence supporting this claim comes from an experiment run by Tomasello et al. (2007) in which human infants and other great apes were compared in terms of their tendency to rely on eyes or head direction in following the gaze of a human experimenter. They found that while human infants rely only on eyes, great apes rely mainly on head direction. This suggests that the presence of a white sclera co-evolved with human cooperative interaction, making it easier for others to perceive what we are looking at (also at some distance), contrary to other great apes who mainly interact on a competitive basis.7 7 In the seminal paper “On the human interaction engine”, Levinson (2006) claims that human beings have a unique propensity to interact and their interactions have a distinctive structure, when compared to other species. 20 20 claws and the development of grasping extremities would point toward insect eating and predation with the important aid of vision. Allman (1977) confirmed that the frontal position of eyes with a spherical lens favors visual acuity even more than stereopsis (animals with eyes on the side have more blurred vision), and this is particularly important at night, in low light. In other words, the predators that favored frontal eyes are mainly nocturnal predators, because for diurnal ones the lateral position of the eyes is less of a problem. Alternatively, Sussman (1991) suggests that what would have favored certain evolutionary adaptation in primates is fruit foraging (better perception of edges, shapes and distance would help seeing them). Moreover, contrary to most other mammals, primates have trichromatic vision and this appears to have developed in relation to fruit foraging because it helped recognizing mature fruits through leaves and vegetation (see, e.g., Jacobs, 1995; Regan et al., 2001). If binocular vision and stereopsis contribute to 3-dimensional vision and allow a better perception of shapes and distances, then it is no surprise to find out that human beings look at objects anticipatorily, before reaching for them. They do so in order to calibrate their movements (see, e.g., Hayhoe & Ballard, 2005; Johansson, Westling, Backstrom, & Flanagan, 2001; Land, 2006), and also to guide their movements while approaching an object to grab it (see, e.g., Jeannerod, 1986 on how finger posturing anticipates the real grasp, occurs during the transportation of the hand and partly relies on visual feedback). Looking at objects helps establishing distances and therefore facilitates the motor control of our hands and arms when we reach for things. Whenever we are involved in a task such as preparing a cup of tea (Land, Mennie, & Rusted, 1999) or a sandwich (Hayhoe, 2000), our eyes mainly perform four monitoring functions: i) locating objects used later in the process; ii) directing the hand or object in the hand to a new location; iii) guiding the approach of one object to another; and, iv) checking the state of some variable relevant for the task. Usually the eyes reach the object that we reach for slightly less than a second before manipulating it, and usually our eyes only visit objects that are relevant for the task-at-hand, independently of how conspicuous other objects in the environment might be. The eyes, then, help us to plan our movements and provide feedback on our motor performance; feedback that becomes less salient the more routinized the action becomes. 21All of the above implies that when we observe individuals in interaction looking at their glass or a cookie before they reach for it, we are simply observing a behavior that has evolved because of the specificity of primate eyes and the advantages and limits of their frontal position. We will see how this phenomenon plays a role in everyday interactions in the empirical chapters that follow. 1.2.2 Eye Morphology in Primates and Eye Movements Human gaze behavior is a highly evolved system: unique amongst primate species, the human orbit has evolved to display the sclera, in humans also referred to as “the white of the eye”, and our eye outline is extraordinarily elongated in the horizontal direction. Human beings are the only primate species with a sclera without pigmentation (see Figure 1.5). The function of these physiological differences seems to be to make gaze direction discernable to others at a distance, as we have lost the “gaze-camouflaging” function common to other primate eyes and have enhanced the “gaze signaling” function (Kobayashi, 1997, 2001; Kobayashi & Hashiya, 2011; Morris, 1985). Some empirical evidence supporting this claim comes from an experiment run by Tomasello et al. (2007) in which human infants and other great apes were compared in terms of their tendency to rely on eyes or head direction in following the gaze of a human experimenter. They found that while human infants rely only on eyes, great apes rely mainly on head direction. This suggests that the presence of a white sclera co-evolved with human cooperative interaction, making it easier for others to perceive what we are looking at (also at some distance), contrary to other great apes who mainly interact on a competitive basis.7 7 In the seminal paper “On the human interaction engine”, Levinson (2006) claims that human beings have a unique propensity to interact and their interactions have a distinctive structure, when compared to other species. 2122 Figure 1.5. Human eyes vs. chimpanzee eyes. Moreover, it has long been established that humans can judge the direction of other humans’ gaze to within a few degrees of arc (Gibson & Pick, 1963). This capacity has been claimed to be crucial for the development of joint attention and human social cognition (see, e.g., Butterworth & Cochran, 1980; Scaife & Bruner, 1975; Tomasello, 1995, 1999), and for language acquisition (see, e.g., Bruner, 1983; Tomasello & Farrar, 1986; Tomasello & Todd, 1983). The capacity to follow gaze, which human infants already develop as early 3 months of age (D'Entremont, Hains, & Muir, 1997) and enables the possibility of establishing joint attention with another individual, is particularly important from a developmental point of view because it helps in understanding what others are thinking or intending to do (e.g., Baron-Cohen, 1995; Frith & Frith, 2001). Moreover, correct gaze following at 10 months predicts language comprehension and gesture production at 18 months, therefore suggesting a strong positive correlation between gaze following and language acquisition (Brooks & Meltzoff, 2005). Individuals with autism appear to have a different way of engaging in gaze following and also an impairment of joint attention (Carpenter & Tomasello, 2000; Charman, 2004; Dawson et al., 2004; Loveland & Landry, 1986; Mundy, 1995; Mundy & Newell, 2007). By the end of the first month of life, human infants show the emergence of “effortful” concentration on the mother’s face (Lavelli & Fogel, 2005) and at 6 months of age, infants are sensitive to the social communicative value of gaze, as they follow gaze only if the gaze shift is preceded by an ostensive signal, such as gaze contact (Senju & Csibra, 2008). Further evidence of human infants’ predisposition to detect gaze behavior as a socially relevant cue comes from the work of Farroni and colleagues (Farroni, Csibra, Simion, & Johnson, 2002; Farroni, Menon, & Johnson, 2006; Farroni, Pividori, Simion, Massaccesi, & Johnson, 2004). This research shows that newborns have a preferential attention toward faces with direct gaze toward the observer, even when these faces are only schematic ones. Interestingly, even newborn chimpanzees have a preference for human faces with gaze directed toward them 23(Myowa-Yamakoshi, Tomonaga, Tanaka, & Matsuzawa, 2003). Thus, all of these findings demonstrate that there is an important relation between being able to detect where somebody else’s eyes are directed and becoming social beings. Focusing on the interaction between eyes and development, one of the most notable features of a human infant face is the size of her/his eyes: they are particularly large compared to the rest of their body. At birth, a human eye has a diameter that is approximately 70% of the size of an adult eye: 17 mm at birth vs. 24 mm as an adult (Gordon & Donzis, 1985). In this respect, the eye is probably the human organ that grows the least during development. Two kinds of vertebrate tend to have large eyes compared to their body size: nocturnal predators and herbivore mammals such as cows and rabbits (see, e.g., Howland, Merola, & Basarab, 2004 on vertebrate eye sizes). In the first case, big eyes facilitate the detection of prey at a distance; in the latter they help the detection of predators. Clearly, a newborn human is anything but a great predator and has little capacity to escape if attacked. Moreover, if one of the advantages of having big eyes is that they improve visual acuity (and therefore facilitate the detection of prey at a distance), this cannot apply to human newborns because they can only see at a distance of about 8-15 inches in the first few weeks. This distance is important because it is approximately the distance between the faces of a baby and its caregiver when the latter holds the baby in her/his arms. So, we are not born with highly developed eyes that can easily scan the environment at a distance and detect things, but newborn eyes are particularly sensitive to human faces, and to black and white forms, much more than to colors (see, e.g., Fantz, 1963). A possible hypothesis concerning the evolutionary advantage of infant eye size and perceptual capacities at birth is that the range of visual acuity and large white sclera can facilitate mother-infant communication. Thus, they may have evolved this way for social/communicative purposes (see Ross & Kirk, 2007 for a review on eye size in primates). 2222 Figure 1.5. Human eyes vs. chimpanzee eyes. Moreover, it has long been established that humans can judge the direction of other humans’ gaze to within a few degrees of arc (Gibson & Pick, 1963). This capacity has been claimed to be crucial for the development of joint attention and human social cognition (see, e.g., Butterworth & Cochran, 1980; Scaife & Bruner, 1975; Tomasello, 1995, 1999), and for language acquisition (see, e.g., Bruner, 1983; Tomasello & Farrar, 1986; Tomasello & Todd, 1983). The capacity to follow gaze, which human infants already develop as early 3 months of age (D'Entremont, Hains, & Muir, 1997) and enables the possibility of establishing joint attention with another individual, is particularly important from a developmental point of view because it helps in understanding what others are thinking or intending to do (e.g., Baron-Cohen, 1995; Frith & Frith, 2001). Moreover, correct gaze following at 10 months predicts language comprehension and gesture production at 18 months, therefore suggesting a strong positive correlation between gaze following and language acquisition (Brooks & Meltzoff, 2005). Individuals with autism appear to have a different way of engaging in gaze following and also an impairment of joint attention (Carpenter & Tomasello, 2000; Charman, 2004; Dawson et al., 2004; Loveland & Landry, 1986; Mundy, 1995; Mundy & Newell, 2007). By the end of the first month of life, human infants show the emergence of “effortful” concentration on the mother’s face (Lavelli & Fogel, 2005) and at 6 months of age, infants are sensitive to the social communicative value of gaze, as they follow gaze only if the gaze shift is preceded by an ostensive signal, such as gaze contact (Senju & Csibra, 2008). Further evidence of human infants’ predisposition to detect gaze behavior as a socially relevant cue comes from the work of Farroni and colleagues (Farroni, Csibra, Simion, & Johnson, 2002; Farroni, Menon, & Johnson, 2006; Farroni, Pividori, Simion, Massaccesi, & Johnson, 2004). This research shows that newborns have a preferential attention toward faces with direct gaze toward the observer, even when these faces are only schematic ones. Interestingly, even newborn chimpanzees have a preference for human faces with gaze directed toward them 23(Myowa-Yamakoshi, Tomonaga, Tanaka, & Matsuzawa, 2003). Thus, all of these findings demonstrate that there is an important relation between being able to detect where somebody else’s eyes are directed and becoming social beings. Focusing on the interaction between eyes and development, one of the most notable features of a human infant face is the size of her/his eyes: they are particularly large compared to the rest of their body. At birth, a human eye has a diameter that is approximately 70% of the size of an adult eye: 17 mm at birth vs. 24 mm as an adult (Gordon & Donzis, 1985). In this respect, the eye is probably the human organ that grows the least during development. Two kinds of vertebrate tend to have large eyes compared to their body size: nocturnal predators and herbivore mammals such as cows and rabbits (see, e.g., Howland, Merola, & Basarab, 2004 on vertebrate eye sizes). In the first case, big eyes facilitate the detection of prey at a distance; in the latter they help the detection of predators. Clearly, a newborn human is anything but a great predator and has little capacity to escape if attacked. Moreover, if one of the advantages of having big eyes is that they improve visual acuity (and therefore facilitate the detection of prey at a distance), this cannot apply to human newborns because they can only see at a distance of about 8-15 inches in the first few weeks. This distance is important because it is approximately the distance between the faces of a baby and its caregiver when the latter holds the baby in her/his arms. So, we are not born with highly developed eyes that can easily scan the environment at a distance and detect things, but newborn eyes are particularly sensitive to human faces, and to black and white forms, much more than to colors (see, e.g., Fantz, 1963). A possible hypothesis concerning the evolutionary advantage of infant eye size and perceptual capacities at birth is that the range of visual acuity and large white sclera can facilitate mother-infant communication. Thus, they may have evolved this way for social/communicative purposes (see Ross & Kirk, 2007 for a review on eye size in primates). 23 24 1.2.3 Eyes and Vision Our retina, the light sensitive layer at the back of our eye, is made of two basic photosensitive cells: rods and cones. Rods are photoreceptors that work particularly well in conditions of low light and allow scotopic vision; that is, the monochromatic vision of the eye in conditions of low luminance (i.e., when it is dark). Given that cone cells cannot function in conditions of low luminance, only rod cells are active in those conditions and this means that in situations of darkness there is no color perception. Rods are particularly important for night vision, motion detection and peripheral vision. On the other hand, cones are the cells responsible for color vision,8 the ones that provide us with the highest visual acuity. There are no rods in the foveal pit, the center of the fovea, the part of the retina responsible for sharp vision, while cones are mostly concentrated in the fovea. This distribution of these two photoreceptors allows us to see over a wider range. The area rich in cones (the fovea) makes us see with high visual acuity during bright light, while the area rich in rods (i.e., the rest of the retina) provides high sensitivity vision in dim light. Such a distribution allows for a good balance between visual acuity and visual sensitivity. The fovea represents less than 1% of the retina and we can see sharply only through the fovea. This explains why we can see sharply only about 15-30 degrees of arc, while we have a basic perception of about 200 degrees (Bruce et al., 2003). This biological configuration also explains why we have to move our eyes to scan the environment and perceive the shapes of objects in a room with a decent level of visual acuity. It also explains why we move our eyes to look at a cup before we reach for it: excluding for a moment the role of the lens in visual acuity, if we had cones more evenly distributed throughout the entire retina, we would be able to perceive the position of the cup and its shape with high definition9 while still looking at the face of our interlocutor. However, given that our retinas do not have this configuration, our eyes have to be redirected toward the object and moved away from the other participant to properly control our hand movement. 8 Most mammals are dichromats. Primates are the only mammals having trichromatic vision (Jacobs, 1993, 1995). Visual acuity of humans is not really remarkable, in comparison to other species. Indeed, most birds that hunt can see better than us (e.g. the eagle) and many diurnal birds can see ultraviolet light. 9 Note that visual acuity is not only dependent on the presence of cones in the retina, but other factors affect it as well, among which how we can modify the thickness of our lenses to bring nearby objects into focus, through a process called accommodation (see, e.g., Bruce et al., 2003). 25In the second half of the 20th century developmental and psycholinguistic researchers exploited the correlation between eye movements and foveal vision with cognition and attention. This happened in particular as a development of the initial investigations in the 1950s and 1960s made by Robert Fantz with infants and by Alfred L. Yarbus on scene perception and saccadic eye movements in adults. Fixation duration, direction of first look and frequency of switching between co-present objects have been interpreted as reflections of underlying cognitive processing. A detailed investigation of the history and findings of looking-time research in developmental psychology and of eye movement research in reading, during scene perception and language production (see, e.g., the success of the Visual World Paradigm in psycholinguistics) is beyond the scope of this dissertation. Some comprehensive reviews and methodological discussions of the above approaches can be found, however, in Aslin (2007); Griffin (2004); Rayner (1998), Spelke (1985) and Tanenhaus et al. (2000) and an extensive overview of recent works on eyes and perception can be found in Liversedge et al. (2011). Given the focus here on the use of eye-gaze during social interaction, no detailed review of the visual perception of humans will be provided here. However, it is worth reporting how different models of visual perception have strongly influenced our understanding of the function of our eyes and how they interact with the environment that surrounds us. A behaviorist approach to perception, such as the one proposed by Watson (1913, 1924), attempted to reduce perception to basic sensations excluding introspection. This was countered by the Gestalt psychology approach, which pushed for a more phenomenological and nativist approach to vision, with a strong focus on introspection. In recent years, Gregory (1972, 1980) proposed a top-down approach to vision, where hypotheses about the world are constantly tested against fragmented evidence based on the images we perceive. This was countered by Marr (1982) with an alternative computational bottom-up approach, emerging from artificial intelligence. Marr’s approach is fundamentally unconscious and computational in scope, which relies on inferences about the images perceived. It also provides humans with a certain level of awareness of the visual world by processing the information obtained via the retina at different levels by using statistical regularities. Finally, an influential approach to vision that developed in the last part of the 20th century is the ecological one, which owes its initial framework to the work of J.J. Gibson (e.g., 1979). The core intuition of this approach is that action and perception are intimately intertwined and cannot be treated as separate problems. The point of departure should not be the stimulation of the light on the retinal image, but rather the optic array provided by the 24 24 1.2.3 Eyes and Vision Our retina, the light sensitive layer at the back of our eye, is made of two basic photosensitive cells: rods and cones. Rods are photoreceptors that work particularly well in conditions of low light and allow scotopic vision; that is, the monochromatic vision of the eye in conditions of low luminance (i.e., when it is dark). Given that cone cells cannot function in conditions of low luminance, only rod cells are active in those conditions and this means that in situations of darkness there is no color perception. Rods are particularly important for night vision, motion detection and peripheral vision. On the other hand, cones are the cells responsible for color vision,8 the ones that provide us with the highest visual acuity. There are no rods in the foveal pit, the center of the fovea, the part of the retina responsible for sharp vision, while cones are mostly concentrated in the fovea. This distribution of these two photoreceptors allows us to see over a wider range. The area rich in cones (the fovea) makes us see with high visual acuity during bright light, while the area rich in rods (i.e., the rest of the retina) provides high sensitivity vision in dim light. Such a distribution allows for a good balance between visual acuity and visual sensitivity. The fovea represents less than 1% of the retina and we can see sharply only through the fovea. This explains why we can see sharply only about 15-30 degrees of arc, while we have a basic perception of about 200 degrees (Bruce et al., 2003). This biological configuration also explains why we have to move our eyes to scan the environment and perceive the shapes of objects in a room with a decent level of visual acuity. It also explains why we move our eyes to look at a cup before we reach for it: excluding for a moment the role of the lens in visual acuity, if we had cones more evenly distributed throughout the entire retina, we would be able to perceive the position of the cup and its shape with high definition9 while still looking at the face of our interlocutor. However, given that our retinas do not have this configuration, our eyes have to be redirected toward the object and moved away from the other participant to properly control our hand movement. 8 Most mammals are dichromats. Primates are the only mammals having trichromatic vision (Jacobs, 1993, 1995). Visual acuity of humans is not really remarkable, in comparison to other species. Indeed, most birds that hunt can see better than us (e.g. the eagle) and many diurnal birds can see ultraviolet light. 9 Note that visual acuity is not only dependent on the presence of cones in the retina, but other factors affect it as well, among which how we can modify the thickness of our lenses to bring nearby objects into focus, through a process called accommodation (see, e.g., Bruce et al., 2003). 25In the second half of the 20th century developmental and psycholinguistic researchers exploited the correlation between eye movements and foveal vision with cognition and attention. This happened in particular as a development of the initial investigations in the 1950s and 1960s made by Robert Fantz with infants and by Alfred L. Yarbus on scene perception and saccadic eye movements in adults. Fixation duration, direction of first look and frequency of switching between co-present objects have been interpreted as reflections of underlying cognitive processing. A detailed investigation of the history and findings of looking-time research in developmental psychology and of eye movement research in reading, during scene perception and language production (see, e.g., the success of the Visual World Paradigm in psycholinguistics) is beyond the scope of this dissertation. Some comprehensive reviews and methodological discussions of the above approaches can be found, however, in Aslin (2007); Griffin (2004); Rayner (1998), Spelke (1985) and Tanenhaus et al. (2000) and an extensive overview of recent works on eyes and perception can be found in Liversedge et al. (2011). Given the focus here on the use of eye-gaze during social interaction, no detailed review of the visual perception of humans will be provided here. However, it is worth reporting how different models of visual perception have strongly influenced our understanding of the function of our eyes and how they interact with the environment that surrounds us. A behaviorist approach to perception, such as the one proposed by Watson (1913, 1924), attempted to reduce perception to basic sensations excluding introspection. This was countered by the Gestalt psychology approach, which pushed for a more phenomenological and nativist approach to vision, with a strong focus on introspection. In recent years, Gregory (1972, 1980) proposed a top-down approach to vision, where hypotheses about the world are constantly tested against fragmented evidence based on the images we perceive. This was countered by Marr (1982) with an alternative computational bottom-up approach, emerging from artificial intelligence. Marr’s approach is fundamentally unconscious and computational in scope, which relies on inferences about the images perceived. It also provides humans with a certain level of awareness of the visual world by processing the information obtained via the retina at different levels by using statistical regularities. Finally, an influential approach to vision that developed in the last part of the 20th century is the ecological one, which owes its initial framework to the work of J.J. Gibson (e.g., 1979). The core intuition of this approach is that action and perception are intimately intertwined and cannot be treated as separate problems. The point of departure should not be the stimulation of the light on the retinal image, but rather the optic array provided by the 25 26 environment. The way in which light extends over space and time provides information to the observer about shapes and guides the movements of animals through space. We do not just try to perceive shapes to memorize pictures in our brain; we use vision to make sense of what we can do in and with an environment. In particular, when we perceive an object, we do not just see what it looks like, but we also see its affordances, the actions that we can perform with it, its possible uses. Contrary to conventional theories of vision, which postulate a view of perception that is mediated by inferences about the world that develop by transforming mental representations, Gibson suggests that perception is not mediated by any psychological process such as memory or inferences but is rather an immediate picking up of invariants in the optic array. Recently, however, Milner and Goodale (1995) have put forward a different hypothesis about vision, one that differentiates a visual path for action from a visual path for perception. Basing their claim initially on findings from research on individuals with brain damage, they suggest that there are two different pathways in the brain, as far as our visual system is concerned: one dedicated to perception (the ventral stream) and one dedicated to action (the dorsal stream). The idea is that visual input affects our actions in different ways. The dorsal pathway controls fast movements and allows us to grasp, for example, the handle of a cup. On the other hand, through the ventral pathway, the visual input interacts with memory and allows us to construct a conscious and more accessible representation of the world, allowing us to plan for more complex courses of action. Many recent experiments conducted either on individuals with brain damage or using visual illusions seem to confirm the existence of these two different pathways (see Milner & Goodale, 2008 for a review). This model combines the insights of Marr’s computational model and Gibson’s ecological approach, by suggesting that both types of processing of visual information are actually occurring in our brain at the same time. 271.3 To Be Seen A recent study found that by simply placing a pair of printed eyes on an honesty box used to collect money for drinks in a university coffee room, people paid three times as much for their drinks than they did when a control image was used (Bateson, Nettle & Roberts, 2006). Just seeing a pair of eyes had apparently caused people to behave quite differently, in particular being more cooperative and generous, than in the control condition. One of the accounts for this finding is that at an unconscious level, printed open eyes facing the participants induced the perception of being watched (and therefore evoked an association to their accountability to others). Another experiment by von Grünau and Anston (1995) showed that people were much faster and more precise in detecting the presence of a straight gaze directed toward them among a large number of averted gaze, rather than the other way around. This special alertness to a straight gaze stimuli has a clear adaptive explanation: staring can be a sign of hostility or anger, but eye contact can also be a sign of liking or friendliness, and as the authors explain “whether maintained stare is a sign of dislike or like, it is certainly an indication of a potential social interaction.” (p. 1297) We know that humans have neurons that respond very selectively to stimuli that look like faces and eyes (e.g., Emery, 2000; Haxby, Hoffman, & Gobbini, 2000), and we have already mentioned how human infants are sensitive to direct gaze toward them even when schematic faces are used, rather than photographs of real faces (Farroni et al., 2004). One reason why the impression of being watched might affect human behaviors could be that we are socially concerned with maintaining face (Goffman, 1955, 1963) and naturally concerned with maintaining a positive reputation (see, e.g., Alexander, 1987; Milinski, Semmann, & Krambeck, 2002). However, another option is possible as well: we are more cooperative when we think we are being watched because we are concerned about punishment. From a sociological point of view, for example, we know from the work of Foucault in Discipline and Punish (1979) that the control of a prisoner and his behavior can be institutionally exerted just by making the prisoner believe that he is being observed. Foucault cites in particular the plan for a perfect prison, originally suggested by Bentham, that he calls Panopticon, in which a single tower in the middle of a prison courtyard with a 360 degrees view of every cell’s window, would immediately make prisoners be more cooperative, as they would never know when they are actually observed by a guard. So the fear of being 26 26 environment. The way in which light extends over space and time provides information to the observer about shapes and guides the movements of animals through space. We do not just try to perceive shapes to memorize pictures in our brain; we use vision to make sense of what we can do in and with an environment. In particular, when we perceive an object, we do not just see what it looks like, but we also see its affordances, the actions that we can perform with it, its possible uses. Contrary to conventional theories of vision, which postulate a view of perception that is mediated by inferences about the world that develop by transforming mental representations, Gibson suggests that perception is not mediated by any psychological process such as memory or inferences but is rather an immediate picking up of invariants in the optic array. Recently, however, Milner and Goodale (1995) have put forward a different hypothesis about vision, one that differentiates a visual path for action from a visual path for perception. Basing their claim initially on findings from research on individuals with brain damage, they suggest that there are two different pathways in the brain, as far as our visual system is concerned: one dedicated to perception (the ventral stream) and one dedicated to action (the dorsal stream). The idea is that visual input affects our actions in different ways. The dorsal pathway controls fast movements and allows us to grasp, for example, the handle of a cup. On the other hand, through the ventral pathway, the visual input interacts with memory and allows us to construct a conscious and more accessible representation of the world, allowing us to plan for more complex courses of action. Many recent experiments conducted either on individuals with brain damage or using visual illusions seem to confirm the existence of these two different pathways (see Milner & Goodale, 2008 for a review). This model combines the insights of Marr’s computational model and Gibson’s ecological approach, by suggesting that both types of processing of visual information are actually occurring in our brain at the same time. 271.3 To Be Seen A recent study found that by simply placing a pair of printed eyes on an honesty box used to collect money for drinks in a university coffee room, people paid three times as much for their drinks than they did when a control image was used (Bateson, Nettle & Roberts, 2006). Just seeing a pair of eyes had apparently caused people to behave quite differently, in particular being more cooperative and generous, than in the control condition. One of the accounts for this finding is that at an unconscious level, printed open eyes facing the participants induced the perception of being watched (and therefore evoked an association to their accountability to others). Another experiment by von Grünau and Anston (1995) showed that people were much faster and more precise in detecting the presence of a straight gaze directed toward them among a large number of averted gaze, rather than the other way around. This special alertness to a straight gaze stimuli has a clear adaptive explanation: staring can be a sign of hostility or anger, but eye contact can also be a sign of liking or friendliness, and as the authors explain “whether maintained stare is a sign of dislike or like, it is certainly an indication of a potential social interaction.” (p. 1297) We know that humans have neurons that respond very selectively to stimuli that look like faces and eyes (e.g., Emery, 2000; Haxby, Hoffman, & Gobbini, 2000), and we have already mentioned how human infants are sensitive to direct gaze toward them even when schematic faces are used, rather than photographs of real faces (Farroni et al., 2004). One reason why the impression of being watched might affect human behaviors could be that we are socially concerned with maintaining face (Goffman, 1955, 1963) and naturally concerned with maintaining a positive reputation (see, e.g., Alexander, 1987; Milinski, Semmann, & Krambeck, 2002). However, another option is possible as well: we are more cooperative when we think we are being watched because we are concerned about punishment. From a sociological point of view, for example, we know from the work of Foucault in Discipline and Punish (1979) that the control of a prisoner and his behavior can be institutionally exerted just by making the prisoner believe that he is being observed. Foucault cites in particular the plan for a perfect prison, originally suggested by Bentham, that he calls Panopticon, in which a single tower in the middle of a prison courtyard with a 360 degrees view of every cell’s window, would immediately make prisoners be more cooperative, as they would never know when they are actually observed by a guard. So the fear of being 27 28 perceived breaking the rules and as such risking being punished would make them cooperate more. Many writers have provided admirable descriptions of how characters in their works experience the look of somebody else toward them (see, e.g., Boll, 1994 [1963]; Sartre, 1956 [1943]).10 Yet one aspect often neglected is how these glances are made possible during a conversation. The answer lies in the ways in which we position our bodies when we talk to each other. In the 50s and 60s Sommer developed a series of studies concerning how participants position themselves in space depending on the kind of interaction they want to have (Sommer, 1959, 1962, 1967), observations later confirmed and developed, among others, by Scheflen (1964, 1975a), Ekman and Friesen (1974) and Kendon (1977). The main aspect of Sommer’s work that matters for us is the idea that participants would position themselves so that they could monitor one another and make eye-contact comfortable, if they were planning to have a conversation. This idea of positioning our bodies so that mutual monitoring becomes easier has been further developed by Goffman (1963, 1964, 1981b) and Marjorie Goodwin (e.g., 1980), who claimed it to be a crucial component of a social situation and showed us how it plays out in real time. Kendon (1977) even coined a specific new term to describe the configuration which participants assume when they have a social interaction: F-formation. An F-formation arises whenever two or more people sustain a spatial and orientational relationship in which the space between them is one to which they have equal, direct, and exclusive access. Such a pattern can be seen in the circle of the free-standing conversational group. […] The F-formation system serves as an important means of maintaining the separate identity and integrity of an interactional situation. It provides a means by which the participants can maintain differential access to one another and it facilitates the maintenance of a common focus of attention. It is thus an important part of the means by which behaviour is organized in occasions of face-to-face interaction such as conversations. (p. 209) The effect of being watched on people’s behavior suggests that eye-gaze has a strong regulatory power, which enables participants to affect the way others behave in their presence. We’ll see later how this has been investigated with respect to social interaction and conversation. 10 See Tomkins (1963) for a historical review concerning how writers have written about the use of eyes in relationships. 29We have already mentioned that two of the most typical situations, in the absence of talk, in which two individuals would find each other sustaining mutual eye contact are situations of threatening and flirting. Even though they seem to represent the two ends of an emotional spectrum, one having to do with menace and aggression and the other one with intimacy and desire for closeness, they have something in common: they are both actions that provoke high arousal (see Eibl-Eiblesfeldt, 1989 for a detailed account of how these arousing situations are managed and how universal and possibly innate they appear to be). Again, if we move from the behavioral level to its biological motivation, we can note how the very perception of two open eyes looking at us, when perceived, provokes a systematic increase in galvanic skin response (GSR) (see e.g., Nichols & Champness, 1971; Hietanen et al. 2008), which is to say we get aroused. Yet, the fact that people look at each other’s face during a conversation regularly would suggest that we have found a way of canalizing these arousing emotions into a communicative practice, as Lorenz (1966) suggested. It should be noted here that other primates, while monitoring each other’s behavior in a manner similar to the one used by human beings, tend to engage in eye contact mainly in agonistic situations, such as threatening, or to solicit sexual favors (see, e.g., Goodenough, McGuire, & Wallace, 1993; Hinde & Rowell, 1962; Kaplan & Rogers, 2002; Redican, 1975). There are, however, reports of the engagement of mutual gaze to initiate reconciliations (de Waal, 2001), and in non-aggressive and non-sexual social interactions (Yamagiwa, 1992), mainly deployed by juveniles. Mutual gaze, even in chimpanzees, occurs quite regularly in mother-infant interactions (Bard et al., 2005; Ehardt & Blount, 1984), while it rarely occurs during adulthood and there appear to be differences in the amount of eye contact that usually occurs not just across species (see, e.g., Kaplan & Rogers, 2002), but also across groups within the same species (Bard et al., 2005). Nonetheless, the length of mutual gaze that occurs during a natural conversation between two human beings clearly far exceeds what is usually observed in other non-human primates. It therefore appears that some special evolutionary adaptation occurred in the human line that allows us to sustain mutual gaze in friendly and yet not arousing situations, even beyond childhood. If we combine this with what was previously reported concerning the different color of human sclera (white vs. dark one for other primates), it should be clear that we are biologically evolved and adapted to rely on gaze direction and mutual gaze during social interaction and communication. 28 28 perceived breaking the rules and as such risking being punished would make them cooperate more. Many writers have provided admirable descriptions of how characters in their works experience the look of somebody else toward them (see, e.g., Boll, 1994 [1963]; Sartre, 1956 [1943]).10 Yet one aspect often neglected is how these glances are made possible during a conversation. The answer lies in the ways in which we position our bodies when we talk to each other. In the 50s and 60s Sommer developed a series of studies concerning how participants position themselves in space depending on the kind of interaction they want to have (Sommer, 1959, 1962, 1967), observations later confirmed and developed, among others, by Scheflen (1964, 1975a), Ekman and Friesen (1974) and Kendon (1977). The main aspect of Sommer’s work that matters for us is the idea that participants would position themselves so that they could monitor one another and make eye-contact comfortable, if they were planning to have a conversation. This idea of positioning our bodies so that mutual monitoring becomes easier has been further developed by Goffman (1963, 1964, 1981b) and Marjorie Goodwin (e.g., 1980), who claimed it to be a crucial component of a social situation and showed us how it plays out in real time. Kendon (1977) even coined a specific new term to describe the configuration which participants assume when they have a social interaction: F-formation. An F-formation arises whenever two or more people sustain a spatial and orientational relationship in which the space between them is one to which they have equal, direct, and exclusive access. Such a pattern can be seen in the circle of the free-standing conversational group. […] The F-formation system serves as an important means of maintaining the separate identity and integrity of an interactional situation. It provides a means by which the participants can maintain differential access to one another and it facilitates the maintenance of a common focus of attention. It is thus an important part of the means by which behaviour is organized in occasions of face-to-face interaction such as conversations. (p. 209) The effect of being watched on people’s behavior suggests that eye-gaze has a strong regulatory power, which enables participants to affect the way others behave in their presence. We’ll see later how this has been investigated with respect to social interaction and conversation. 10 See Tomkins (1963) for a historical review concerning how writers have written about the use of eyes in relationships. 29We have already mentioned that two of the most typical situations, in the absence of talk, in which two individuals would find each other sustaining mutual eye contact are situations of threatening and flirting. Even though they seem to represent the two ends of an emotional spectrum, one having to do with menace and aggression and the other one with intimacy and desire for closeness, they have something in common: they are both actions that provoke high arousal (see Eibl-Eiblesfeldt, 1989 for a detailed account of how these arousing situations are managed and how universal and possibly innate they appear to be). Again, if we move from the behavioral level to its biological motivation, we can note how the very perception of two open eyes looking at us, when perceived, provokes a systematic increase in galvanic skin response (GSR) (see e.g., Nichols & Champness, 1971; Hietanen et al. 2008), which is to say we get aroused. Yet, the fact that people look at each other’s face during a conversation regularly would suggest that we have found a way of canalizing these arousing emotions into a communicative practice, as Lorenz (1966) suggested. It should be noted here that other primates, while monitoring each other’s behavior in a manner similar to the one used by human beings, tend to engage in eye contact mainly in agonistic situations, such as threatening, or to solicit sexual favors (see, e.g., Goodenough, McGuire, & Wallace, 1993; Hinde & Rowell, 1962; Kaplan & Rogers, 2002; Redican, 1975). There are, however, reports of the engagement of mutual gaze to initiate reconciliations (de Waal, 2001), and in non-aggressive and non-sexual social interactions (Yamagiwa, 1992), mainly deployed by juveniles. Mutual gaze, even in chimpanzees, occurs quite regularly in mother-infant interactions (Bard et al., 2005; Ehardt & Blount, 1984), while it rarely occurs during adulthood and there appear to be differences in the amount of eye contact that usually occurs not just across species (see, e.g., Kaplan & Rogers, 2002), but also across groups within the same species (Bard et al., 2005). Nonetheless, the length of mutual gaze that occurs during a natural conversation between two human beings clearly far exceeds what is usually observed in other non-human primates. It therefore appears that some special evolutionary adaptation occurred in the human line that allows us to sustain mutual gaze in friendly and yet not arousing situations, even beyond childhood. If we combine this with what was previously reported concerning the different color of human sclera (white vs. dark one for other primates), it should be clear that we are biologically evolved and adapted to rely on gaze direction and mutual gaze during social interaction and communication. 29 30 However, it is important to note here that when we talk to each other we tend to fixate not each other’s eyes,11 but rather the upper part of the nose, which sort of represents the center of our face, to be able to fully perceive facial expressions (see, e.g., Gullberg & Holmqvist, 2006). Nonetheless, the very possibility of looking at each other and the tendency to often engage in mutual gaze, at least in Western cultures, have been noted by Simmel as the “uniquely sociological function” of the eye and the “union and interaction of individuals [being] based upon mutual glances” (Simmel, 1969: 358). In particular he claimed that “the totality of social relations of human beings, their self-assertions and self-abnegation, their intimacies and estrangements, would be changed in unpredictable ways if there occurred no glance of eye to eye” (1969: 358). Seeing others and being seen, engaging in mutual gaze, appear to have a special significance in human interactions, which might go beyond the mere perceptual or communicative functions of the eyes. The old saying that the eyes are the mirror of the soul, modified by Cicero who claimed that “the face is a picture of the mind as the eyes are its interpreter”, reflects a belief about the relationship between eyes and emotions that has been investigated intensively in the last two centuries. In his work On the expression of emotions in man and animals (1872), for example, Darwin claimed that feelings such as pride, humility, guilt, conceit, slyness, suspicion and others could be detected not just by the facial expression of an individual but more simply by their eyes. Most recent research on facial expressions, in particular the ones by Ekman and colleagues (Ekman, 1992, 1993; Ekman & Friesen, 1978; Ekman & Oster, 1979) but also recent conversation analytic work (Ruusuvuori & Peräkylä, 2009), has looked at the role that eyes are said to play in the compositionality of facial expressions and has considered not only the movements of the eyes but also all the muscle movements going on around them, including the brows, to accomplish, for example, frowns. Despite the importance of the eyes in displays of emotions and in the composition of facial expressions, this dissertation will not focus on these aspects, but rather on the orientation of gaze during social interaction and its communicative and regulatory functions. In what follows I will provide some reasons for this focus. 11 When we actually look at each other’s eyes and not just each other’s face we usually fixate one of the two eyes, as fixation must converge on a single point of focus. 311.3.1 Eyes, Faces and Brain Recent research in neuroscience has focused on what has been called the social brain (e.g., Adolphs, 2009; Brothers, 1990; Dunbar, 1998; Grossmann & Johnson, 2007; Johnson et al., 2005), which is to say the neural networks involved in human social interaction and communication. Among the areas identified as part of the social brain, there are the prefrontal cortex, the superior temporal gyrus, the fusiform gyrus, the cingulate gyrus, the superior temporal sulcus (STS) and the amygdala. These areas are specialized in processing information such as face, gaze, biological motion, human action, goal directedness, theory of mind and empathy. Before focusing on the brain regions involved in the processing of eye gaze, consider two recent findings concerning the differential processing of face and eyes. Haxby et al. (2000) found that while changeable, dynamic aspects of faces such as expressions and gaze tend to activate the STS (area involved in the detection of biological motion, Allison, Puce, & McCarthy, 2000), the static aspects of faces that enable recognition and identification activate the fusiform gyrus. More recently, Calder et al. (2007) provided an even more fine-grained finding concerning the compositional nature of face processing and the separation of gaze direction detection and facial identity: left and right gaze direction of the other are coded in the right anterior STS and in the inferior parietal lobule (involved in the orienting of attention), while facial identity is coded in the posterior fusiform gyrus. Moreover, we know that while the anterior STS is involved in the coding of different gaze directions, the posterior STS is actually sensitive to the intentionality of gaze and other biological signals (Pelphrey, Morris, & McCarthy, 2004), suggesting that a further distinction can be made between processing the movement of somebody’s eyes and detecting whether they are meant to communicate something or whether they are doing more than simply scanning the environment. Besides the right regions of the STS, there are other brain areas involved in the processing of gaze (see Grossmann & Farroni, 2009; Senju & Johnson, 2009 for more detailed overviews): the amygdala (activated in the processing of emotions or by arousing stimuli), the medial prefrontal cortex (involved in mentalizing12 and detecting and decoding communicative intentions toward self) and the orbitofrontal cortex (involved in decision 12 With the term mentalizing I am here referring to the capacity to attribute intentions to self and others, a mechanism considered to be an automatic cognitive process (Leslie, 1987). 30 30 However, it is important to note here that when we talk to each other we tend to fixate not each other’s eyes,11 but rather the upper part of the nose, which sort of represents the center of our face, to be able to fully perceive facial expressions (see, e.g., Gullberg & Holmqvist, 2006). Nonetheless, the very possibility of looking at each other and the tendency to often engage in mutual gaze, at least in Western cultures, have been noted by Simmel as the “uniquely sociological function” of the eye and the “union and interaction of individuals [being] based upon mutual glances” (Simmel, 1969: 358). In particular he claimed that “the totality of social relations of human beings, their self-assertions and self-abnegation, their intimacies and estrangements, would be changed in unpredictable ways if there occurred no glance of eye to eye” (1969: 358). Seeing others and being seen, engaging in mutual gaze, appear to have a special significance in human interactions, which might go beyond the mere perceptual or communicative functions of the eyes. The old saying that the eyes are the mirror of the soul, modified by Cicero who claimed that “the face is a picture of the mind as the eyes are its interpreter”, reflects a belief about the relationship between eyes and emotions that has been investigated intensively in the last two centuries. In his work On the expression of emotions in man and animals (1872), for example, Darwin claimed that feelings such as pride, humility, guilt, conceit, slyness, suspicion and others could be detected not just by the facial expression of an individual but more simply by their eyes. Most recent research on facial expressions, in particular the ones by Ekman and colleagues (Ekman, 1992, 1993; Ekman & Friesen, 1978; Ekman & Oster, 1979) but also recent conversation analytic work (Ruusuvuori & Peräkylä, 2009), has looked at the role that eyes are said to play in the compositionality of facial expressions and has considered not only the movements of the eyes but also all the muscle movements going on around them, including the brows, to accomplish, for example, frowns. Despite the importance of the eyes in displays of emotions and in the composition of facial expressions, this dissertation will not focus on these aspects, but rather on the orientation of gaze during social interaction and its communicative and regulatory functions. In what follows I will provide some reasons for this focus. 11 When we actually look at each other’s eyes and not just each other’s face we usually fixate one of the two eyes, as fixation must converge on a single point of focus. 311.3.1 Eyes, Faces and Brain Recent research in neuroscience has focused on what has been called the social brain (e.g., Adolphs, 2009; Brothers, 1990; Dunbar, 1998; Grossmann & Johnson, 2007; Johnson et al., 2005), which is to say the neural networks involved in human social interaction and communication. Among the areas identified as part of the social brain, there are the prefrontal cortex, the superior temporal gyrus, the fusiform gyrus, the cingulate gyrus, the superior temporal sulcus (STS) and the amygdala. These areas are specialized in processing information such as face, gaze, biological motion, human action, goal directedness, theory of mind and empathy. Before focusing on the brain regions involved in the processing of eye gaze, consider two recent findings concerning the differential processing of face and eyes. Haxby et al. (2000) found that while changeable, dynamic aspects of faces such as expressions and gaze tend to activate the STS (area involved in the detection of biological motion, Allison, Puce, & McCarthy, 2000), the static aspects of faces that enable recognition and identification activate the fusiform gyrus. More recently, Calder et al. (2007) provided an even more fine-grained finding concerning the compositional nature of face processing and the separation of gaze direction detection and facial identity: left and right gaze direction of the other are coded in the right anterior STS and in the inferior parietal lobule (involved in the orienting of attention), while facial identity is coded in the posterior fusiform gyrus. Moreover, we know that while the anterior STS is involved in the coding of different gaze directions, the posterior STS is actually sensitive to the intentionality of gaze and other biological signals (Pelphrey, Morris, & McCarthy, 2004), suggesting that a further distinction can be made between processing the movement of somebody’s eyes and detecting whether they are meant to communicate something or whether they are doing more than simply scanning the environment. Besides the right regions of the STS, there are other brain areas involved in the processing of gaze (see Grossmann & Farroni, 2009; Senju & Johnson, 2009 for more detailed overviews): the amygdala (activated in the processing of emotions or by arousing stimuli), the medial prefrontal cortex (involved in mentalizing12 and detecting and decoding communicative intentions toward self) and the orbitofrontal cortex (involved in decision 12 With the term mentalizing I am here referring to the capacity to attribute intentions to self and others, a mechanism considered to be an automatic cognitive process (Leslie, 1987). 31 32 making). Moreover, a study by Kampe et al. (2003) has shown that hearing somebody calling our name and seeing somebody’s eyes directed toward us for a prolonged amount of time, while differing in modality and sensory channel, tend to activate similar brain areas, usually involved in mentalizing, namely the paracingulate cortex and the temporal lobes. The meaning of this finding is that both calling somebody’s name and directing the eyes toward that person, while acting like a summoning, signal the intention of initiating a communication by the other participant and our brain registers it. This explains the common experience of being in a public place, registering that somebody is looking at us and by the time we look up toward them, the person who was staring at us swiftly lowers the gaze or looks away. When this does not happen, the alternative is usually the initiation of a conversational exchange. Prolonged gaze toward our face works fundamentally like a verbal summoning, at least as far as our brain is concerned. All the findings reported above show that when we observe somebody’s eyes (directed toward us or even averted) a series of reactions occur in our brain, that might have to do with arousal and emotional responses if the context suggests so, but more generally that involve detecting communicative intentions directed toward us (i.e. somebody wants to communicate with us) and deciding what to do next, how to respond. The fact that gaze directed toward us indicates not only that we are the focus of somebody’s attention but also an intention to communicate with us is further confirmed by recent experimental work with infants developed by Csibra and Gergely and their model of natural pedagogy (see Csibra & Gergely, 2009 for a recent summary of their model). In their model of infant social learning, gaze toward infants is one of the ostensive cues that infants use to recognize that they are the ones being addressed, to expect the occurrence of some referential act in an ostensive context and to interpret the content of the communication as kind-relevant and generalizable. We will see in the following chapters how these neurological and psychological findings contribute to our understanding of the mechanisms at play during social interaction, in particular with respect to the use and perception of eye-gaze. 331.4 Gaze and Social Interaction Most of the detailed work on gaze in interaction has been conducted by social psychologists or kinesic researchers working on interaction in English or other European languages. The main focus of social psychologists has been to determine how specific factors would correlate with the amount of looking toward each other’s faces during a dyadic interaction, using such variables as personality, gender, age, social status, asymmetric relationships, seating arrangements and similar (see Kleinke, 1986 for a review of research in this area). Researchers such as Ralph Exline, Michael Argyle and Phoebe Ellsworth pioneered research in this area in the 60s and 70s. Alongside the social psychological approach, there was a move from linguists and anthropologists, together with those interested in cybernetics and psychotherapy, toward including nonverbal features of the interaction in their analysis and descriptions. I have already mentioned Edward Sapir, but there were many others as well including Elliot Chapple, Edward T. Hall, Gregory Bateson, Albert Scheflen and Ray Birdwhistell. The work of Birdwhistell, founder of the kinesic approach, has been particularly inspiring for many generations of researchers (e.g., Adam Kendon among others). Birdwhistell (see, e.g., 1970) tried to provide the field with a method and theoretical tools to produce an accurate and detailed description of body movements during an interaction, assuming their compositionality, communicative import and cultural variability (see, e.g., Leach, 1972). He assumed that there was nothing in peoples’ behavior that should be disregarded a priori as meaningless. This assumption has later been pushed even further by Conversation Analysts as summarized in Harvey Sacks’ phrase “order at all points” (Sacks, 1992 [1964-72]: 483). On the other hand, the obsession with wanting to provide a fully accurate description of any body movement, the focus on single case analysis and the attempt to divide body movement in pseudo-linguistic elements, similar to morphemes, led to the development of extremely long descriptions of body behavior that could not be used in any systematic comparison or corpus study. The fact that Birdwhistell never fully completed a description of any stretch of interaction he investigated confirmed the problem of having a too fine-grained tool to analyze something that might be interpreted and understood using different, broader, categories. There are, however, a series of insights from that approach, as reported by Albert Scheflen (1975b), a close collaborator of Birdwhistell: 32 32 making). Moreover, a study by Kampe et al. (2003) has shown that hearing somebody calling our name and seeing somebody’s eyes directed toward us for a prolonged amount of time, while differing in modality and sensory channel, tend to activate similar brain areas, usually involved in mentalizing, namely the paracingulate cortex and the temporal lobes. The meaning of this finding is that both calling somebody’s name and directing the eyes toward that person, while acting like a summoning, signal the intention of initiating a communication by the other participant and our brain registers it. This explains the common experience of being in a public place, registering that somebody is looking at us and by the time we look up toward them, the person who was staring at us swiftly lowers the gaze or looks away. When this does not happen, the alternative is usually the initiation of a conversational exchange. Prolonged gaze toward our face works fundamentally like a verbal summoning, at least as far as our brain is concerned. All the findings reported above show that when we observe somebody’s eyes (directed toward us or even averted) a series of reactions occur in our brain, that might have to do with arousal and emotional responses if the context suggests so, but more generally that involve detecting communicative intentions directed toward us (i.e. somebody wants to communicate with us) and deciding what to do next, how to respond. The fact that gaze directed toward us indicates not only that we are the focus of somebody’s attention but also an intention to communicate with us is further confirmed by recent experimental work with infants developed by Csibra and Gergely and their model of natural pedagogy (see Csibra & Gergely, 2009 for a recent summary of their model). In their model of infant social learning, gaze toward infants is one of the ostensive cues that infants use to recognize that they are the ones being addressed, to expect the occurrence of some referential act in an ostensive context and to interpret the content of the communication as kind-relevant and generalizable. We will see in the following chapters how these neurological and psychological findings contribute to our understanding of the mechanisms at play during social interaction, in particular with respect to the use and perception of eye-gaze. 331.4 Gaze and Social Interaction Most of the detailed work on gaze in interaction has been conducted by social psychologists or kinesic researchers working on interaction in English or other European languages. The main focus of social psychologists has been to determine how specific factors would correlate with the amount of looking toward each other’s faces during a dyadic interaction, using such variables as personality, gender, age, social status, asymmetric relationships, seating arrangements and similar (see Kleinke, 1986 for a review of research in this area). Researchers such as Ralph Exline, Michael Argyle and Phoebe Ellsworth pioneered research in this area in the 60s and 70s. Alongside the social psychological approach, there was a move from linguists and anthropologists, together with those interested in cybernetics and psychotherapy, toward including nonverbal features of the interaction in their analysis and descriptions. I have already mentioned Edward Sapir, but there were many others as well including Elliot Chapple, Edward T. Hall, Gregory Bateson, Albert Scheflen and Ray Birdwhistell. The work of Birdwhistell, founder of the kinesic approach, has been particularly inspiring for many generations of researchers (e.g., Adam Kendon among others). Birdwhistell (see, e.g., 1970) tried to provide the field with a method and theoretical tools to produce an accurate and detailed description of body movements during an interaction, assuming their compositionality, communicative import and cultural variability (see, e.g., Leach, 1972). He assumed that there was nothing in peoples’ behavior that should be disregarded a priori as meaningless. This assumption has later been pushed even further by Conversation Analysts as summarized in Harvey Sacks’ phrase “order at all points” (Sacks, 1992 [1964-72]: 483). On the other hand, the obsession with wanting to provide a fully accurate description of any body movement, the focus on single case analysis and the attempt to divide body movement in pseudo-linguistic elements, similar to morphemes, led to the development of extremely long descriptions of body behavior that could not be used in any systematic comparison or corpus study. The fact that Birdwhistell never fully completed a description of any stretch of interaction he investigated confirmed the problem of having a too fine-grained tool to analyze something that might be interpreted and understood using different, broader, categories. There are, however, a series of insights from that approach, as reported by Albert Scheflen (1975b), a close collaborator of Birdwhistell: 33 34 1) The dichotomy according to which language is communicative, while every other visible behavior simply works as a cue for who is speaking or what is supposed to happen next is wrong; 2) A participant does not speak, gesture, smile, hold a posture simultaneously to form a single message with redundant parts. Each modality is employed for specific purposes, some of which could be purely communicational, others might be regulatory and others again might be used to induce or sustain specific relationships between the participants in the interaction. The program was pushed for an understanding of visible behavior as a core part of the communicative process and not simply as a mainly redundant side thing, just used to cue who has to start or who has to stop doing something. Their main insights are as follows: 1. Regulatory functions are not the only functions of visible behavior. 2. Visible behavior is organized sequentially (an insight outlined by conversation analysts, e.g., Sacks, 1992 [1964-72]; Schegloff, 1968). 3. What is managed during a conversation is not just the exchange of information and not just direct communication, but a more complex set of things, including relationships and projects that might not be part of the content of the conversational exchange. All these concerns have informed the work contained in this dissertation and will be emphasized in the analysis of specific stretches of interaction. These insights have permeated most of the work on gaze in social interaction, in particular the work of Adam Kendon and Charles Goodwin, whose studies will be described in more detail below. From a conversation analytic point of view, research on gaze in social interaction has focused mainly on three different dimensions: first, its relationship to participation in the conversation; second, its regulatory functions and its role in turn-taking; and third, its role in action formation. These areas, however, have received different amounts of attention, as will be clear from the following report of the main findings in each one of them. 351.4.1 Participation Roles and Participation Frameworks, Engagement and Disengagement A great deal of work has been dedicated to the relationship between gaze, participant roles in conversation and the participation framework in place. The idea that gaze is closely related to participant role (speaking, or being addressed, in particular) is persistent across the literature. In particular, it has long been claimed that in dyadic interaction, people tend to look at the other participant more when they are listening than when they are speaking (Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Bavelas, Coates, & Johnson, 2002; Duncan & Fiske, 1977; Exline, 1963; Goodwin, 1981; Kendon, 1967, 1973, 1990; Kleinke, 1986; Nielsen, 1962; Rutter, 1984). The first person to note this difference was Gerhard Nielsen in his seminal work on self-confrontation (1962). Interested in the timing of visual behavior in face-to-face interaction, he found that not only is there a “tendency to look more at the alter while listening to him than during one’s own speaking periods” but that most of the dyads “show a striking regularity: looking at alter decreased when speaking increased and looking at alter increased while listening increased” (Nielsen, 1962: 141). Kendon (1967) provided a more precise description of the different patterns of speaker and hearer gaze. He claimed that hearers give speakers long looks interrupted by brief glances away, while speakers alternate looks toward and looks away from the recipient of approximately equal length. These studies implicitly suggest that such gaze behaviors are independent of attributes such as race, culture and gender, for instance. There are, however, studies that claim racial differences with respect to the use of gaze to display engagement. Specifically, Black Americans have been shown to look at the recipient more while speaking than while listening, while White Americans follow the opposite pattern (Erickson, 1979; LaFrance, 1974; LaFrance & Mayo, 1976). Moreover, recent work by Rossano et al. (2009) shows that the amount and type of gaze interaction between speaker and recipient may vary across cultures and may also be strongly related to the social actions the participants are initiating through their talk. Indeed, they find that in the context of question-answer sequences in the three cultures investigated (speakers of Italian, speakers of Tzeltal, a Mayan population from Mexico, and speakers of Yélî Dnye, from Rossel Island, a remote island of Papua New Guinea), questioners are more likely to look at their recipients than vice versa, and they tend to do so by looking toward their recipient from the very beginning of the question without alternating looks toward and looks away. 34 34 1) The dichotomy according to which language is communicative, while every other visible behavior simply works as a cue for who is speaking or what is supposed to happen next is wrong; 2) A participant does not speak, gesture, smile, hold a posture simultaneously to form a single message with redundant parts. Each modality is employed for specific purposes, some of which could be purely communicational, others might be regulatory and others again might be used to induce or sustain specific relationships between the participants in the interaction. The program was pushed for an understanding of visible behavior as a core part of the communicative process and not simply as a mainly redundant side thing, just used to cue who has to start or who has to stop doing something. Their main insights are as follows: 1. Regulatory functions are not the only functions of visible behavior. 2. Visible behavior is organized sequentially (an insight outlined by conversation analysts, e.g., Sacks, 1992 [1964-72]; Schegloff, 1968). 3. What is managed during a conversation is not just the exchange of information and not just direct communication, but a more complex set of things, including relationships and projects that might not be part of the content of the conversational exchange. All these concerns have informed the work contained in this dissertation and will be emphasized in the analysis of specific stretches of interaction. These insights have permeated most of the work on gaze in social interaction, in particular the work of Adam Kendon and Charles Goodwin, whose studies will be described in more detail below. From a conversation analytic point of view, research on gaze in social interaction has focused mainly on three different dimensions: first, its relationship to participation in the conversation; second, its regulatory functions and its role in turn-taking; and third, its role in action formation. These areas, however, have received different amounts of attention, as will be clear from the following report of the main findings in each one of them. 351.4.1 Participation Roles and Participation Frameworks, Engagement and Disengagement A great deal of work has been dedicated to the relationship between gaze, participant roles in conversation and the participation framework in place. The idea that gaze is closely related to participant role (speaking, or being addressed, in particular) is persistent across the literature. In particular, it has long been claimed that in dyadic interaction, people tend to look at the other participant more when they are listening than when they are speaking (Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Bavelas, Coates, & Johnson, 2002; Duncan & Fiske, 1977; Exline, 1963; Goodwin, 1981; Kendon, 1967, 1973, 1990; Kleinke, 1986; Nielsen, 1962; Rutter, 1984). The first person to note this difference was Gerhard Nielsen in his seminal work on self-confrontation (1962). Interested in the timing of visual behavior in face-to-face interaction, he found that not only is there a “tendency to look more at the alter while listening to him than during one’s own speaking periods” but that most of the dyads “show a striking regularity: looking at alter decreased when speaking increased and looking at alter increased while listening increased” (Nielsen, 1962: 141). Kendon (1967) provided a more precise description of the different patterns of speaker and hearer gaze. He claimed that hearers give speakers long looks interrupted by brief glances away, while speakers alternate looks toward and looks away from the recipient of approximately equal length. These studies implicitly suggest that such gaze behaviors are independent of attributes such as race, culture and gender, for instance. There are, however, studies that claim racial differences with respect to the use of gaze to display engagement. Specifically, Black Americans have been shown to look at the recipient more while speaking than while listening, while White Americans follow the opposite pattern (Erickson, 1979; LaFrance, 1974; LaFrance & Mayo, 1976). Moreover, recent work by Rossano et al. (2009) shows that the amount and type of gaze interaction between speaker and recipient may vary across cultures and may also be strongly related to the social actions the participants are initiating through their talk. Indeed, they find that in the context of question-answer sequences in the three cultures investigated (speakers of Italian, speakers of Tzeltal, a Mayan population from Mexico, and speakers of Yélî Dnye, from Rossel Island, a remote island of Papua New Guinea), questioners are more likely to look at their recipients than vice versa, and they tend to do so by looking toward their recipient from the very beginning of the question without alternating looks toward and looks away. 35 36 Returning to general claims about gaze patterns in face-to-face interaction, Goodwin proposed two rules (1980: 275, 287; 1981: 57) that would account for gaze behavior in conversation, according to a case by case analysis of his data, and further supported by quantification in a small corpus: 1) “A speaker should obtain the gaze of [her] recipient during the course of a turn at talk.” 2) “A recipient should be gazing at the speaker when the speaker is gazing at the hearer.” If the recipient looks most of the time, then the speaker will find her/him gazing back any time the speaker looks toward the recipient. If the recipient is not looking at the speaker, the speaker has resources (phrasal breaks, pauses, restarting the turn) to solicit the recipient’s gaze. By proposing these as “rules”, Goodwin claims that participants’ gaze behavior is interrelated rather than independent and suggests a normative order to which participants are oriented during any turn-at-talk. In his work on gaze behavior during the course of a story telling, Goodwin (1984) emphasizes how participants’ visible—and, in particular, gaze—behavior helps shape the unfolding of the telling. Moreover, he proposes that a way to understand why gazing away from the speaker during a telling is rarely sanctioned or treated as problematic is that there is a relaxation of the gaze rule due to the co-occurrence of competing activities. However, the very suggestion of a rule that can be ‘relaxed’ immediately calls into question the normative strength of the rule suggested and whether it would not be more appropriate to come up with a rule that would account for this behavior, rather than requiring its relaxation. This dissertation will show that gaze is mainly organized at a different level than the one proposed by Goodwin. In addition to his seminal paper on how gaze deployment affects the construction of an utterance in progress (Goodwin, 1979), the most important work on gaze in social interaction from a conversation analytic perspective is Goodwin’s (1981) Conversational Organization: interaction between speakers and hearers. In this book, he describes gaze behavior as a display of attention and (dis)engagement in the conversation, or more precisely, a display of the type of participation framework the participants are engaged in. From this perspective, looking away is noticeable and potentially sanctionable because it displays diminished engagement in the conversation. On the other hand, engagement in some 37competing activity (e.g. eating) provides a ready account for looking away rather than at the interlocutor’s face and thus makes it less sanctionable. However, this claim also implicitly suggests that participants are relatively free to remove their gaze from co-participants, provided that they direct it toward another activity in which they are engaged. This appears to be problematic from a closer look at a larger number of interactions. Indeed, as it will be later shown (chapter 2), some activities require more sustained gaze by the recipient toward the speaker (e.g., tellings) or by the speaker toward the recipient (e.g., questions) than others, suggesting that this relative freedom actually depends on the gaze expectations associated with the ongoing course of action. Moreover, participants appear to calibrate their looking toward competing activities so that they do not disrupt the progressivity of the talk; that is, for example, they might look toward a glass and pick it up to drink when they expect they won’t have to produce the next turn at talk (see chapter 4), so that drinking will not create a silence when talk would be otherwise expected. Further work on the relationship between gaze and participation has been developed by Heath (1984), who showed how a gaze and postural shift directed toward a co-participant can be used to “display recipiency”. As such, it “is sequentially implicative for an action by a co-participant; it breaks the environment of continuous opportunity, and declares an interest in having some particular action occur in immediate juxtaposition with the display” (1984: 253). In this sense, a body movement can elicit speech by the other participant or it can elicit a gaze re-orientation. In this way, the person who produced the body movement in first position can obtain recipient gaze and begin to speak, making the beginning or the continuation of a conversation possible. Focusing on gaze and body posture as a way of displaying (dis)engagement in a course of action has been used in a fruitful way in the analysis of doctor-patient interactions (see, e.g., Heath, 1986; Robinson, 1998), and in broadcast interactions (Ekström, 2011). The work reported above shows that participant roles (i.e. speaker vs. recipient) affect participant gaze behavior. It also indicates that there may be normative expectations of gaze by recipients toward speakers, at least when they are not involved in competing activities and are listening to an extended telling. Looking toward or looking away from the other participant is often a good clue in terms of participants’ (dis)engagement in the conversation. However, as this dissertation will show, research on gaze and participation frameworks may have not adequately addressed differences related to social action. 36 36 Returning to general claims about gaze patterns in face-to-face interaction, Goodwin proposed two rules (1980: 275, 287; 1981: 57) that would account for gaze behavior in conversation, according to a case by case analysis of his data, and further supported by quantification in a small corpus: 1) “A speaker should obtain the gaze of [her] recipient during the course of a turn at talk.” 2) “A recipient should be gazing at the speaker when the speaker is gazing at the hearer.” If the recipient looks most of the time, then the speaker will find her/him gazing back any time the speaker looks toward the recipient. If the recipient is not looking at the speaker, the speaker has resources (phrasal breaks, pauses, restarting the turn) to solicit the recipient’s gaze. By proposing these as “rules”, Goodwin claims that participants’ gaze behavior is interrelated rather than independent and suggests a normative order to which participants are oriented during any turn-at-talk. In his work on gaze behavior during the course of a story telling, Goodwin (1984) emphasizes how participants’ visible—and, in particular, gaze—behavior helps shape the unfolding of the telling. Moreover, he proposes that a way to understand why gazing away from the speaker during a telling is rarely sanctioned or treated as problematic is that there is a relaxation of the gaze rule due to the co-occurrence of competing activities. However, the very suggestion of a rule that can be ‘relaxed’ immediately calls into question the normative strength of the rule suggested and whether it would not be more appropriate to come up with a rule that would account for this behavior, rather than requiring its relaxation. This dissertation will show that gaze is mainly organized at a different level than the one proposed by Goodwin. In addition to his seminal paper on how gaze deployment affects the construction of an utterance in progress (Goodwin, 1979), the most important work on gaze in social interaction from a conversation analytic perspective is Goodwin’s (1981) Conversational Organization: interaction between speakers and hearers. In this book, he describes gaze behavior as a display of attention and (dis)engagement in the conversation, or more precisely, a display of the type of participation framework the participants are engaged in. From this perspective, looking away is noticeable and potentially sanctionable because it displays diminished engagement in the conversation. On the other hand, engagement in some 37competing activity (e.g. eating) provides a ready account for looking away rather than at the interlocutor’s face and thus makes it less sanctionable. However, this claim also implicitly suggests that participants are relatively free to remove their gaze from co-participants, provided that they direct it toward another activity in which they are engaged. This appears to be problematic from a closer look at a larger number of interactions. Indeed, as it will be later shown (chapter 2), some activities require more sustained gaze by the recipient toward the speaker (e.g., tellings) or by the speaker toward the recipient (e.g., questions) than others, suggesting that this relative freedom actually depends on the gaze expectations associated with the ongoing course of action. Moreover, participants appear to calibrate their looking toward competing activities so that they do not disrupt the progressivity of the talk; that is, for example, they might look toward a glass and pick it up to drink when they expect they won’t have to produce the next turn at talk (see chapter 4), so that drinking will not create a silence when talk would be otherwise expected. Further work on the relationship between gaze and participation has been developed by Heath (1984), who showed how a gaze and postural shift directed toward a co-participant can be used to “display recipiency”. As such, it “is sequentially implicative for an action by a co-participant; it breaks the environment of continuous opportunity, and declares an interest in having some particular action occur in immediate juxtaposition with the display” (1984: 253). In this sense, a body movement can elicit speech by the other participant or it can elicit a gaze re-orientation. In this way, the person who produced the body movement in first position can obtain recipient gaze and begin to speak, making the beginning or the continuation of a conversation possible. Focusing on gaze and body posture as a way of displaying (dis)engagement in a course of action has been used in a fruitful way in the analysis of doctor-patient interactions (see, e.g., Heath, 1986; Robinson, 1998), and in broadcast interactions (Ekström, 2011). The work reported above shows that participant roles (i.e. speaker vs. recipient) affect participant gaze behavior. It also indicates that there may be normative expectations of gaze by recipients toward speakers, at least when they are not involved in competing activities and are listening to an extended telling. Looking toward or looking away from the other participant is often a good clue in terms of participants’ (dis)engagement in the conversation. However, as this dissertation will show, research on gaze and participation frameworks may have not adequately addressed differences related to social action. 37 38 1.4.2 Regulatory Functions of Gaze Much research on gaze in social interaction has focused on its regulatory functions. In what follows I outline what we know about gaze in turn taking, turn allocation, mobilizing response and sequence organization. Early studies of the regulating functions of gaze in turn taking argued that gaze works to show that speaker A is finished talking and thus is used to signal the handing over of the floor. Some researchers have focused on the importance of gaze in monitoring each other’s behavior and facial expressions (e.g., M. H. Goodwin, 1980; Kendon, 1967). Kendon (1967) and Duncan and colleagues (Duncan, 1975; Duncan & Fiske, 1977; Duncan & Niederehe, 1974) have argued that speaker gaze has a ‘floor apportionment’ function in conversation and can function as a turn-yielding cue.13 In particular, they claim that speakers tend to gaze away at the beginning of turns and tend to look up toward the recipient when approaching turn completion in order to signal that they are ready to turn the floor over to the other participant. In his seminal work, Kendon (1967) not only claims that speaker gaze displays the relevance of response but also affects its timing. However, Kendon deals only with the general dynamics of turn taking. He makes no distinction between sentence types (such as interrogatives or imperatives) nor between action types performed through those turns at talk (e.g. announcements, challenges, complaints), but rather writes only in terms of “long” or “short” utterances. Beyond these early studies, subsequent work found no evidence that speakers use gaze as a turn-yielding cue. Beattie (1978, 1979) suggests that a speaker’s looking away during early utterance production, and reengagement during final production, are occasioned purely by the need to reduce cognitive load and that they do not have any regulating function in terms of turn taking.14 In trying to verify previous claims about the occurrence of looking toward a recipient approaching the end of the turn, Torres, et al. (1997) found that, of all turn endings in their data, only 16% included a look toward the recipient by the speaker and these ‘look-toward’ only accounted for 15% of all the speaker ‘look-toward’. Most recently, De Ruiter (2005), examining task-based dialogues, confirmed the lack of a systematic 13 Duncan and colleagues (Duncan, 1975; Duncan & Fiske, 1977; Duncan & Niederehe, 1974) actually refer to a shift in head direction as a turn-yielding cue, but specify that this should be taken as a proxy for ‘eye direction’ (1977: 211). 14 In a direct response to Beattie’s paper, Kendon (1978) argues that Beattie’s data (conversations between a student and her/his supervisor) were not comparable to his (ordinary conversation between Oxford undergraduates), indicating that the kind of interactional situation participants are dealing with may well affect the deployment of gaze. 39relationship between gaze and turn-taking in general, thereby reminding us of the need for a better description of gaze functions in face-to-face interaction. Another related line of research has focused on turn allocation or speaker selection in the context of multi-party conversations, rather than on turn transition. First, Goodwin (1979: 99) claimed that, while uttering a sentence, “the gaze of a speaker should locate the party being gazed at as an addressee of his utterance,” and showed that the very construction of a sentence can be affected by whether recipients return the gaze of the speaker during the uttering of the sentence, thereby allowing for the establishment of eye contact between the participants. More recently, developing Sacks, Schegloff and Jefferson’s (1974) work on turn allocation, Lerner (2003) compared gaze and address terms as ways of addressing a participant and showed that a speaker’s look toward a participant is an explicit form of addressing a participant, but its success is contingent on the gaze practices enacted by the other participants, in particular on whether both the addressed individual and the non-addressed ones perceive the speaker’s gaze. A third, larger stream of research regarding the regulatory functions of gaze has adopted a somewhat different tack in suggesting that gaze works to solicit response. In their work on the social organization of word searches, Goodwin & Goodwin (1986) refer to the occurrence of speaker gaze toward a recipient in two examples of word searches, and they describe its function as a way of soliciting aid from the other participant. However, no systematic evidence for this claim is presented and, in both examples, this apparent solicitation through gaze is, in fact, unsuccessful. More recently, in an experimental setting, Bavelas, et al. (2002: 576-577) find that “the listener tended to respond when the speaker looked at her, and the speaker tended to look away soon after the listener responded. Together, speakers and listeners created and used the gaze window [mutual gaze] to coordinate their actions”. Their notion of the gaze window describes a mutual gaze situation in which it is not just that the speaker’s gaze elicits a response, but rather the listener’s response seems to terminate the speaker’s gaze. By ‘listener responses’ they refer mainly to mm hm, uh huh and nods, which have been labeled back channels in the literature (Yngve, 1970) or, within CA, continuers (Goodwin, 1986b; Schegloff, 1982). Bavelas et al. assert that one of the main features that distinguishes the gaze window from the pattern previously described in turn exchange (i.e. looking toward the recipient indicates that the current speaker is approaching completion of his/her turn and is ready to leave the floor to the other participant) is that there is no exchange of roles between speaker and listener. 38 38 1.4.2 Regulatory Functions of Gaze Much research on gaze in social interaction has focused on its regulatory functions. In what follows I outline what we know about gaze in turn taking, turn allocation, mobilizing response and sequence organization. Early studies of the regulating functions of gaze in turn taking argued that gaze works to show that speaker A is finished talking and thus is used to signal the handing over of the floor. Some researchers have focused on the importance of gaze in monitoring each other’s behavior and facial expressions (e.g., M. H. Goodwin, 1980; Kendon, 1967). Kendon (1967) and Duncan and colleagues (Duncan, 1975; Duncan & Fiske, 1977; Duncan & Niederehe, 1974) have argued that speaker gaze has a ‘floor apportionment’ function in conversation and can function as a turn-yielding cue.13 In particular, they claim that speakers tend to gaze away at the beginning of turns and tend to look up toward the recipient when approaching turn completion in order to signal that they are ready to turn the floor over to the other participant. In his seminal work, Kendon (1967) not only claims that speaker gaze displays the relevance of response but also affects its timing. However, Kendon deals only with the general dynamics of turn taking. He makes no distinction between sentence types (such as interrogatives or imperatives) nor between action types performed through those turns at talk (e.g. announcements, challenges, complaints), but rather writes only in terms of “long” or “short” utterances. Beyond these early studies, subsequent work found no evidence that speakers use gaze as a turn-yielding cue. Beattie (1978, 1979) suggests that a speaker’s looking away during early utterance production, and reengagement during final production, are occasioned purely by the need to reduce cognitive load and that they do not have any regulating function in terms of turn taking.14 In trying to verify previous claims about the occurrence of looking toward a recipient approaching the end of the turn, Torres, et al. (1997) found that, of all turn endings in their data, only 16% included a look toward the recipient by the speaker and these ‘look-toward’ only accounted for 15% of all the speaker ‘look-toward’. Most recently, De Ruiter (2005), examining task-based dialogues, confirmed the lack of a systematic 13 Duncan and colleagues (Duncan, 1975; Duncan & Fiske, 1977; Duncan & Niederehe, 1974) actually refer to a shift in head direction as a turn-yielding cue, but specify that this should be taken as a proxy for ‘eye direction’ (1977: 211). 14 In a direct response to Beattie’s paper, Kendon (1978) argues that Beattie’s data (conversations between a student and her/his supervisor) were not comparable to his (ordinary conversation between Oxford undergraduates), indicating that the kind of interactional situation participants are dealing with may well affect the deployment of gaze. 39relationship between gaze and turn-taking in general, thereby reminding us of the need for a better description of gaze functions in face-to-face interaction. Another related line of research has focused on turn allocation or speaker selection in the context of multi-party conversations, rather than on turn transition. First, Goodwin (1979: 99) claimed that, while uttering a sentence, “the gaze of a speaker should locate the party being gazed at as an addressee of his utterance,” and showed that the very construction of a sentence can be affected by whether recipients return the gaze of the speaker during the uttering of the sentence, thereby allowing for the establishment of eye contact between the participants. More recently, developing Sacks, Schegloff and Jefferson’s (1974) work on turn allocation, Lerner (2003) compared gaze and address terms as ways of addressing a participant and showed that a speaker’s look toward a participant is an explicit form of addressing a participant, but its success is contingent on the gaze practices enacted by the other participants, in particular on whether both the addressed individual and the non-addressed ones perceive the speaker’s gaze. A third, larger stream of research regarding the regulatory functions of gaze has adopted a somewhat different tack in suggesting that gaze works to solicit response. In their work on the social organization of word searches, Goodwin & Goodwin (1986) refer to the occurrence of speaker gaze toward a recipient in two examples of word searches, and they describe its function as a way of soliciting aid from the other participant. However, no systematic evidence for this claim is presented and, in both examples, this apparent solicitation through gaze is, in fact, unsuccessful. More recently, in an experimental setting, Bavelas, et al. (2002: 576-577) find that “the listener tended to respond when the speaker looked at her, and the speaker tended to look away soon after the listener responded. Together, speakers and listeners created and used the gaze window [mutual gaze] to coordinate their actions”. Their notion of the gaze window describes a mutual gaze situation in which it is not just that the speaker’s gaze elicits a response, but rather the listener’s response seems to terminate the speaker’s gaze. By ‘listener responses’ they refer mainly to mm hm, uh huh and nods, which have been labeled back channels in the literature (Yngve, 1970) or, within CA, continuers (Goodwin, 1986b; Schegloff, 1982). Bavelas et al. assert that one of the main features that distinguishes the gaze window from the pattern previously described in turn exchange (i.e. looking toward the recipient indicates that the current speaker is approaching completion of his/her turn and is ready to leave the floor to the other participant) is that there is no exchange of roles between speaker and listener. 39 40 One of the main conclusions that Bavelas, et al. (2002) draw is that the speaker does not look at the listener to monitor him/her for action but rather to solicit a response. A second conclusion is that listeners display their recognition and understanding of the action performed through gaze by responding “immediately and appropriately”. At the same time, it is unclear in which way the occurrence of a response before the withholding of gaze becomes evidence that listeners are responding specifically because of the gaze. Indeed, because of the coding system adopted for this study, a participant could have been looking for 10 seconds and if s/he looks away after a response, then the response would be considered related to the sustaining of the gaze, while the actual talk is not taken into account. A further problem associated with this study is that other communicative behaviors to solicit a response were not taken into account, and, as such, the specific actions performed through talk ignored. This suggests that the claim of a relationship between speaker gaze and listener’s response needs further specification, which has been provided, at least to some extent, by other researchers in more recent studies, to which I now turn. In a paper focusing on the resources that participants deploy to mobilize a response by a recipient, Stivers and Rossano (2010) identify speaker gaze as one such resource. They show that in face-to-face interactions, when the speaker is gazing at the recipient during an assessment or other non-canonical actions, the recipient usually responds to the initial assessment, while assessments that are not responded to are usually produced without speaker gaze or other response mobilizing features. The claim is that speaker gaze plays a role in mobilizing recipient response. Additional evidence for this claim comes from my work on question-answer sequences in Italian dyadic conversations (Rossano, 2010). I focus on the resources that speakers deploy to indicate to recipients that they are producing a question (and, therefore, that they are pressuring recipients for some response). I find that when speakers produce polar (yes/no) questions that do not have a prototypical interrogative intonational contour (i.e. a rising intonation), they are significantly more likely to look toward their recipient than when they are producing questions with prototypical interrogative contours. Given that polar questions in Italian can be marked intonationally but not morpho-syntactically, the occurrence of speaker gaze toward the recipient appears to work as an additional cue that the turn should be responded to. Additionally, in work focusing on the timing of responses to yes-no questions in multiple languages, Stivers, et al. (2009) show that the occurrence of speaker gaze correlates with faster responses to participant questions. Although they do not take into account recipient gaze, and the number of participants involved in the interactions varies, the finding is nevertheless supported in nine out of the ten 41typologically diverse languages investigated. Another recent cross-linguistic study that takes the sequential environment of talk and gaze into account provides additional evidence for the existence of a relationship between the occurrence of speaker gaze and the expectation and timing of a response. In their work comparing gaze behavior during questions in three unrelated languages and cultures mentioned earlier, Rossano, et al. (2009) show that, on average, speakers look at recipients during questions in 73% of cases. They also show that speaker gaze behavior during questions is similar across the three cultures, while recipient gaze behavior differs. Moreover, they find that questions are usually responded to and the ones that do not elicit response are typically produced with the recipient not looking at the speaker. Chapter 3 of this dissertation investigates further how gaze can be used to mobilize and pursue a response when the latter is missing. Chapter 4 instead will investigate how participants orient to the relevance of gaze withdrawal at sequence completion, particularly when this coincides with the completion of a course of action (for other means to project possible completion of a sequence see, e.g., Mondada, 2006). In prior work, gaze withdrawal was described as a function of reduced engagement with a conversation (e.g. Goodwin, 1981, 1984), yet the suggestion here is that the level at which gaze works is more fine-grained than previously described. In their work on assessments, for example, Goodwin and Goodwin (1987) argue that gaze withdrawal is a way of displaying diminished participation in the activity, and discuss assessments as a resource for closing topics and mutual orientation. Interestingly, though, in all examples shown in that paper, gaze withdrawal is not followed by immediate topic closure but rather by further talk (at least an additional TCU) either by the participant who withdraws gaze (if s/he was the speaker) or by the current speaker (if the person withdrawing gaze is the recipient). As will be shown in chapter 4, gaze withdrawal is indeed a resource for making a bid for closure, or for displaying a specific understanding of the ongoing development of the course of action. By bidding for closure, participants display diminished participation in the activity. In this section we have seen the claims about the regulatory functions of gaze being organized with respect to turn-taking, yet this dissertation will challenge those claims by suggesting that gaze behavior is organized with respect to the sequential organization of courses of action. Moreover, claims about the role of gaze in soliciting a response will be partly confirmed and partly refined, suggesting that this is the case in specific sequential environments, and not just at any point in a conversation. 40 40 One of the main conclusions that Bavelas, et al. (2002) draw is that the speaker does not look at the listener to monitor him/her for action but rather to solicit a response. A second conclusion is that listeners display their recognition and understanding of the action performed through gaze by responding “immediately and appropriately”. At the same time, it is unclear in which way the occurrence of a response before the withholding of gaze becomes evidence that listeners are responding specifically because of the gaze. Indeed, because of the coding system adopted for this study, a participant could have been looking for 10 seconds and if s/he looks away after a response, then the response would be considered related to the sustaining of the gaze, while the actual talk is not taken into account. A further problem associated with this study is that other communicative behaviors to solicit a response were not taken into account, and, as such, the specific actions performed through talk ignored. This suggests that the claim of a relationship between speaker gaze and listener’s response needs further specification, which has been provided, at least to some extent, by other researchers in more recent studies, to which I now turn. In a paper focusing on the resources that participants deploy to mobilize a response by a recipient, Stivers and Rossano (2010) identify speaker gaze as one such resource. They show that in face-to-face interactions, when the speaker is gazing at the recipient during an assessment or other non-canonical actions, the recipient usually responds to the initial assessment, while assessments that are not responded to are usually produced without speaker gaze or other response mobilizing features. The claim is that speaker gaze plays a role in mobilizing recipient response. Additional evidence for this claim comes from my work on question-answer sequences in Italian dyadic conversations (Rossano, 2010). I focus on the resources that speakers deploy to indicate to recipients that they are producing a question (and, therefore, that they are pressuring recipients for some response). I find that when speakers produce polar (yes/no) questions that do not have a prototypical interrogative intonational contour (i.e. a rising intonation), they are significantly more likely to look toward their recipient than when they are producing questions with prototypical interrogative contours. Given that polar questions in Italian can be marked intonationally but not morpho-syntactically, the occurrence of speaker gaze toward the recipient appears to work as an additional cue that the turn should be responded to. Additionally, in work focusing on the timing of responses to yes-no questions in multiple languages, Stivers, et al. (2009) show that the occurrence of speaker gaze correlates with faster responses to participant questions. Although they do not take into account recipient gaze, and the number of participants involved in the interactions varies, the finding is nevertheless supported in nine out of the ten 41typologically diverse languages investigated. Another recent cross-linguistic study that takes the sequential environment of talk and gaze into account provides additional evidence for the existence of a relationship between the occurrence of speaker gaze and the expectation and timing of a response. In their work comparing gaze behavior during questions in three unrelated languages and cultures mentioned earlier, Rossano, et al. (2009) show that, on average, speakers look at recipients during questions in 73% of cases. They also show that speaker gaze behavior during questions is similar across the three cultures, while recipient gaze behavior differs. Moreover, they find that questions are usually responded to and the ones that do not elicit response are typically produced with the recipient not looking at the speaker. Chapter 3 of this dissertation investigates further how gaze can be used to mobilize and pursue a response when the latter is missing. Chapter 4 instead will investigate how participants orient to the relevance of gaze withdrawal at sequence completion, particularly when this coincides with the completion of a course of action (for other means to project possible completion of a sequence see, e.g., Mondada, 2006). In prior work, gaze withdrawal was described as a function of reduced engagement with a conversation (e.g. Goodwin, 1981, 1984), yet the suggestion here is that the level at which gaze works is more fine-grained than previously described. In their work on assessments, for example, Goodwin and Goodwin (1987) argue that gaze withdrawal is a way of displaying diminished participation in the activity, and discuss assessments as a resource for closing topics and mutual orientation. Interestingly, though, in all examples shown in that paper, gaze withdrawal is not followed by immediate topic closure but rather by further talk (at least an additional TCU) either by the participant who withdraws gaze (if s/he was the speaker) or by the current speaker (if the person withdrawing gaze is the recipient). As will be shown in chapter 4, gaze withdrawal is indeed a resource for making a bid for closure, or for displaying a specific understanding of the ongoing development of the course of action. By bidding for closure, participants display diminished participation in the activity. In this section we have seen the claims about the regulatory functions of gaze being organized with respect to turn-taking, yet this dissertation will challenge those claims by suggesting that gaze behavior is organized with respect to the sequential organization of courses of action. Moreover, claims about the role of gaze in soliciting a response will be partly confirmed and partly refined, suggesting that this is the case in specific sequential environments, and not just at any point in a conversation. 41 42 1.4.3 Gaze in Action Formation and as a Social Act A third line of research, largely developed within the last few years, and for the most part undertaken by Kidwell, has addressed the role of gaze in implementing social actions, depending on its context and delivery. Kidwell’s work focuses, for example, on the recognizability of participants’ looking practices in interaction, particularly in a childcare setting and in interactions with young infants. Kidwell (2005) investigates how very young children can differentiate two practices of looking by caregivers, termed a mere look and the look. While a mere look tends to be of short duration, alights briefly on a target and is usually produced concurrently with other activities that the caregiver is involved in, the look is relatively longer, is fixated on a target and is produced as a discrete activity. Kidwell finds that young children treat these two practices of looking differently: whereas they continue to engage in whatever they were doing when a caregiver directs a mere look at them, those same children treat the look as prefiguring sanctioning. As such, after a caregiver’s ‘look’, children tend to inspect their own actions for its source (i.e. sanctionable behavior or activity such as harassing other children), and this often leads to a disruption of that behavior. In a related study, Kidwell (2009) shows how, in the context of children harassing other children, a gaze shift by the harassed child may be interpreted alternatively as “looking to” the caregiver as an appeal for her assistance, as “noticing” that the caregiver is approaching or as “searching” her out. The looking to gaze shift is “one in which the peer, when he or she shifts her gaze, continues her concurrent activities, readily alights and holds on a target, and directs action to the target” (2009: 150). The “noticing” gaze shift is “one in which the peer halts concurrent activities, readily alights and holds on a target, but does not produce action for the target” (2009: 153). Finally, the “search” gaze shift is one in which “the peer may stop or continue a concurrent activity, does not readily alight and hold on a target and makes appeals for assistance” (2009: 156). Thus, each type of gaze implicates a distinct social action that makes different responses relevant next. Focusing this time on gaze withdrawal, Kidwell (2006) shows how it can be differentially interpreted depending on whether it occurs during a conversation or when it is responsive to an imperative to comply with a directive action (e.g. an order) such as the ones a police officer might produce during his line of duty. Gaze withdrawal in this case can be taken as an act of resistance, and is thus dispreferred. Evidence is provided in that mutual gaze is pursued via both embedded methods (e.g. speech cut offs, gaze-tracking and embodied summonses) or via exposed methods (e.g. verbal commands to Listen and Look). 43Finally, Haddington’s work (2006) focuses on how gaze can be used, in relation to the production of assessments, to display stances toward what has been assessed. He presents three types of gaze behavior (looking together at an assessable, mutual gaze during an agreeing second stance and cut-off gaze during actions that display divergent stances) and describes how they achieve stance-taking by interacting with what the participants are saying and doing during the interaction. While this work on gaze and its relationship to social action is relatively new, research has sometimes touched on the importance of mutual gaze or eye contact in conversation in ad hoc situations, while focusing on other interactional practices. For example, Egbert (1996) suggests that in German, the use of the repair initiator Bitte? is highly context sensitive. Germans tend to use Bitte? when there is no mutual gaze between the participants (including situations in which eye contact is impossible, such as phone conversations), while they tend to use other repair initiators such as Was? when they are co-present and have mutual visual access. Sidnell (2006) also touches on the relevance of gaze in his investigation of how recipients identify and distinguish re-enactments from the main reporting of facts during a telling. In particular, he shows that during re-enactments speakers tend to look away from the addressee and keep their eyes away from the audience during the re-enactment. By doing this, the speaker shows that s/he is doing something different than addressing the other participant (cf. Goodwin, 1979), and indeed s/he is doing a re-enactment. Finally, in an examination of interviewee laughter in broadcast news interviews, Romaniuk (2009) suggests that interviewers withdraw their gaze when interviewees are laughing in such a way as to systematically terminate the relevance of responsive laughter. This section has described some recent studies that focus on how gaze and other embodied displays can be used by themselves to accomplish social actions, and also presented research that shows how gaze behaviors are constitutive of social actions, even ones performed mainly through talk. More generally, in agreement with Goodwin’s statement (1981: 30) that gaze “is not simply a means of obtaining information, the receiving end of a communication system, but is itself a social act”, this dissertation will focus on the systematic organization and deployment of eye gaze in social interaction. However, contrary to previous work, this work will show that gaze in interaction is not organized primarily by reference to turns-at-talk. Gaze behavior is mainly organized in relation to sequences of talk and the development of courses of action, to be subsumed under the general umbrella of “ongoing projects”, so that most of the variation in gaze direction should be observed at the beginning or at possible completion points of courses of action accomplished through one or more 42 42 1.4.3 Gaze in Action Formation and as a Social Act A third line of research, largely developed within the last few years, and for the most part undertaken by Kidwell, has addressed the role of gaze in implementing social actions, depending on its context and delivery. Kidwell’s work focuses, for example, on the recognizability of participants’ looking practices in interaction, particularly in a childcare setting and in interactions with young infants. Kidwell (2005) investigates how very young children can differentiate two practices of looking by caregivers, termed a mere look and the look. While a mere look tends to be of short duration, alights briefly on a target and is usually produced concurrently with other activities that the caregiver is involved in, the look is relatively longer, is fixated on a target and is produced as a discrete activity. Kidwell finds that young children treat these two practices of looking differently: whereas they continue to engage in whatever they were doing when a caregiver directs a mere look at them, those same children treat the look as prefiguring sanctioning. As such, after a caregiver’s ‘look’, children tend to inspect their own actions for its source (i.e. sanctionable behavior or activity such as harassing other children), and this often leads to a disruption of that behavior. In a related study, Kidwell (2009) shows how, in the context of children harassing other children, a gaze shift by the harassed child may be interpreted alternatively as “looking to” the caregiver as an appeal for her assistance, as “noticing” that the caregiver is approaching or as “searching” her out. The looking to gaze shift is “one in which the peer, when he or she shifts her gaze, continues her concurrent activities, readily alights and holds on a target, and directs action to the target” (2009: 150). The “noticing” gaze shift is “one in which the peer halts concurrent activities, readily alights and holds on a target, but does not produce action for the target” (2009: 153). Finally, the “search” gaze shift is one in which “the peer may stop or continue a concurrent activity, does not readily alight and hold on a target and makes appeals for assistance” (2009: 156). Thus, each type of gaze implicates a distinct social action that makes different responses relevant next. Focusing this time on gaze withdrawal, Kidwell (2006) shows how it can be differentially interpreted depending on whether it occurs during a conversation or when it is responsive to an imperative to comply with a directive action (e.g. an order) such as the ones a police officer might produce during his line of duty. Gaze withdrawal in this case can be taken as an act of resistance, and is thus dispreferred. Evidence is provided in that mutual gaze is pursued via both embedded methods (e.g. speech cut offs, gaze-tracking and embodied summonses) or via exposed methods (e.g. verbal commands to Listen and Look). 43Finally, Haddington’s work (2006) focuses on how gaze can be used, in relation to the production of assessments, to display stances toward what has been assessed. He presents three types of gaze behavior (looking together at an assessable, mutual gaze during an agreeing second stance and cut-off gaze during actions that display divergent stances) and describes how they achieve stance-taking by interacting with what the participants are saying and doing during the interaction. While this work on gaze and its relationship to social action is relatively new, research has sometimes touched on the importance of mutual gaze or eye contact in conversation in ad hoc situations, while focusing on other interactional practices. For example, Egbert (1996) suggests that in German, the use of the repair initiator Bitte? is highly context sensitive. Germans tend to use Bitte? when there is no mutual gaze between the participants (including situations in which eye contact is impossible, such as phone conversations), while they tend to use other repair initiators such as Was? when they are co-present and have mutual visual access. Sidnell (2006) also touches on the relevance of gaze in his investigation of how recipients identify and distinguish re-enactments from the main reporting of facts during a telling. In particular, he shows that during re-enactments speakers tend to look away from the addressee and keep their eyes away from the audience during the re-enactment. By doing this, the speaker shows that s/he is doing something different than addressing the other participant (cf. Goodwin, 1979), and indeed s/he is doing a re-enactment. Finally, in an examination of interviewee laughter in broadcast news interviews, Romaniuk (2009) suggests that interviewers withdraw their gaze when interviewees are laughing in such a way as to systematically terminate the relevance of responsive laughter. This section has described some recent studies that focus on how gaze and other embodied displays can be used by themselves to accomplish social actions, and also presented research that shows how gaze behaviors are constitutive of social actions, even ones performed mainly through talk. More generally, in agreement with Goodwin’s statement (1981: 30) that gaze “is not simply a means of obtaining information, the receiving end of a communication system, but is itself a social act”, this dissertation will focus on the systematic organization and deployment of eye gaze in social interaction. However, contrary to previous work, this work will show that gaze in interaction is not organized primarily by reference to turns-at-talk. Gaze behavior is mainly organized in relation to sequences of talk and the development of courses of action, to be subsumed under the general umbrella of “ongoing projects”, so that most of the variation in gaze direction should be observed at the beginning or at possible completion points of courses of action accomplished through one or more 43 44 sequences of talk. Note here that with the terms “courses of action accomplished through one or more sequences of talk” I refer to the fact that, for the interactional project to be considered completed, most actions require at least the occurrence of some sort of response or reaction by the other participant, and therefore an initiating action usually starts the development of a course of action produced by more than one participant. For the project to be complete and considered accomplished, more things have to happen. For example, a request for information, the action accomplished for example by a turn at talk, can be considered accomplished only if the other participant provides the information, and an offer is completed only if it is accepted or rejected and the thing offered is provided to the person to whom it had been offered. This means that the occurrence of an utterance and the action(s) that it implements opens the possibility of (and sometimes it normatively expects) the occurrence of another set of utterances or actions that would allow the initiating action to reach socially appropriate completion. This can be achieved in two turns or may require larger structures and therefore more than one sequence, though the participants’ orientation toward completing the gist of the initial action remains the same. More details concerning previous claims about gaze behavior in social interaction and how my position differs or aligns with those claims will be further outlined in the introduction of each following chapter. 451.5 Data and Method In order to analyze and comprehend how participants approach the boundaries of sequences of talk and actions, I recorded a series of videos of naturally occurring interactions, interactions in which people are having dinner together, playing cards, preparing food, having coffee or traveling together in a car, without providing them with any kind of script or instructions for the interactions that later occurred and without asking them to get together just for the recording. The recordings, then, were of interactions that would have naturally occurred independently of the presence of my camcorders and of my request to tape them and therefore their occurrence was more spontaneous and less likely to be staged. Most of these interactions were dyadic, though some triadic interactions have been used as well to develop basic observations about the organization of gaze behavior. Details about the participants, the situation, the location and the main activity performed during each interaction recorded are presented in Appendix A. Table 1.1 shows the amount of data collected, transcribed, annotated and analyzed for the work reported in this dissertation. Table 1.1. Amount of data collected, transcribed, annotated and analyzed. What How Many Data collected Approx. 18 hours Data transcribed Approx. 5 hours and 30 minutes Data fully annotated15 Approx. 3 hours and 30 minutes Data analyzed qualitatively Approx. 12 hours and 30 minutes Data analyzed quantitatively Approx. 1 hour and 40 minutes Total number of participants in data analyzed 28 Using this kind of data is crucial for the kind of investigation that I wanted to develop. Only by looking at what people really do in interaction was it possible to observe some recurrent practices for doing things with gaze. The need for this qualitative analysis, already emphasized in the quote by Lorenz reported above, is strongly related to something often observed by conversation analysts and remarked 40 years ago by Harvey Sacks (1992: 240, 15 A large part of the data has been annotated for any shift in eye direction, for any shift in head direction and any postural shift and any nod by each participant. Multiple other annotations have been added with respect to the organization of the talk and social actions in those conversations, in relation to the specific practices under investigation, as it will be outlined in the following chapters. 44 44 sequences of talk. Note here that with the terms “courses of action accomplished through one or more sequences of talk” I refer to the fact that, for the interactional project to be considered completed, most actions require at least the occurrence of some sort of response or reaction by the other participant, and therefore an initiating action usually starts the development of a course of action produced by more than one participant. For the project to be complete and considered accomplished, more things have to happen. For example, a request for information, the action accomplished for example by a turn at talk, can be considered accomplished only if the other participant provides the information, and an offer is completed only if it is accepted or rejected and the thing offered is provided to the person to whom it had been offered. This means that the occurrence of an utterance and the action(s) that it implements opens the possibility of (and sometimes it normatively expects) the occurrence of another set of utterances or actions that would allow the initiating action to reach socially appropriate completion. This can be achieved in two turns or may require larger structures and therefore more than one sequence, though the participants’ orientation toward completing the gist of the initial action remains the same. More details concerning previous claims about gaze behavior in social interaction and how my position differs or aligns with those claims will be further outlined in the introduction of each following chapter. 451.5 Data and Method In order to analyze and comprehend how participants approach the boundaries of sequences of talk and actions, I recorded a series of videos of naturally occurring interactions, interactions in which people are having dinner together, playing cards, preparing food, having coffee or traveling together in a car, without providing them with any kind of script or instructions for the interactions that later occurred and without asking them to get together just for the recording. The recordings, then, were of interactions that would have naturally occurred independently of the presence of my camcorders and of my request to tape them and therefore their occurrence was more spontaneous and less likely to be staged. Most of these interactions were dyadic, though some triadic interactions have been used as well to develop basic observations about the organization of gaze behavior. Details about the participants, the situation, the location and the main activity performed during each interaction recorded are presented in Appendix A. Table 1.1 shows the amount of data collected, transcribed, annotated and analyzed for the work reported in this dissertation. Table 1.1. Amount of data collected, transcribed, annotated and analyzed. What How Many Data collected Approx. 18 hours Data transcribed Approx. 5 hours and 30 minutes Data fully annotated15 Approx. 3 hours and 30 minutes Data analyzed qualitatively Approx. 12 hours and 30 minutes Data analyzed quantitatively Approx. 1 hour and 40 minutes Total number of participants in data analyzed 28 Using this kind of data is crucial for the kind of investigation that I wanted to develop. Only by looking at what people really do in interaction was it possible to observe some recurrent practices for doing things with gaze. The need for this qualitative analysis, already emphasized in the quote by Lorenz reported above, is strongly related to something often observed by conversation analysts and remarked 40 years ago by Harvey Sacks (1992: 240, 15 A large part of the data has been annotated for any shift in eye direction, for any shift in head direction and any postural shift and any nod by each participant. Multiple other annotations have been added with respect to the organization of the talk and social actions in those conversations, in relation to the specific practices under investigation, as it will be outlined in the following chapters. 45 46 Vol. II): “a base for using close looking at the world for theorizing about it is that from close looking at the world you can find things that we couldn’t, by imagination, assert were there.” Given the focus on the systematic organization of gaze behavior in relation to social action, mainly performed through talk-in-interaction, I needed a methodology that would allow me to dissect and identify the conversational structures independently of what my working hypotheses concerning gaze were. Conversation Analysis (henceforth CA) provided this framework. This methodology focuses on action and interaction rather than on language and linguistic categories. CA allows a researcher to enter the granularity of interactions and to analyze details without losing touch with such macro structures as “sequence organization” and the “overall structural organization” of the whole event. Moreover, the focus on participants’ orientations and displays of understanding of the actions of the other co-present participants, prevents an ad hoc use of categorical labels and analysis that cannot be supported by what can be tracked back in the data. The “proof procedure” of participants’ uptake, central to the entire conversation analytic enterprise, has been fully adopted in this work: I will not simply describe when people move their eyes, but try to document what kind of interactional consequences a shift in gaze orientation can reliably produce. I will be using Conversation Analysis because it aims to investigate empirically how talk becomes action, how action is structurally organized through talk and how participants understand the sense of what they are doing while they interact. However, the data reported here have not been analyzed only qualitatively. Insights from other disciplines will also be used in the general analysis of the data. As previously stated, I started from a qualitative analysis of single stretches of interactions, comparing ones in which some specific practice appeared to be deployed. After outlining the basic interactional features of each practice, I went back to the data, coded systematically what was happening in a specific amount of minutes (usually 10 minutes, as is indicated in each chapter) randomly picked from each interaction. The coding was done on the variables that had been previously identified as crucial for the practice investigated. I then ran statistical analysis of the coded data (mainly logistic regressions)16 to identify significant correlations between variables, most prominent predictors of specific outcomes, and more generally to 16 Logistic regression is a statistical model used to predict outcome variables that are categorical (e.g. yes or no). In this case, if a participation role (e.g. being a speaker) predicts significantly better than the other (being a recipient) whether an individual will be looking at the other participant, then we can say that being a speaker rather than a recipient significantly affects the likelihood of the occurrence of gaze toward the other participant. In other words, participants look at each other differently depending on which participation role they have during the social action under investigation. 47identify and quantify the systematicity of a practice by trying to falsify the empirical hypothesis concerning the function that a specific gaze practice might have, hypothesis developed while qualitatively analyzing the participants’ gaze in my corpus (Popper, 1959 [1934]). In other words, I tried to put to practice the method indicated by Darwin (1872) and further summarized by Lorenz (1966) as “a biological approach to behaviour” in the quote reported above. The use of a statistical approach fully informed by CA findings is not alien to recent CA research. As Heritage and Maynard (2006a) put it: To extract robust outcome-based conclusions about how physicians (or patients) should conduct themselves in specific moments in the flow of the medical encounter, it is important to find a meeting point between the two methodologies of coding and microanalysis […] In other words, beyond the intrinsic worth of an analytical framework responsive to very granular, individual moments in the physician-patient encounter, we need one that simultaneously supports coding at a broader level of granularity sufficient to reach beyond individual cases to generate findings at a statistical evidential standard.” (Heritage & Maynard, 2006a: 8) In recent years, several CA works usually in institutional settings, have looked at an interactional phenomenon and used quantitative methods to assess the association between that phenomenon and some exogenous variable whether that be time (e.g. in Clayman, Heritage, Elliott, & McDonald, 2007), prescribing outcomes/provider perception (e.g. in Mangione-Smith, Stivers, Elliott, McDonald, & Heritage, 2003) or race and class (e.g. in Stivers & Majid, 2007). One reason for adopting a statistical approach was to understand whether gaze practices could be systematically identified across situations and individuals and to see whether the predictions about their generality, using qualitative methods, would survive an attempt to put them to test, in particular once applied to a larger number of instances, across different speakers and recipients, where the actions and context are in some sense comparable. Further details about how comparable actions and contexts have been identified for quantification purposes will be reported in each chapter. To annotate the details of talk and visible behavior from each interaction, I used the software ELAN (http://www.lat-mpi.eu/tools/elan/), developed at the Max Planck Institute for Psycholinguistics to allow researchers to view and simultaneously annotate multiple recordings (video and audio) of the same interaction (see Fig. 1.6 below). 46 46 Vol. II): “a base for using close looking at the world for theorizing about it is that from close looking at the world you can find things that we couldn’t, by imagination, assert were there.” Given the focus on the systematic organization of gaze behavior in relation to social action, mainly performed through talk-in-interaction, I needed a methodology that would allow me to dissect and identify the conversational structures independently of what my working hypotheses concerning gaze were. Conversation Analysis (henceforth CA) provided this framework. This methodology focuses on action and interaction rather than on language and linguistic categories. CA allows a researcher to enter the granularity of interactions and to analyze details without losing touch with such macro structures as “sequence organization” and the “overall structural organization” of the whole event. Moreover, the focus on participants’ orientations and displays of understanding of the actions of the other co-present participants, prevents an ad hoc use of categorical labels and analysis that cannot be supported by what can be tracked back in the data. The “proof procedure” of participants’ uptake, central to the entire conversation analytic enterprise, has been fully adopted in this work: I will not simply describe when people move their eyes, but try to document what kind of interactional consequences a shift in gaze orientation can reliably produce. I will be using Conversation Analysis because it aims to investigate empirically how talk becomes action, how action is structurally organized through talk and how participants understand the sense of what they are doing while they interact. However, the data reported here have not been analyzed only qualitatively. Insights from other disciplines will also be used in the general analysis of the data. As previously stated, I started from a qualitative analysis of single stretches of interactions, comparing ones in which some specific practice appeared to be deployed. After outlining the basic interactional features of each practice, I went back to the data, coded systematically what was happening in a specific amount of minutes (usually 10 minutes, as is indicated in each chapter) randomly picked from each interaction. The coding was done on the variables that had been previously identified as crucial for the practice investigated. I then ran statistical analysis of the coded data (mainly logistic regressions)16 to identify significant correlations between variables, most prominent predictors of specific outcomes, and more generally to 16 Logistic regression is a statistical model used to predict outcome variables that are categorical (e.g. yes or no). In this case, if a participation role (e.g. being a speaker) predicts significantly better than the other (being a recipient) whether an individual will be looking at the other participant, then we can say that being a speaker rather than a recipient significantly affects the likelihood of the occurrence of gaze toward the other participant. In other words, participants look at each other differently depending on which participation role they have during the social action under investigation. 47identify and quantify the systematicity of a practice by trying to falsify the empirical hypothesis concerning the function that a specific gaze practice might have, hypothesis developed while qualitatively analyzing the participants’ gaze in my corpus (Popper, 1959 [1934]). In other words, I tried to put to practice the method indicated by Darwin (1872) and further summarized by Lorenz (1966) as “a biological approach to behaviour” in the quote reported above. The use of a statistical approach fully informed by CA findings is not alien to recent CA research. As Heritage and Maynard (2006a) put it: To extract robust outcome-based conclusions about how physicians (or patients) should conduct themselves in specific moments in the flow of the medical encounter, it is important to find a meeting point between the two methodologies of coding and microanalysis […] In other words, beyond the intrinsic worth of an analytical framework responsive to very granular, individual moments in the physician-patient encounter, we need one that simultaneously supports coding at a broader level of granularity sufficient to reach beyond individual cases to generate findings at a statistical evidential standard.” (Heritage & Maynard, 2006a: 8) In recent years, several CA works usually in institutional settings, have looked at an interactional phenomenon and used quantitative methods to assess the association between that phenomenon and some exogenous variable whether that be time (e.g. in Clayman, Heritage, Elliott, & McDonald, 2007), prescribing outcomes/provider perception (e.g. in Mangione-Smith, Stivers, Elliott, McDonald, & Heritage, 2003) or race and class (e.g. in Stivers & Majid, 2007). One reason for adopting a statistical approach was to understand whether gaze practices could be systematically identified across situations and individuals and to see whether the predictions about their generality, using qualitative methods, would survive an attempt to put them to test, in particular once applied to a larger number of instances, across different speakers and recipients, where the actions and context are in some sense comparable. Further details about how comparable actions and contexts have been identified for quantification purposes will be reported in each chapter. To annotate the details of talk and visible behavior from each interaction, I used the software ELAN (http://www.lat-mpi.eu/tools/elan/), developed at the Max Planck Institute for Psycholinguistics to allow researchers to view and simultaneously annotate multiple recordings (video and audio) of the same interaction (see Fig. 1.6 below). 47 48 Fig. 1.6 Screenshot of recordings of an interaction annotated with ELAN The playback and search functions proved extremely functional both for qualitative analysis and for the analysis of a large database. The layout of the software, that shows the annotations on a time scale, facilitates the identification of silences and overlaps. When this is combined with annotations in multiple layers, it allows the researcher to immediately identify whether specific visible behaviors were deployed, e.g., before or during the occurrence of a silence. Using the playback functions then the researcher can control for the correctness of the observation, as it is then possible to play just the specific bit of interaction under examination and verify in more than one video what happened exactly there. This software allows independent coding for any category we might be interested in, but at the same time it shows its role in context, as other annotations and layers can be visible at the same time. For the statistical analysis I used the software SPSS and the software STATA. 491.5.1 Transcription System The transcript of example 1.1 contains features that I will now explain. Talk is transcribed using the Jeffersonian system (Atkinson & Heritage, 1984; Jefferson, 2004a) usually adopted by Conversation Analysis. Given the use of Italian data, I adopted a 3 lines type of transcript, in which the first line is the text in Italian (the language in which the interactions occurred), the second line is a word by word translation from Italian to English and the third line is an English ‘free translation’, meant to convey the main gist of the utterance, both semantically and style wise. In addition to the content and style of the talk, additional symbols are present in those transcripts and they represent a new transcription system for eye-gaze in face-to-face interaction that I developed and that can be found in Appendix B. A published example of it can be found in Rossano et al. (2009). It differs from the previous transcription system used within Conversation Analysis and invented by Goodwin (e.g., 1979, 1980, 1981). In what follows I outline some of the basic features of my system and some of the differences between Goodwin’s and mine. 1. It is ideal for dyadic interaction, though it could be used for multi-party conversations as well. 2. It is more iconic, through the use of ovals representing faces and arrows representing gaze direction, than any other system currently used. 3. It is combinatorial. Each participant’s gaze behavior can vary independently from the gaze behavior of the other. The number of possible symbols depends on the number of individual gaze orientation we deem worth distinguishing. In other words, the system can easily be increased if more features are considered worth reporting in a transcript. 4. It does not simply report whether a participant is looking at the other’s face or not and whether there is a gaze shift. It adds many additional gaze configurations (e.g., looking at an object, closing eyes, looking away, looking up or down, etc.) that are interactionally relevant and meaningfully different from each other. 5. By using the directionality of arrows, it partly communicates not just the direction of the eyes but also head movements (for example in the symbols 48 48 Fig. 1.6 Screenshot of recordings of an interaction annotated with ELAN The playback and search functions proved extremely functional both for qualitative analysis and for the analysis of a large database. The layout of the software, that shows the annotations on a time scale, facilitates the identification of silences and overlaps. When this is combined with annotations in multiple layers, it allows the researcher to immediately identify whether specific visible behaviors were deployed, e.g., before or during the occurrence of a silence. Using the playback functions then the researcher can control for the correctness of the observation, as it is then possible to play just the specific bit of interaction under examination and verify in more than one video what happened exactly there. This software allows independent coding for any category we might be interested in, but at the same time it shows its role in context, as other annotations and layers can be visible at the same time. For the statistical analysis I used the software SPSS and the software STATA. 491.5.1 Transcription System The transcript of example 1.1 contains features that I will now explain. Talk is transcribed using the Jeffersonian system (Atkinson & Heritage, 1984; Jefferson, 2004a) usually adopted by Conversation Analysis. Given the use of Italian data, I adopted a 3 lines type of transcript, in which the first line is the text in Italian (the language in which the interactions occurred), the second line is a word by word translation from Italian to English and the third line is an English ‘free translation’, meant to convey the main gist of the utterance, both semantically and style wise. In addition to the content and style of the talk, additional symbols are present in those transcripts and they represent a new transcription system for eye-gaze in face-to-face interaction that I developed and that can be found in Appendix B. A published example of it can be found in Rossano et al. (2009). It differs from the previous transcription system used within Conversation Analysis and invented by Goodwin (e.g., 1979, 1980, 1981). In what follows I outline some of the basic features of my system and some of the differences between Goodwin’s and mine. 1. It is ideal for dyadic interaction, though it could be used for multi-party conversations as well. 2. It is more iconic, through the use of ovals representing faces and arrows representing gaze direction, than any other system currently used. 3. It is combinatorial. Each participant’s gaze behavior can vary independently from the gaze behavior of the other. The number of possible symbols depends on the number of individual gaze orientation we deem worth distinguishing. In other words, the system can easily be increased if more features are considered worth reporting in a transcript. 4. It does not simply report whether a participant is looking at the other’s face or not and whether there is a gaze shift. It adds many additional gaze configurations (e.g., looking at an object, closing eyes, looking away, looking up or down, etc.) that are interactionally relevant and meaningfully different from each other. 5. By using the directionality of arrows, it partly communicates not just the direction of the eyes but also head movements (for example in the symbols 49 50 indicating gaze shifts, like turning or raising eyes up, or in symbols such as looking away or looking down). 6. It does not represent individuals’ gaze behavior as separate from the gaze of the other participant. By having both participants’ gaze behavior represented in the same symbol, the system claims that the combination of both participants’ gaze orientation is an interactionally significantly different configuration from any other. So if the speaker looks at the recipient but the latter has her eyes closed, this configuration has very different interactional consequences than a configuration in which they are both looking at each other. Facial expressions, or more simply a direct stare toward the recipient will not be perceivable by the latter and as such will affect what the speaker might expect from the other participants in terms of uptake. This also means, for example, that if the gaze configurations change during a silence, then that silence could be decomposed into different phases, to take into account the different potential moves that could occur at each point and to record the occurrence of specific interactional moves performed through gaze (chapter 3 will show examples of gaze pursuits during silence, gaze movements that perform social actions). 7. By being located on top of brackets that have a beginning and end point (either on top of talk, or on symbols indicating the length of silence), each symbol indicates the stable gaze configuration in place during that stretch of talk or silence. The claim is that the length of a specific configuration might have interactional consequences (e.g., looking at each other during silence for an extended amount of time). As far as gaze is concerned, what matters is not just position and composition of the signal (Schegloff, 2007b) but also duration. This observation matches the “typical intensity” of a communicative signal as outlined in Morris (1957). Moreover, by indicating when it starts and when it ends, we can have a better grasp of the possible impact it might have on what gets said or done before or after that shift, knowing for example that the shift occurs on top of a specific word or after a specific amount of silence (see chapter 3 for a reflection on the length of silence and occurrence of pursuits). 518. The transcription system takes its metric from the talk, given that the focus of this work is to see the systematicity of gaze behavior with respect to social actions in conversation. The meaning of each gaze symbol is described in Appendix B. 1.6 Structure of the Dissertation This dissertation investigates how participants use their eyes at the beginning, in the middle and at the end of sequences of talk in interaction. Each empirical chapter specifically investigates one of these three contexts. Chapter 2 investigates how gaze behavior is deployed at the beginning of specific conversational units such as extended tellings and adjacency pair based sequences. I show how recipients use gaze to systematically differentiate between the two conversational environments. The chapter introduces the relevance of investigating the structure of social actions performed through talk to better understand how gaze behavior is deployed during a conversation. Moreover, it investigates how recipients manage to enact the practice reported here so quickly and so reliably. Chapter 3 investigates how speakers behave in the middle of a sequence of talk and how they can use gaze to pressure for responses. I focus on situations in which speakers who have produced a sequence initiating action that did not obtain a response pursue a response not simply verbally but also through using gaze toward the recipient. I first show how speaker and recipient gaze is deployed during sequence-initiating actions and how it affects the occurrence and the timing of a response. Then I move to instances in which a response appears to be missing and it is pursued through gaze. Finally I show how the practice of soliciting response/uptake via gaze can be deployed in contexts other than a transition relevance place. Chapter 4 investigates how people close conversational units such as courses of action and sequences of talk. In particular, it shows how withdrawing gaze from the other participant approaching sequence possible completion can work as a bid for closure and if aligned by the other participant can lead to the actual closure of a sequence. Sustaining gaze toward the other person when a possible completion of the sequence is reached, indicates the intention to expand or the understanding that the sequence is not complete as such and would need to be expanded. 50 50 indicating gaze shifts, like turning or raising eyes up, or in symbols such as looking away or looking down). 6. It does not represent individuals’ gaze behavior as separate from the gaze of the other participant. By having both participants’ gaze behavior represented in the same symbol, the system claims that the combination of both participants’ gaze orientation is an interactionally significantly different configuration from any other. So if the speaker looks at the recipient but the latter has her eyes closed, this configuration has very different interactional consequences than a configuration in which they are both looking at each other. Facial expressions, or more simply a direct stare toward the recipient will not be perceivable by the latter and as such will affect what the speaker might expect from the other participants in terms of uptake. This also means, for example, that if the gaze configurations change during a silence, then that silence could be decomposed into different phases, to take into account the different potential moves that could occur at each point and to record the occurrence of specific interactional moves performed through gaze (chapter 3 will show examples of gaze pursuits during silence, gaze movements that perform social actions). 7. By being located on top of brackets that have a beginning and end point (either on top of talk, or on symbols indicating the length of silence), each symbol indicates the stable gaze configuration in place during that stretch of talk or silence. The claim is that the length of a specific configuration might have interactional consequences (e.g., looking at each other during silence for an extended amount of time). As far as gaze is concerned, what matters is not just position and composition of the signal (Schegloff, 2007b) but also duration. This observation matches the “typical intensity” of a communicative signal as outlined in Morris (1957). Moreover, by indicating when it starts and when it ends, we can have a better grasp of the possible impact it might have on what gets said or done before or after that shift, knowing for example that the shift occurs on top of a specific word or after a specific amount of silence (see chapter 3 for a reflection on the length of silence and occurrence of pursuits). 518. The transcription system takes its metric from the talk, given that the focus of this work is to see the systematicity of gaze behavior with respect to social actions in conversation. The meaning of each gaze symbol is described in Appendix B. 1.6 Structure of the Dissertation This dissertation investigates how participants use their eyes at the beginning, in the middle and at the end of sequences of talk in interaction. Each empirical chapter specifically investigates one of these three contexts. Chapter 2 investigates how gaze behavior is deployed at the beginning of specific conversational units such as extended tellings and adjacency pair based sequences. I show how recipients use gaze to systematically differentiate between the two conversational environments. The chapter introduces the relevance of investigating the structure of social actions performed through talk to better understand how gaze behavior is deployed during a conversation. Moreover, it investigates how recipients manage to enact the practice reported here so quickly and so reliably. Chapter 3 investigates how speakers behave in the middle of a sequence of talk and how they can use gaze to pressure for responses. I focus on situations in which speakers who have produced a sequence initiating action that did not obtain a response pursue a response not simply verbally but also through using gaze toward the recipient. I first show how speaker and recipient gaze is deployed during sequence-initiating actions and how it affects the occurrence and the timing of a response. Then I move to instances in which a response appears to be missing and it is pursued through gaze. Finally I show how the practice of soliciting response/uptake via gaze can be deployed in contexts other than a transition relevance place. Chapter 4 investigates how people close conversational units such as courses of action and sequences of talk. In particular, it shows how withdrawing gaze from the other participant approaching sequence possible completion can work as a bid for closure and if aligned by the other participant can lead to the actual closure of a sequence. Sustaining gaze toward the other person when a possible completion of the sequence is reached, indicates the intention to expand or the understanding that the sequence is not complete as such and would need to be expanded. 51 52 Finally, chapter 5 provides the conclusions of the dissertation. It summarizes the main findings placing them in the broader framing partly introduced in chapter 1. Finally, it discusses a model that describes how gaze behavior in interaction is deployed, what are the driving forces in place and which kind of factors might constitute exceptions to its otherwise predictable and systematic deployment. This dissertation also provides a methodological framework that can be used to extend the findings reported here to related conversational environments or interactions occurring in other cultures or among other species. The findings should provide a useful improvement of our current knowledge about eye movement and gaze behavior during social interaction and may be of particular use for researchers interested in implementations for artificial intelligence or the study of cross-cultural interactions. 53 2 The Organization of Gaze Behavior in Two Different Sequential Environments How we depict any segment of the sequence is related to our conception of the whole. Edward M. Bruner Ethnography as Narrative (1986) 2.1 Introduction When do people look at others and when do they look away? What are they doing by looking at another person and by looking away? How do people decide when to look at and when to look away? In this chapter I present work that contributes to answering these questions. I show that gaze behavior in face-to-face interaction is orderly, specifically in its relationship to the sequential environment of the talk and the activities that the participants are implementing through their turns. In particular, I contrast recipient gaze patterns in two sequential environments: during the first turn constructional unit1 (henceforth TCU) of an extended-telling sequence (e.g., a story telling, a report) and during the first TCU of an adjacency-pair-based sequence (e.g., requests or offers). I first demonstrate that there is a systematic difference between the gaze behavior of a participant listening to the first TCU of an extended-telling sequence (henceforth ETS) and the behavior of the same participant listening to the first TCU of an adjacency-pair-based sequence (henceforth APBS). Whereas recipients regularly look toward speakers as soon as an ETS is projectably underway and sustain gaze toward speakers throughout the telling, they usually do not do so while listening to a first pair part2 turn. This suggests that being the recipient of a telling requires a different type of behavioral participation when compared to listening to a simple question. As such the findings presented here will provide a more fine- 1 Turn constructional units (TCU) are the building blocks of turns at talk. A TCU is the most minimal utterance produced by one participant that, in context, can be considered a comprehensible contribution to the interaction and a complete one. They “can by themselves constitute possibly complete turns and on their possible completion, transition to a next speaker becomes relevant (although not necessarily accomplished) (Schegloff, 1996c: 55). See also § 2.2.4 for further clarification of this concept. 2 A first pair part turn is a turn that makes conditionally relevant the occurrence of a second turn and its absence a noticeable absence. For example, an invitation makes relevant an acceptance or a rejection, a greeting makes relevant the return of the greeting by the other participant (see Schegloff, 2007b for a systematic description of the types of first pair parts deployed in conversation). 52 52 Finally, chapter 5 provides the conclusions of the dissertation. It summarizes the main findings placing them in the broader framing partly introduced in chapter 1. Finally, it discusses a model that describes how gaze behavior in interaction is deployed, what are the driving forces in place and which kind of factors might constitute exceptions to its otherwise predictable and systematic deployment. This dissertation also provides a methodological framework that can be used to extend the findings reported here to related conversational environments or interactions occurring in other cultures or among other species. The findings should provide a useful improvement of our current knowledge about eye movement and gaze behavior during social interaction and may be of particular use for researchers interested in implementations for artificial intelligence or the study of cross-cultural interactions. 53 2 The Organization of Gaze Behavior in Two Different Sequential Environments How we depict any segment of the sequence is related to our conception of the whole. Edward M. Bruner Ethnography as Narrative (1986) 2.1 Introduction When do people look at others and when do they look away? What are they doing by looking at another person and by looking away? How do people decide when to look at and when to look away? In this chapter I present work that contributes to answering these questions. I show that gaze behavior in face-to-face interaction is orderly, specifically in its relationship to the sequential environment of the talk and the activities that the participants are implementing through their turns. In particular, I contrast recipient gaze patterns in two sequential environments: during the first turn constructional unit1 (henceforth TCU) of an extended-telling sequence (e.g., a story telling, a report) and during the first TCU of an adjacency-pair-based sequence (e.g., requests or offers). I first demonstrate that there is a systematic difference between the gaze behavior of a participant listening to the first TCU of an extended-telling sequence (henceforth ETS) and the behavior of the same participant listening to the first TCU of an adjacency-pair-based sequence (henceforth APBS). Whereas recipients regularly look toward speakers as soon as an ETS is projectably underway and sustain gaze toward speakers throughout the telling, they usually do not do so while listening to a first pair part2 turn. This suggests that being the recipient of a telling requires a different type of behavioral participation when compared to listening to a simple question. As such the findings presented here will provide a more fine- 1 Turn constructional units (TCU) are the building blocks of turns at talk. A TCU is the most minimal utterance produced by one participant that, in context, can be considered a comprehensible contribution to the interaction and a complete one. They “can by themselves constitute possibly complete turns and on their possible completion, transition to a next speaker becomes relevant (although not necessarily accomplished) (Schegloff, 1996c: 55). See also § 2.2.4 for further clarification of this concept. 2 A first pair part turn is a turn that makes conditionally relevant the occurrence of a second turn and its absence a noticeable absence. For example, an invitation makes relevant an acceptance or a rejection, a greeting makes relevant the return of the greeting by the other participant (see Schegloff, 2007b for a systematic description of the types of first pair parts deployed in conversation). 5354grained deconstruction of the hearer role than the one outlined by Goffman in Footing (1981a). I then examine the mechanisms that allow recipients to project which type of sequence a particular TCU is in the service of. To do that, I introduce the turn design features that tend to occur at the beginning of these different sequential environments and I propose that these features are crucial for the orderly occurrence of recipient’s orientation toward the speaker. Finally, I suggest some implications of the findings for our understanding of how human cognition works in interaction, and I address the issue of the proper structure of heuristics for human interaction. 2.2 Speaker and Recipient Gaze In this section I first review what existing studies have shown about the difference between speaker and recipient gaze in interactions. I then challenge the general claim about their differences on the basis of a qualitative analysis of dyadic interactions. In order to determine an appropriate way of investigating the level of order of the new patterns identified, I propose further refining the domain of investigation. Finally, I provide an overview of what is known about ETSs in conversation as a way into the comparison of gaze behavior in different sequential environments. 2.2.1 Background It has long been claimed that in dyadic interaction people tend to look at the other participant more when they are listening than when they are speaking (Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Bavelas, Coates, & Johnson, 2002; Duncan & Fiske, 1977; Exline, 1963; C. Goodwin, 1980, 1981; Kendon, 1967, 1973, 1990; Kleinke, 1986; Nielsen, 1962; Rutter, 1984). As said in chapter 1, the first person to note this difference was Gerhard Nielsen in his work on self-confrontation (1962). It was, however, Adam Kendon (1967: 27) who provided a more precise description of the different patterns of speaker and hearer gaze: during listening, p [the speaker] looks at q [the hearer] with fairly long gazes, broken by very brief gazes away, whereas during speaking he alternates between gazes to the recipient and gazes away of more 55equal length, the gazes away being longer than those that occur during listening. (1967: 27) Charles Goodwin suggested that quantitative measures of overall frequencies of gaze “may not be an appropriate way to study the organization of gaze as an activity in its own right” (1981: 94) and relying on a case by case analysis of the data he proposed two rules (1980: 275, 287; 1981: 57; 1984: 230) to account for participants’ gaze behavior: 1. “A speaker should obtain the gaze of his recipient during the course of a turn at talk” 2. “A recipient should be gazing at the speaker when the speaker is gazing at the hearer” According to Goodwin, if the recipient looks most of the time, then the speaker will find her/him gazing back any time the speaker looks toward the recipient. On the other hand, if s/he is not looking at the speaker, the latter has resources (phrasal breaks, pauses, restarting the turn) to solicit and obtain the recipient’s gaze. By proposing these as rules3 Goodwin claims that participants’ gaze behavior is interrelated rather than independent and suggests normative measures to maintain that level of order possibly for every turn of talk, giving Kendon’s observations a normative force. Indeed, if the recipient looks all the time and the speaker glances back and forth then there should be no need to deploy procedures to obtain recipient’s gaze orientation toward the speaker. Therefore Kendon’s gaze patterns should be the default for every turn of talk.4 Kendon’s remarks suggest something intriguing from both an interactional and a semiotic point of view: a careful observation of just the eyes of two participants in interaction would allow us to distinguish who is speaking from who is listening and when they exchange these roles. “Kendon’s gaze patterns” have been apparently replicated and re-stated in all the subsequent studies mentioned above. 3 In a footnote Goodwin (1981: 57) admits that even though he proposes these as rules applying to turns in general, this pattern is not found in every turn of talk. 4 Kendon (1967) does actually indicate that this is a generalization and not every single turn of talk necessarily follows the pattern he is suggesting. The goal here is to take seriously the pattern proposed and at the same time the possibility of deviating from this pattern, only to suggest a different level of organization for gaze behavior that does not treat deviation from the pattern as “deviant behavior”, but rather normative gaze behavior, only limited to a specific sequential environment. 5454grained deconstruction of the hearer role than the one outlined by Goffman in Footing (1981a). I then examine the mechanisms that allow recipients to project which type of sequence a particular TCU is in the service of. To do that, I introduce the turn design features that tend to occur at the beginning of these different sequential environments and I propose that these features are crucial for the orderly occurrence of recipient’s orientation toward the speaker. Finally, I suggest some implications of the findings for our understanding of how human cognition works in interaction, and I address the issue of the proper structure of heuristics for human interaction. 2.2 Speaker and Recipient Gaze In this section I first review what existing studies have shown about the difference between speaker and recipient gaze in interactions. I then challenge the general claim about their differences on the basis of a qualitative analysis of dyadic interactions. In order to determine an appropriate way of investigating the level of order of the new patterns identified, I propose further refining the domain of investigation. Finally, I provide an overview of what is known about ETSs in conversation as a way into the comparison of gaze behavior in different sequential environments. 2.2.1 Background It has long been claimed that in dyadic interaction people tend to look at the other participant more when they are listening than when they are speaking (Argyle & Cook, 1976; Argyle & Dean, 1965; Argyle & Graham, 1976; Bavelas, Coates, & Johnson, 2002; Duncan & Fiske, 1977; Exline, 1963; C. Goodwin, 1980, 1981; Kendon, 1967, 1973, 1990; Kleinke, 1986; Nielsen, 1962; Rutter, 1984). As said in chapter 1, the first person to note this difference was Gerhard Nielsen in his work on self-confrontation (1962). It was, however, Adam Kendon (1967: 27) who provided a more precise description of the different patterns of speaker and hearer gaze: during listening, p [the speaker] looks at q [the hearer] with fairly long gazes, broken by very brief gazes away, whereas during speaking he alternates between gazes to the recipient and gazes away of more 55equal length, the gazes away being longer than those that occur during listening. (1967: 27) Charles Goodwin suggested that quantitative measures of overall frequencies of gaze “may not be an appropriate way to study the organization of gaze as an activity in its own right” (1981: 94) and relying on a case by case analysis of the data he proposed two rules (1980: 275, 287; 1981: 57; 1984: 230) to account for participants’ gaze behavior: 1. “A speaker should obtain the gaze of his recipient during the course of a turn at talk” 2. “A recipient should be gazing at the speaker when the speaker is gazing at the hearer” According to Goodwin, if the recipient looks most of the time, then the speaker will find her/him gazing back any time the speaker looks toward the recipient. On the other hand, if s/he is not looking at the speaker, the latter has resources (phrasal breaks, pauses, restarting the turn) to solicit and obtain the recipient’s gaze. By proposing these as rules3 Goodwin claims that participants’ gaze behavior is interrelated rather than independent and suggests normative measures to maintain that level of order possibly for every turn of talk, giving Kendon’s observations a normative force. Indeed, if the recipient looks all the time and the speaker glances back and forth then there should be no need to deploy procedures to obtain recipient’s gaze orientation toward the speaker. Therefore Kendon’s gaze patterns should be the default for every turn of talk.4 Kendon’s remarks suggest something intriguing from both an interactional and a semiotic point of view: a careful observation of just the eyes of two participants in interaction would allow us to distinguish who is speaking from who is listening and when they exchange these roles. “Kendon’s gaze patterns” have been apparently replicated and re-stated in all the subsequent studies mentioned above. 3 In a footnote Goodwin (1981: 57) admits that even though he proposes these as rules applying to turns in general, this pattern is not found in every turn of talk. 4 Kendon (1967) does actually indicate that this is a generalization and not every single turn of talk necessarily follows the pattern he is suggesting. The goal here is to take seriously the pattern proposed and at the same time the possibility of deviating from this pattern, only to suggest a different level of organization for gaze behavior that does not treat deviation from the pattern as “deviant behavior”, but rather normative gaze behavior, only limited to a specific sequential environment. 5556The literature suggests that there are few doubts about the causality of this phenomenon. Indeed, the general idea is that the different gaze patterns of the participants are due to the fact that somebody is speaking and somebody else is listening and the social roles of speaker and listener associated with them. The claim usually is that the occurrence of talk in a face-to-face interaction leads to the occurrence of these gaze patterns because of a conventionalized way5 of looking associated with a specific interactional role. The evidence for this claim comes from the fact that the participant role (of speaker and listener) is usually taken as the only independent variable and the occurrence of gaze as the dependent one. In other words, a participant would look at the other in a certain way and for a certain amount of time because s/he is speaking or because s/he is listening to the other participant, independently of the actions performed through the talk. Regulatory functions, in particular in relation to turn taking, have been claimed for the deployment of gaze (see chapters 1 and 3 for a review). For example looking toward a recipient or looking away could be used as a cue that a speaker is or is not going to select that person to speak next (see, e.g., Lerner, 2003). This would suggest that gaze also affects the exchange of speaker/hearer roles. As such, a straightforward causal chain that links how gaze is deployed to the simple occurrence of talk should be questioned. This leads to an empirical question: is the order of gaze behavior simply due to the participation role of speaker and listener, or does it depend on the details of what participants in those roles do and have to do in terms of social actions? If gaze is organized in relation to the participation role of speaker and listener, then speakership and recipiency would matter. There should be no difference if any distinction is made within the stretches of talk, in particular in terms of the sequential environment in which they are uttered and the social actions that are performed through their talk. If gaze is organized in relation to social action and what speakers and recipients do in each specific social action, then it should be possible to observe some systematic difference in people’s gaze patterns in relation to what they are doing with their talk. In what follows I test these hypotheses through a systematic analysis of empirical data. 5 Some evidence that specific gaze behavior might be conventionalized comes from recent work (Rossano, Brown, & Levinson, 2009) comparing speaker and recipient gaze behavior during questions in three different cultures that displays that there is some cultural variability, in particular in terms of how recipiency is enacted and sustained. However this work does not distinguish speaker and recipient in general, but only during question turns and therefore focuses on more specific action types than Kendon’s work. 572.2.2 Contrastive Evidence for Kendon’s Pattern A qualitative analysis of twelve hours of recordings of naturally occurring dyadic Italian data suggests caution in embracing Kendon’s gaze patterns as they stand. Although it is true that on average listeners look at speakers more than speakers look at listeners, they do not do so invariably. A systematic analysis of video data shows that the length and the patterns of gaze toward the other participant vary widely during a single conversation. At times a recipient does not look at all at the speaker, and a speaker sometimes holds her/his gaze toward a recipient even when the latter is not looking back or during gaps.6 This means that there could be a substantial amount of time in which Kendon’s patterns would not accurately describe the data and thus undermine prior claims (e.g. by Goodwin) that the recipient looking toward the speaker is a norm-governed phenomenon. In order to show some of the differences between these situations, I will present four extracts taken from two different interactions (two supporting Kendon’s patterns and two that contrast with his claim). I then suggest that what distinguishes them and what matters for the participants’ organization of gaze behavior is not only who is speaking and who is listening but also the sequential organization of talk. 2.2.2.1 Supportive Evidence The two following examples represent supportive evidence for the gaze patterns described by Kendon. They are both dyadic interactions in which the participants are female undergraduate students and long time friends. In example 2.1, A and B are two women in their 20s. They are sitting on two couches drinking a cup of tea and chatting about their babysitting jobs. Before this extract, B told A that she does not know how to ask to be paid more by the family she is babysitting for because they are also friends of hers. A says that she is a friend of the main family she works for too but on the other hand it is also fair to be paid given that it is a job (see line 1). The gaze behavior of both participants strongly resembles the one proposed in Kendon’s pattern, in which the speaker alternates looking toward and looking away from the other participant while the recipient tends to look toward the speaker. 6 See chapters 3 and 4 and Rossano et al. (2009) for further evidence supporting this claim. 5656The literature suggests that there are few doubts about the causality of this phenomenon. Indeed, the general idea is that the different gaze patterns of the participants are due to the fact that somebody is speaking and somebody else is listening and the social roles of speaker and listener associated with them. The claim usually is that the occurrence of talk in a face-to-face interaction leads to the occurrence of these gaze patterns because of a conventionalized way5 of looking associated with a specific interactional role. The evidence for this claim comes from the fact that the participant role (of speaker and listener) is usually taken as the only independent variable and the occurrence of gaze as the dependent one. In other words, a participant would look at the other in a certain way and for a certain amount of time because s/he is speaking or because s/he is listening to the other participant, independently of the actions performed through the talk. Regulatory functions, in particular in relation to turn taking, have been claimed for the deployment of gaze (see chapters 1 and 3 for a review). For example looking toward a recipient or looking away could be used as a cue that a speaker is or is not going to select that person to speak next (see, e.g., Lerner, 2003). This would suggest that gaze also affects the exchange of speaker/hearer roles. As such, a straightforward causal chain that links how gaze is deployed to the simple occurrence of talk should be questioned. This leads to an empirical question: is the order of gaze behavior simply due to the participation role of speaker and listener, or does it depend on the details of what participants in those roles do and have to do in terms of social actions? If gaze is organized in relation to the participation role of speaker and listener, then speakership and recipiency would matter. There should be no difference if any distinction is made within the stretches of talk, in particular in terms of the sequential environment in which they are uttered and the social actions that are performed through their talk. If gaze is organized in relation to social action and what speakers and recipients do in each specific social action, then it should be possible to observe some systematic difference in people’s gaze patterns in relation to what they are doing with their talk. In what follows I test these hypotheses through a systematic analysis of empirical data. 5 Some evidence that specific gaze behavior might be conventionalized comes from recent work (Rossano, Brown, & Levinson, 2009) comparing speaker and recipient gaze behavior during questions in three different cultures that displays that there is some cultural variability, in particular in terms of how recipiency is enacted and sustained. However this work does not distinguish speaker and recipient in general, but only during question turns and therefore focuses on more specific action types than Kendon’s work. 572.2.2 Contrastive Evidence for Kendon’s Pattern A qualitative analysis of twelve hours of recordings of naturally occurring dyadic Italian data suggests caution in embracing Kendon’s gaze patterns as they stand. Although it is true that on average listeners look at speakers more than speakers look at listeners, they do not do so invariably. A systematic analysis of video data shows that the length and the patterns of gaze toward the other participant vary widely during a single conversation. At times a recipient does not look at all at the speaker, and a speaker sometimes holds her/his gaze toward a recipient even when the latter is not looking back or during gaps.6 This means that there could be a substantial amount of time in which Kendon’s patterns would not accurately describe the data and thus undermine prior claims (e.g. by Goodwin) that the recipient looking toward the speaker is a norm-governed phenomenon. In order to show some of the differences between these situations, I will present four extracts taken from two different interactions (two supporting Kendon’s patterns and two that contrast with his claim). I then suggest that what distinguishes them and what matters for the participants’ organization of gaze behavior is not only who is speaking and who is listening but also the sequential organization of talk. 2.2.2.1 Supportive Evidence The two following examples represent supportive evidence for the gaze patterns described by Kendon. They are both dyadic interactions in which the participants are female undergraduate students and long time friends. In example 2.1, A and B are two women in their 20s. They are sitting on two couches drinking a cup of tea and chatting about their babysitting jobs. Before this extract, B told A that she does not know how to ask to be paid more by the family she is babysitting for because they are also friends of hers. A says that she is a friend of the main family she works for too but on the other hand it is also fair to be paid given that it is a job (see line 1). The gaze behavior of both participants strongly resembles the one proposed in Kendon’s pattern, in which the speaker alternates looking toward and looking away from the other participant while the recipient tends to look toward the speaker. 6 See chapters 3 and 4 and Rossano et al. (2009) for further evidence supporting this claim. 5758(2.1) 2GSOFA_aumentato 23:32 01 A: .hh e’ giusto anche che me li diano .hh be.3s right also that me cl. give .hh it is also fair that they give them to me .hh 02 pero’ in effetti non saprei neanch’io co-<↑una volta but in effect not know.1s neither I wh- one time but actually I would not even know wh-<↑once 03 io ho aumentato I have raised I have raised ((the price)) 04 (0.5) (0.2) (0.7) 05 A: mi ricordo che: .hh ho aumentato e >ho detto tipo< me remember that have.1s raised and have.1s said like I remember tha:t .hh I raised ((the price)) and >I said like< 06 hh c(h)ioe’ non mi r(h)icordo bene le mod(h)alita’ that is not me remember well the modalities hh I m(h)ean I don’t r(h)emember well the det(h)ails 07 pero’ .h tipo uhm:: (0.6) era stato magari .h dopo l'estate but like uhm was.3s maybe after the summer but .h like uhm:: (0.6) it was maybe .h after summer 08 (0.4) quindi ero tornata (0.3) e >tipo ho detto< so was.1s come back and like have.1s said (0.4) so I had come back ((from holidays)) (0.3) and >like I said< 59 09 beh dai da quest’anno faccio otto euroho detto tipo< me remember that have.1s raised and have.1s said like I remember tha:t .hh I raised ((the price)) and >I said like< 06 hh c(h)ioe’ non mi r(h)icordo bene le mod(h)alita’ that is not me remember well the modalities hh I m(h)ean I don’t r(h)emember well the det(h)ails 07 pero’ .h tipo uhm:: (0.6) era stato magari .h dopo l'estate but like uhm was.3s maybe after the summer but .h like uhm:: (0.6) it was maybe .h after summer 08 (0.4) quindi ero tornata (0.3) e >tipo ho detto< so was.1s come back and like have.1s said (0.4) so I had come back ((from holidays)) (0.3) and >like I said< 59 09 beh dai da quest’anno faccio otto euro

Binocular interactions

Abstract

Extracted data

Binocular interactions

Abstract

Extracted data

Related items

Related items