Neural Network for Eye Contact Detection

Eye contact is fundamental to understanding many psychology and cognitive science questions. Human gaze plays an important role in social interaction as it conveys a lot of information in face-to-face communication. Thus, the ability to use computational analysis to identify eye contact in a social interactive environment with high accuracy can facilitate many other research areas. The problem of understanding eye contact belongs to the bigger problem of understanding gaze. In previous studies, one approach is appearance-based gaze estimation, and most recent works [2][3][5][6][7][8] in this area consist of using a static camera to estimate gaze direction. By obtaining data from a variety of image datasets, the researchers train and develop models to estimate gaze direction [3]. Previous research provides two ways of collecting gaze and head pose pair data for gaze analysis [3][4]. The first is simply obtaining data from a set of calibrated cameras [3]. The second is building a 3D model of a face and generating real time 3D face images as data [4]. So far, the appearance-based gaze estimation approach shows a state-of-the-art performance in gaze estimation. In my study I proposes an eye contact behavior in a naturalistic social interaction by using a point-of-view (POV) camera [11], which is a 2-3 minutes play interaction between an adult examiner. In this setup, the child is interacting with a social partner, who wears glasses with an outward-facing camera in the bridge over those nose, aligned right between two eyes. Using video collected in this setup, we apply face detection [12] and facial landmark [12] analysis algorithm to obtain head pose and eye area images . Then we have these two pieces of information as the input to train a convolutional neural network (CNN). In the end, this CNN will serve as a classifier to detect eye contact between the child and the social partner.Undergraduat