Instrumentation for Automatic Monitoring of Affective State in Human-Computer Interaction

In the modern world we interact with computers for many activities, such as working and learning. Computers are great tools for these tasks. However, some times our interaction with computers is degraded because they are inflexible and unaware of our own level of comfort in that interaction. The emerging field of ‘Affective Computing’ strives to give computers an awareness of the affective state of the user, and the ability to adjust to it. This research aims at sensing and recognizing typical negative emotional experiences, especially ‘frustration’, which may arise when people interact with computers. An integrated hardware – software setup has been developed to achieve real-time assessment of the affective status of a computer user. Three physiological signals: Blood Volume Pulse (BVP), Galvanic Skin Response (GSR) and Pupil Diameter (PD) were collected and synchronized from each computer user participating in our experiments. Several signal processing approaches were then applied to recognize the ‘frustration’ state of computer users. Preliminary results indicate that there exists a strong correlation between changes in these physiological signals and the shift in the emotional states of a computer user when a frustrating stimulus is applied to the interaction environment