Ideomotor simulation in manual action - Combining brain activity and eye-tracking measurements-

Mey S, Koester D, Schack T. Ideomotor simulation in manual action - Combining brain activity and eye-tracking measurements-. Presented at the KogWis , Darmstadt.Communication between humans and computers is getting more and more relevant for all fields of our daily life. It is important to make this communication as intuitive and applicable as possible for all classes of population, independent of personal skills and abilities. Because human-computer interfaces are already involved in daily life actions it is necessary to find a method which is usable in everyday life and not only under laboratory conditions. Compared to existing FRP (fixation- related potentials) studies, we combined simultaneously recorded EEG with eye-tracking data to distinguish between task-relevant and irrelevant objects in a more natural environment. The experiment was done in an electrically shielded room by the measure of a 64 electrode EEG system combined and synchronized with a remote eye-tracker. The task was presented on a computer screen in front of the participants. The experiment consist of two conditions. A counting condition in which participants had to search for particular object classes, and an exploration condition in which the participants only had to explore the scene. The crowded scenes consist of at least two object categories with some similar features like shape or functionality, and additional background objects. We predict a significant difference in the brain activity by observing task relevant and irrelevant objects. Because the attention should be focused on task relevant objects, especially a higher P300 for this objects is assumed. Our experiments show that dependent on the fact an object is important for a given situation, the neurocognitive signature is different by observing relevant than irrelevant objects. Coming from a visible search task, the results show that for analysing FRPs, because of the novelty effect, it is enough to have one to three fixations on an object. The FRP component effects seem to decrease with every fixation on the same object. The FRPs are averaged potentials after each fixation onset on an object of task relevant or irrelevant object categories. A closer look to the affected brain areas shows that positive effects occur in frontal central regions by observing relevant objects. This, and also the time window of 350-700ms after fixation onset, is an indicator for a P300 component, and particular concerning the region, a P3a effect. The event-related potential component P3a is known as an indicator related to attention and processing novel stimuli. As summary it is shown that there is a higher cognitive effect by evaluating task relevant stimuli compared to non-relevant stimuli shown in a P3a. These findings can be used as basis for interaction designs for brain-computer interfaces (e.g. online classification algorithms)