Direct brain-computer communication through scalp recorded EEG signals

Scalp recorded electroencephalogram signals (EEG) reflect the combined synaptic and axonal activity of groups of neurons. In addition to their clinical applications, EEG signals can be used as support for direct brain-computer communication devices (Brain-Computer Interfaces BCIs). Indeed, during the performance of mental activities, EEG patterns that characterize them emerge. If actions executed by the BCI, are associated with classes of patterns resulting from mental activities that do not involve any physical effort, communication by means of thoughts is achieved. The subject operates the BCI by performing mental activities which are recognized by the BCI through comparison with recognition models that are set up during a training phase. In this thesis we consider a 2D object positioning application in a computer-rendered environment (CRE) that is operated with four mental activities (controlling MAs). BCI operation is asynchronous, namely the system is always active and reacts only when it recognizes any of the controlling MIAs. The BCI analyzes segments of EEG (EEG-trials) and executes actions on the CRE in accordance with a set of rules (action rules) adapted to the subject controlling skills. EEG signals have small amplitudes and are therefore sensitive to external electromagnetic perturbations. In addition, subject-generated artifacts (ocular and muscular) can hinder BCI operation and even lead to misleading conclusions regarding the real controlling skills of a subject. Thus, it is especially important to remove external perturbations and detect subject-generated artifacts. External perturbations are removed using established signal processing techniques and artifacts are detected through a singular event detection algorithm based on kernel methods. The detection parameters are calibrated at the beginning of each experimental session through an interactive procedure. Whenever an artifact is detected in an EEG-trial the BCI notifies the subject by executing a special action. Features that are relevant for the recognition of the controlling MIAs are extracted from EEG-trials (free of artifacts) through the statistical analysis of their time, frequency, and phase properties. Since a complete analysis covering all these aspects, would result in a very large number of features, various hypotheses on the nature of EEG are considered in order to reduce the number of needed features. Features are grouped into feature vectors that are used to build the recognition models using machine learning concepts. From a machine learning point of view, low dimensional feature vectors are preferred as they reduce the risk of over-fitting. Recognition models are built based on statistical learning theory and kernel methods. The advantage of these methods resides in their high recognition accuracy and flexibility. A particular requirement of BCI systems is to continuously adapt to possible EEG changes resulting from external factors or subject adaptation to the BCI. This requirement is fulfilled by means of an online learning framework that makes the parameters of the recognition models easily updatable in a computationally efficient way. After the completion of a series of training sessions, the feature extraction methods are chosen (according to an optimality criterion based on the recognition error), the initial recognition models are built for each controlling MA, and the action rules are set. In these sessions, the subject is asked to perform the controlling MAs, in accordance to a training protocol which determines the training schedule. In posterior training sessions, the BCI provides feedback indicating the subject how well the asked MA was recognized by the BCI. Thus, the subject can modulate his brain activity so as to obtain positive feedback. Furthermore, at the end of each session the BCI updates its recognition models. Such updating is straightforward as the recognition models can be dynamically updated, i.e. their parameters can be updated as new training data becomes available while progressively forgetting the contribution of old data. Because of the adaptation of the recognition models, the action rules must be adapted as well. This is achieved by considering, in the definition of the action rules, variables that change along with the recognition model parameters. The training schedule is decided based on the recognition error associated with each controlling MA, thus those MAs with large recognition errors are trained more often. The BCI developed in this thesis was validated by experiments on six subjects who participated in nine training sessions. The first three training sessions served to select the feature extraction methods, build the initial recognition models, and set the action rules. In the last six sessions, in addition to the training with feedback, positioning tests were carried out to measure the controlling skills acquired by them during each session. The evaluation was done following two criteria, namely the computation of the theoretical information transfer rate using estimates of the average recognition errors over the controlling MAs, and an experimental measure of the information transfer rate corresponding to the positioning tests. The latter has the advantage of corresponding to a real controlling situation and consequently reflects more closely the actual controlling skills of a subject. Both information transfer rates increased during the last six sessions and reached an average, over subjects of 126 and 25 bits per minute respectively