A new method for detecting P300 signals by using deep learning: Hyperparameter tuning in high-dimensional space by minimizing nonconvex error function

Background: P300 signal detection is an essential problem in many fields of Brain-Computer Interface (BCI) systems. Although deep neural networks have almost ubiquitously used in P300 detection, in such networks, increasing the number of dimensions leads to growth ratio of saddle points to local minimums. This phenomenon results in slow convergence in deep neural network. Hyperparameter tuning is one of the approaches in deep learning, which leads to fast convergence because of its ability to find better local minimums. In this paper, a new adaptive hyperparameter tuning method is proposed to improve training of Convolutional Neural Networks (CNNs). Methods: The aim of this paper is to introduce a novel method to improve the performance of deep neural networks in P300 signal detection. To reach this purpose, the proposed method transferred the non-convex error function of CNN) into Lagranging paradigm, then, Newton and dual active set techniques are utilized for hyperparameter tuning in order to minimize error of objective function in high dimensional space of CNN. Results: The proposed method was implemented on MATLAB 2017 package and its performance was evaluated on dataset of Ecole Polytechnique Fédérale de Lausanne (EPFL) BCI group. The obtained results depicted that the proposed method detected the P300 signals with 95.34% classification accuracy in parallel with high True Positive Rate (i.e., 92.9%) and low False Positive Rate (i.e., 0.77%). Conclusions: To estimate the performance of the proposed method, the achieved results were compared with the results of Naive Hyperparameter (NHP) tuning method. The comparisons depicted the superiority of the proposed method against its alternative, in such way that the best accuracy by using the proposed method was 6.44%, better than the accuracy of the alternative method