Deep Learning-based EEG Decoding for Reliable Brain-Computer Interface Systems

Abstract

I study EEG decoding algorithms to construct a reliable brain-computer interface (BCI) system. The BCI system provides direct interfaces between neural activities containing a user's intention and control signals for external devices. Since electroencephalography (EEG) signal has superior advantages of non-invasiveness, high temporal resolution, and low cost, it has been most widely used in BCI studies. Many researchers have developed hand-crafted feature and classification methods to increase the performance of classifying EEG signals. However, the EEG decoding performance of conventional algorithms is inferior to construct a reliable BCI system. In recent years, deep learning-based EEG decoding has shown the great possibility of increasing the performance of conventional machine learning techniques. Therefore, in this dissertation, three deep learning networks were proposed for improving EEG decoding performance. Firstly, multilevel feature fusion network is proposed to extract spatial information represented by brain functioning: functional segregation and global integration. The low-level deep feature represents segregation which refers to locally specialized information processing of the brain. On the other hand, high-level feature represents integration which refers to the global integration of information across the entire brain. Those features are aggregated by weighting factors, which determine the importance of each feature. The proposed network is evaluated on my dataset and public dataset, and it showed superior performance than conventional EEG decoding algorithms. Secondly, subject-invariant neural network is proposed to alleviate the subject variation of EEG signals. Inherently, EEG signals have subject variation caused by anatomical and physiological differences among subjects. Thus, it dramatically deteriorates the classification performance. Therefore, I propose a baseline correction module to estimate transformation parameters that map deep features to subject-invariant features using baseline-EEG signals. Also, subject-invariant loss aids the baseline correction module extracting subject-invariant feature by gathering the same class regardless of the subject and vice versa for different classes. Experimental results show that the proposed network has robust performance under a subject-independent classification scenario. Lastly, fNIRS-guided attention network is proposed to maximize the performance of hybrid BCI systems. Since EEG signal is vulnerable to movement artifacts and electrical noise, hybrid EEG-fNIRS BCI systems have been introduced to overcome the limitation of EEG-based BCI systems. The EEG and fNIRS signals are highly correlated with spatial manners because of neurovascular coupling, which explains that the neural activities make a subsequent change in cerebral blood flow. The proposed network extracts a joint representation for underlying physiological spatial correlation in order to maximize fusion performance. For extracting joint representation, fNIRS-guided attention layer is proposed where it extracts detailed neural information from EEG signals, while the fNIRS signals are only guided to the spatially important region for brain decoding. Then, a prediction method is proposed for alleviating performance deterioration at the beginning of a trial caused by the delayed hemodynamic response. The final prediction is achieved by the weighted sum of the prediction score of the EEG and fusion feature. The prediction weight is adaptively determined according to the importance of each feature. The experimental results show that the proposed fusion network significantly outperforms the EEG-based BCI system. Furthermore, it shows superior performance than the state-of-the-art fusion algorithms.