Neural networks based EEG-Speech Models
This work addresses EEG-speech modeling for brain-computer interfaces, representing an incremental improvement with hybrid methods.
The paper tackles the problem of mapping imagined EEG signals to phonemes using neural networks, proposing three end-to-end models that outperform a baseline SVM method on EEG-speech classification and achieve comparable results to a deep belief network approach for binary classification.
In this paper, we propose an end-to-end neural network (NN) based EEG-speech (NES) modeling framework, in which three network structures are developed to map imagined EEG signals to phonemes. The proposed NES models incorporate a language model based EEG feature extraction layer, an acoustic feature mapping layer, and a restricted Boltzmann machine (RBM) based the feature learning layer. The NES models can jointly realize the representation of multichannel EEG signals and the projection of acoustic speech signals. Among three proposed NES models, two augmented networks utilize spoken EEG signals as either bias or gate information to strengthen the feature learning and translation of imagined EEG signals. Experimental results show that all three proposed NES models outperform the baseline support vector machine (SVM) method on EEG-speech classification. With respect to binary classification, our approach achieves comparable results relative to deep believe network approach.